A 16X16 cross-correlation matrix was computed, showing the correlation between every pair of probes based on how the pilots responded to the probes. This matrix shows that the probes fall neatly into two major groups. One group includes all of the probes that reflect positive attitudes toward automation while the other includes the probes that reflect negative attitudes toward automation.
One advantage of the raw cross correlation matrix that is lost in the attitude probe clustering described below is that the matrix indicates the negative correlations among specific pairs of probes. For example, it shows that the largest negative correlation is between the probes ``I always know what mode the AP/FD system is in.'' and ``There are still things about the airplane that surprise me.''
For the next set of analyses, the 16 attitude probes were mapped as points in a 547-dimensional space (547 pilots responded to all of the probes). Euclidean distances in this space were computed for every pair of probes. This yields a 16X16 matrix of distances. A complete-linkage cluster analysis was performed on this matrix. The cluster tree is shown in figure 7.
This cluster analysis clearly shows the two major clusters that were identified in the cross correlation matrix. A principal components analysis was then performed on this distance matrix. The first three eigenvalues account for .34, .12, and .09 of the variance for a cumulative total of 56% of the variance. Figure 8 shows the probes in spaces defined by the first/second and first/third principal components. As can be seen in figure 8, the first principal component maps fairly nicely onto the distinction between the probes that reflect positive attitudes toward automation and those that reflect negative attitudes. This comes as no surprise since we, and all of the other researchers using this technique [10], have tried to be sure that a range of positive and negative attitudes were represented in the probes.
The second principal component is more interesting. It appears to encode some aspect of the authority of the pilot. At the high end are probes that concern the direct relationship of the pilot to the behavior of the airplane. On the positive side automation helps the pilot do his job, on the negative side, it may cause him to lose his flying skills and it may surprise him. In the middle are probes that concern what is called ``supervisory control'' [7,1]. These probes concern the mediating role of automation between the pilot and the airplane. On the positive side automation frees the pilot to manage the flight, and keeps him ahead of the airplane. On the negative side, it does not reduce workload because there is more to monitor, and a pilot may spend more time setting up the system than he would hand-flying the same situation. At the low end of this principal component are probes that concern the context for the use of automation. On the positive side, pilots make fewer errors with automation. On the negative side, automation makes it easier to bust an altitude and some pilots use it because of company pressure.
Sherman [8] discusses the underlying domains that the probes in the University of Texas Aviation Automation Survey were designed to tap. The probes in the survey used in the present study were intended to tap pilots' understanding of the automation and of their own practices with automation. We did not choose the probes in order to tap issues concerning the pilot's authority, but the principal components analysis tells us that this is an important dimension for the pilots participating in our study.
It is not clear what the third principal component encodes.