Title of Project:

kdi: a distributed cognition approach to designing digital work materials for collaborative work places.

Principal investigator: Edwin Hutchins, Professor, Cognitive Science

Co-investigators:

James Hollan, Professor, Cognitive Science

David Kirsh, Associate Professor, Cognitive Science

1. Facilities:

The study will be carried out in the human-computer interaction laboratory of the department of cognitive science at UCSD, and depending on negotiations with airlines, in the training facilities and aircraft cockpits of one or more major US-based commercial airline.

2. Duration of the Study:

Three calendar years. 07/01/99 to 06/30/01

3. Specific Aims:

  1. to document the use of annotations in an undergraduate cognitive science course, a scientific laboratory, and airline cockpits and
  2. to design and implement computer-based tools to support annotation activities in the settings described above.

4. Background and Significance:

To ensure that new workplaces and digital work materials serve human needs requires both a different theoretical base and an integration of theory with ethnographic and experimental methods. We propose a research framework based on distributed cognition. Distributed cognition provides a reorientation of how we think about designing and supporting human computer interaction. It permits one to move the boundary of the cognitive unit of analysis out beyond the skin of the individual to include the material and social environment as components of a larger cognitive system. This focuses attention on the processes by which people take advantage of both internal and external resources to organize their actions. For the design of workplaces, this means that work materials are more than stimuli for disembodied cognitive systems. Work materials become elements of the cognitive system itself, and cognition becomes an emergent property of the interactions among people and work materials.

We will apply our integrated approach to developing a distributed cognition based theory of annotation and explore a range of application domains: collaborative scientific research, education, and commercial aviation. In each domain, we will conduct ethnographic and experimental studies, design and implement digital work materials and analyze their effectiveness. In addition, to help us develop and communicate our distributed cognition perspective, we propose: (1) a series of prototypes of history-enriched work materials that support collection and selective sharing of personnel and group activity histories and annotations, (2) annotation facilities to assist collaborations, and (3) information visualization facilities to exploit rich histories of interactions and provide effective access to annotations.

6. Research Design and Methods:

The proposed research will collect survey, interview and observational data. Interviews will be recorded on audio or video tape. Observations of annotation activities may be recorded on video tape.

Subjects will participate in the project in four different contexts:

    1. being observed by a researcher while engaged in real world activities.
    2. participating in interviews with a researcher. The interview data will be collected in unstructured interview formats. Interviews will center on annotation practices. Participants will be asked to describe the sorts of annotations they use and how they make use of annotations in their work and everyday activities. Examples of interview questions are presented in the Appendix.
    3. completing surveys concerning the use of annotations in normal work activities. It is difficult to know which annotation phenomena will be of most interest before observations have begun. The surveys will be developed for each domain after some observational data have been collected. Once the surveys are developed, they will be submitted to the IRB for approval before they are used. The surveys will include questions like the ones listed in the Appendix.
    4. participating in experimental evaluations of specific digital annotation tools. Subjects will be asked to carry out typical annotation tasks using digital annotation tools. Most of these tools will be evaluated in the UCSD HCI laboratory. Some tools may be portable and some subjects may be asked to take the tools with them out of the lab to use them in their everyday activities.

The survey data will be used to compile an inventory of annotation practices in each work domain. This inventory will help us understand the range of annotation practices in each domain and may suggest topics to focus on in the interviews and observations.

Recordings of the interviews will be analyzed for their descriptions of annotation practices. These data will provide insight into the use of annotations and the strengths and weakness of the media used to make annotations.

People are sometimes unable to provide accurate accounts of their own behavior. For this reason, observations will be made of subjects as they go about their normal activities. For the members of the scientific lab, this means the creation and modification of documents and the generation of ideas. The students will be observed as they work alone and with each other on the class assignments. Some of the data collected on students will be captured automatically by the digital tools they use to complete the assignments.

The experimental evaluations of digital annotation tools will require subjects to make use of the tools and to be observed and have their behavior recorded as they make use of the tools. A typical evaluation session might involve a subject using a new style of computer-based text editor. The subject will be asked to use the editor to make changes to a document and to annotate the changes made using the digital tools provided.

7. Human Subjects.

The scientist subjects will be the members of the UCSD Human/Computer Interaction laboratory (including the principal investigators on this research project.) Participation in the study as a subject will be voluntary for all members of the laboratory.

Student subjects will be recruited from at least two courses taught in the Cognitive Science department at UCSD. One class will be an introductory computing class, and the other will be a course that is taught on-line.

No effort will be made to either recruit or exclude any participant on the basis of race or gender as these criteria are believed to be completely irrelevant to the issues being investigated.

8. Informed Consent

Consent will be obtained from each participant by in person by members of the research team. The informed consent forms for each context of participation are appended to this application. The forms for contexts that may include recordings of audio or video data include a statement that the subject may have the recording stopped at any time and may have the recording erased. These forms will be presented to prospective participants who will be asked to read, understand, and sign the form before continuing.

9. Potential Risks

The surveys, interviews, and experiments expose the subjects to negligible risk. Observations of scientists and students at work exposes the subjects to negligible risk.

10. Risk Management

All observations will be treated as items of informant/ethnographer confidentiality.

11. Potential Benefits

The participants in this research will not benefit directly from the study except to the extent that they come to a better understanding of their own annotation activities or improve their annotation practices. The research to be conducted here is part of a wider effort to understand the role of annotation in knowledge work and in coordinating the activities of multiple actors in complex work settings. The digital annotation tools developed in this project promise to contribute to increased productivity in knowledge work settings.

12. Risk/Benefit ratio

The risks to the participants in this study are small. The likelihood of any harm coming to a participant as a consequence of participation is very small. The potential benefits to society in general are significant.

  1. Bibliography

No unusual procedures are to be used in the protocol. The techniques of ethnographic interviewing and ethnographic surveys are described in Werner and Schoepfel (1987). Techniques of ethnographic video analysis are described in Hutchins (1995).

 

Appendix 1: Example interview questions:

As part of the interview, researchers may ask subjects to show examples of documents they have annotated, or to discuss the annotations they have made or made use of in their work environment. Interview questions will focus on the creation and use of these annotations. The actual content of later questions in the interview will depend on how subjects respond to the initial questions. For example, a subject may introduce a form of annotation that merits investigation. The following are examples of questions that will be used to begin the interviews.