HIPerPaper: Pen and Paper interaction with high resolution wall displays

HIPerPaper is a new interface that provides pen and paper interaction with a large wall display. This kind of interaction is promising for several reasons. Paper is lightweight and flexible. Printing on it allows for myriad types of interfaces, from simple commands and menus to demarcation of regions to provide special functions such as zooming or panning. Paper can be customized easily to suit various tasks and interaction needs, then discarded after use. Paper is mobile, meaning that users can freely navigate in the space in front of the dis- play without being constrained by the means of interaction.

The HIPerPaper interface is based on Anoto technology and the iPaper framework. It exploits a novel architecture for handling multi-user, multi-device, multi-channel and multi- modal interactions. The main interface is represented by a scaled version of the HIPerSpace wall printed on paper. Multiple users can interact with the wall us- ing multiple paper-based interfaces and digital pens provided with traditional or non-marking pen tips. Different from existing systems based on interactive paper and wall displays, our design goal is to encourage the main focus of attention to be on the high-resolution wall display. To give users simple locational references, the paper interface includes a scaled outline representation of the tiled displays.

Movements of the pen in the upper region of the main paper interface support coarse navigation of the wall display. The lower region provides multiscale focused access to a particular part of the whole display (i.e., the specific area the user is interacting with on the large overview). After roughly positioning their cursor on the display through the main interface, users can switch to the focused interface for fine positioning.

The digital pen is connected over Bluetooth to the HIPerSpace wall and continuously transmits timestamped (x,y) coordinates to the underlying HIPerPaper software. Besides a direct mapping to the coordinate space of the wall display, this allows exploiting derived information, e.g., speed and acceleration, as a basis for new forms of interaction. Moreover, the pen also records the pressure that the user applies when interacting with the paper interface, the rotation of the pen, and its tilt with respect to a vertical position.

We have explored pen and paper techniques for selecting, moving, scaling, and rotating objects as well as gestures for panning and zooming and for issuing commands to the appli- cations running on the HIPerSpace wall. We use basic (x,y) position tracking of the pen to move a pointer on the display in the default navigation mode. In order to select a specific object and move it around, scale it, rotate it, as well as to re- lease it, we introduced a mechanism for mode switching relying on pen pressure: users touch the paper interface lightly to receive visual feedback regarding their location on the wall, while a firm touch with the pen tip triggers the activation of particular modes on the interface (such as ‘select’), which will be deactivated on the next pen-up event. Despite the low sensitivity of the pressure sensor, it can successfully be used for detecting such an instantaneous change in pressure. Preliminary user testing indicates this is a feasible mechanism for mode switching.

In addition to the main paper interface, the current system in- corporates a set of paper-based buttons and widgets. Touch- ing or moving the pen tip on these widgets issues commands to the application running on the wall. The paper buttons and widgets can be printed directly on the main paper interface, printed on separate papers sheets, or deployed in the form of removable stickers to be placed in convenient locations such as on a clipboard, a table, or even on the face of a watch or other body location. These widgets are easily customizable to suit any given application.

Since scaling is one of the most valuable functions on high- resolution wall displays, we introduced a general paper-based scaling interface based on circular interactions with a widget. Similar to scrolling through music on an Apple iPod, users can scale an object on the HIPerSpace wall by first selecting it and then moving circularly, adjusting their path and speed of movement for fine or coarse scaling.


Related Publications: