Flatland is a visualization/virtual reality application development environment, created at the University of New Mexico (http://www.ahpcc.unm.edu/homunculus). It allows software authors to construct and users to interact with arbitrarily complex graphical and aural representations of data and systems. The system is described in more detail in Box 1. The end result is a virtual-reality immersive environment with sight and sound, in which students using joywands and virtual controls can interact with computer-generated learning scenarios that respond logically to user interaction. Virtual patients can be simulated in any of several circumstances, with any imaginable disease or injury. The activities of a participant can be monitored by faculty and other students for educational and instructional purposes.
Flatland: Technical Details
Flatland is written in C/C++ and uses the standard OpenGL graphics language to produce all graphics. In addition, Flatland uses the standard GLUT library for window, mouse, joywand, and keyboard management. Flatland is object oriented, multithreaded, and uses dynamically loaded libraries to build user applications in the virtual environment, and runs under Linux and Irix operating systems. At the core of Flatland is an open, custom, transformation graph data structure that maintains and potentially animates the geometric relationships between the objects contained in the graph. Graph objects contain all of the information necessary to draw, sound, and control the entity represented by the object. The transformation graph is one part of a higher-level structure referred to in Flatland as a universe. The universe contains the transformation graph, a flat database of objects in the graph, and a reference to the graph vertex that is currently acting as the root of a hierarchically organized tree. This root is usually the graphical camera viewpoint.
Flatland is intrinsically multithreaded, allowing the system to make use of computer systems with multiprocessors and shared memory. The main thread spawns an OpenGL graphics thread, a Flatland sound thread, and a real-time tracker thread. The optional tracker facilitates the use of 3D interaction metaphors with their applications and to use head tracking and 3D joywands or wands. An application in the context of Flatland is a relatively self-contained collection of objects, functions, and data that can be dynamically loaded (and unloaded) into the graph of an environment instantaneously. An application is responsible for creating and attaching its objects to the graph, and for supplying all object functionality. An application is added to Flatland through the use of a configuration file. This structured file is read and parsed when Flatland starts, and contains the name and location of the libraries that have been created for the application, as well as a formal list of parameters and an arbitrary set of arguments for the application.
In Flatland, graphics and sound are treated symmetrically. Each object in the graph contains, among other things, a draw function and a sound function. The draw function contains or calls all of the code to draw and animate the graphics that represents the object. From an author's perspective, all object graphics are based on and drawn in a local coordinate system. Other structures in the graph handle the placement and orientation of the object's model coordinate relative to other objects in the graph and subsequently the camera. The sound function within an object contains all of the calls or code to make sounds that represents that object. Flatland maintains a library of sound function calls that are designed to resemble OpenGL. Wave sound files are treated like OpenGL display lists and are called sound lists. In addition to opening sound lists, functions exist that allow the author to control the starting, looping, stopping, volume, and 3D location of the sound. All sound is emitted in Flatland from point sources in the 3D space. The author specifies the location of the sounds in the same model coordinate system used for the graphics.
Although position-tracking technology is not generally available on computers today, Flatland is designed to make use of these. A tracker is a multiple degree of freedom measurement device that can, in real-time, monitor the position and/or orientation of multiple receiver devices in space, relative to a transmitter device of some sort. As such, Flatland launches a tracker thread to sample the available tracker information and make it available for use by applications. In the standard Flatland configuration, trackers are used to locate hand-held wands and to track the position of the user's head. Head position and orientation is needed in cases that involve the use of head mounted displays or stereo shutter glasses.
User interaction is a central component of Flatland, and as such, each object is controllable in arbitrary ways defined by the designer. Currently, there are four possible methods for the control of objects: (1) GLUT pop-up menus in the main viewer window, (2) the console keyboard, (3) Flatland 2D control panels either in the environment or separate windows, and (4) external systems or simulations. In the future, there will also be available 3D menus and controls in the virtual environment and voice recognition.
An array of controls may be defined when an object is coded by the designer. These controls are managed by Flatland and can be exercised through a designer-defined function that is evoked when either the keystroke is made or a menu item is selected. This function may be arbitrarily complex and may affect objects other than the owner of this control. The control functions associated with objects are the preferred method to change any internal states or data of the object. The mouse and keyboard interactions are provided through the GLUT libraries and a custom 2D widget library. The latter is available to the designer for the creation of standard 2D control panel windows. Finally, external systems may control an object, for example, through a threaded simulation, serial communication (trackers), or Unix sockets to another process.