1997 Reports

Automated Visual Discourse Synthesis: Coherence, Versatility, and Interactivity

Zhou, Michelle X.

In this proposal, we present comprehensive and systematic approaches of building systems that can automatically generate coherent visual discourse for interactive environments. A visual discourse refers to a series of connected visual displays. A coherent visual discourse requires smooth transitions between displays, consistent designs within and among displays, and effective integration of various components. While our main research goal is to develop approaches to automatically create coherent, versatile, and interactive visual discourse, we also emphasize integrating these approaches into a general framework to provide a reference model in which a specific system is considered an instantiation of the framework. In other words, the framework should consist of a knowledge base, an inference engine, a visual realizer, and an interaction handler. As a consequence, not only can a general framework serve as a template from which a specific generation system can be instantiated,but the framework also can be used as a base for comparing or evaluating different systems. We concentrate on the basic issues involved in developing comprehensive and systematic approaches to ensure a visual discourse's coherence, versatility, and interactivity. In particular, To ensure coherence, we have established a set of comprehensive design criteria to measure both expressiveness and effectiveness of a visual discourse. To provide versatility, we address the design of visual discourse for heterogeneous information. Within such discussions, heterogeneous information refers to both quantitative and qualitative, or static and dynamic information. In addition, we are also concerned with characterizing and employing different visual media and a wide variety of visual techniques. To support interaction, we integrate many conventional user interface metaphors and styles into visual discourse design and explore reactive planning approaches to provide proper response to user interactions. To establish the framework, we identify various knowledge sources and determine effective knowledge representation paradigms in constructing the knowledge base. We emphasize the efficiency, usability, and flexibility issues in modeling the inference engine. We are concerned with portability and parallelization issues in building the visual realizer, and we also take into account interaction capabilities for interactive environments. We describe a system called IMPROVISE (Illustrative Metaphor Production in Reactive Object-oriented VISual Environments) that serves as a proof-of-concept prototype. IMPROVISE is built based on our framework, aiming to automatically generate coherent visual discourse for various application domains in interactive environments. IMPROVISE has been used in two testbed application domains to demonstrate its generality and flexibility. Examples from both domains will be given to illustrate IMPROVISE's generation process and to identify the future research areas.