An%20Agent-based%20System%20for%20Capturing%20and%20Indexing%20Software%20Design%20Meetings

1
An Agent-based System for Capturing and Indexing
Software Design Meetings

• Tracy Hammond, Design Rationale Group
• Krzysztof Gajos, Intelligent Room
• Randall Davis, Design Rationale Group
• Howard Shrobe, Intelligent Room

2
Software Design Meetings
Sketch Diagrams

• Discuss Design

3
Capturing Design History

• Videotape the meeting
• Videotape the white board
• Question
• How do you index videotape?

4
Solution Tahuti, Sketch Recognition of Design
Diagrams

• UML Diagrams Agents
• Natural Shape Based
• Projected Screen / Digital Pen
• Agent Arch. Many Computers, Devices

5
Tahuti UML Sketch Recognition

• Sketch Naturally
• Natural Editing Capabilities
• Code Generation Facilities
• Event Indexing

6
Tahuti UML Sketch Recognition

• Sketch Naturally
• Natural Editing Capabilities
• Code Generation Facilities
• Event Indexing

7
Tahuti UML Sketch Recognition

• Sketch Naturally
• Natural Editing Capabilities
• Code Generation Facilities
• Event Indexing

8
Recognizable Shapes

• UML Class Diagram
• General Class
• Interface Class
• Interface Association
• Dependency Association
• Inheritance Association
• Aggregation Association
• Intelligent Room Agents
• Agent Class
• Speech Grammar

9
Sketched Diagram
10
Interpreted Diagram
11
Agent Abstraction
Underlying Object Oriented Design
Metaglue AgentAbstraction
12
Relies On Abstraction
Metaglue Agent Abstraction
Underlying Object Oriented Design
13
Speech Grammar
Metaglue Agent Abstraction
Underlying Object Oriented Design
14
Event Indexing

• Events
• Addition, Deletion, Movement of General Class,
  Interface Class, Agent Class, Sketch Grammar,
  Association, Unrecognized Stroke
• Text update
• Undo, Redo
• Use to Index Videotape
• What was the meeting discussion when this class
  was added?

15
Auto-summarization of Design History by Event
Ranking

• Final Design 12
• User Marked Significant Event 11
• Creation of an Existing Agent 10
• Creation of an Existing General or Interface
  Class 9
• Creation of an Existing Speech Grammar 8
• Creation of an Existing Association 7
• Creation of an Existing Unrecognized Stroke 6
• Text Update 5
• Movement 4
• Creation of Deleted Object 3
• Deletion of an Object 2
• Undo/Redo 1

16
Auto-summarization of Design History by Event
Ranking
Draw Order - Rank
17
Design History
Up to next event gt floor(lowest chosen rank)
Screen shot 1
Screen shot 2
18
Tahuti Architecture
19
Metaglue Agent Architecture

• Support for synchronous and asynchronous
  communication among distributed agents
• Mechanisms for resource discovery and management
• Robust recovery mechanisms for failed components
• Built-in persistent storage
• Support for multimodal interactions through
  speech, gesture, and graphical user interfaces.

http//www.ai.mit.edu/projects/iroom/metaglue
20
Impact and Current Use

• UserStudy
• Users Prefer Tahuti for drawing and editing UML
  diagrams over MS Paint or Rationale Rose
• Deployment
• Used to teach 65 students Object Oriented
  Programming at Columbia University
• Used in the MIT AI Lab Intelligent Room for use
  in software design meetings. Currently extending
  to add speech interface.

21
Related Work

• UML Sketch Recognition
• Hse Wizard of Oz prefer to mouse and palette
• Ideogramic UML, graffiti-like implementation
• Queens Unversity, based on stroke length
• UML for Designing Agent Based Systems
• Odell, Parunak and Bauer
• Bergenti and Poggi
• Indexing Audio-Visual Material
• Brunelli, Mich, and Modena
• Sketch Recognition to Locate Video Moment
• Kato, Kurita, Otsu, and Hirata
• Cho and Yoo
• Jacobs, Finkelstein, and Salesin

22
Future Work

• Expansion of system
• Multiplicity Relationships
• More Symbols
• Deeper layers of software design
• Sequence Diagrams
• Flow Charts
• Generate More Code
• Code Editor

23
Contributions

• Agent-based system for capturing and indexing
  software design meetings
• Natural Sketch Recognition
• Supports Multi-Agent Software Design
• Indexes Design Meeting Videos

24
Tahuti God of Wisdom

• Tahuti is the Egyptian god of wisdom.
• He always carried a pen and scrolls upon which he
  recorded all things.
• A.k.a. Thoth