Yesterday I did the last step in my Media Informatics master endeavor: I presented my master thesis. After quite a few months of development on the Microsoft Surface, the application that resulted was a good start for a (possible) commercial product.
We started from the idea that a multitouch table device would offer more space, possibilities and opportunities to improve the collaboration and communication in the brainstorming phase of any project. This evolved very much during the development and conceiving the master thesis. It became a tool which not only supported and enhanced collaboration, but also, to name a few:
- offered clues gathered from Twitter and Flickr images,
- free drawing and sketching,
- text input using the virtual keyboard,
- audio recognition features,
- save / load of session,
- integration with a CRM, serving as repository;
- user-friendly navigation in sessions using also the CRM repository interface.
This article presents an overview of my Master thesis. It summarizes the work by presenting the contributions to the research field and also the future work based on the already created prototype. The Breiny application is meant to support the Brainstorming process by using computers in a modern way – Natural User Interface. After creating the prototype, a User Evaluation was used to prove or disprove the theoretical hypotheses.
In the Related work is presented the relevant work on which my master thesis relyies on and the systems that are the basis for comparison with the developed prototype. It considers several already available collaboration and brainstorming systems and compares their features with the proposed Framework for analysis. As a conclusion of the chapter, none of the considered systems fulfill all the requirements from the proposed Framework. (more…)
The Users for the first session were 5 males and 1 female, with age ranging from 27 to 49. For the second, there were 4 males with age ranging from 26 to 52, with significant Brainstorming sessions experience, using Mind-mapping software, whiteboard computer based systems. The room setup consisted from a large space around Surface, with chairs surrounding it. All users were in the Contributor role and they had the opportunity to switch to Session Master any time. The application was started, with no Brainstorming Ideas on the tabletop.
The Evaluation session started with the Prototype presentation. This was targeted only on the User Interfaces, explaining how and which actions can be performed with each area of interest (UserControls and Main SurfaceWindow from Implementation Chapter). The users were encouraged to experience the interface and ask questions during the presentation, in order to understand better the interaction with the prototype.
Following the presentation phase was designed as a game, in order to exercise the prototype’s interface and evaluate the system and its features. The game was built around a imaginary, newly created, travel agency. The participants should create the company’s portfolio in a Brainstorming session, composed from the four phases (scenarios) presented in Typical use cases. They were disclosed one by one, in order to not affect the brainstorming process.
The theoretical evaluation of the thesis work starts with by assessing in which measure the research questions were answered by the proposed concept. Then, the analysis will focus on the comparison of the implemented prototype’s features with the ones presented in Related work, in order to see which ones were implemented and how the prototype’s features map on the array of desired ones. Finally, this chapter will conclude with a matrix in which the implemented functional requirement are marked, giving a final overview on the capabilities of the prototype to fulfill all the requirements stated in Requirement elicitation. (more…)
The prototype’s architecture implementation
The architecture was already presented in the Prototype application architecture and now we’ll see the actual implementation. The prototype implementation tries to follow as close as possible the recommended architectural patterns and guidelines. To support the argumentation, a class diagram will be presented to give a visual orientation through the prototype implementation presentation.
The built prototype supporting Brainstorming Sessions on Microsoft Surface, further named Breiny, was developed as part of this thesis and uses all the above framework features. Thus, it is creating the basis for future work by being flexible in development yet fully packed with features.
Breiny’s class diagram
This section will present the Breiny application’s user interface and the available interactions with it, features which will be evaluated both theoretically and by the users. It is composed from a main SurfaceWindow named “Main screen” in figure below, on which all the interaction occurs. Following the requirements for extensibility, modularity and flexibility, the subsequent visible objects were separated into several user controls.
In the following subsections the main screen will be presented first along with its specific interactions, then the discussion will continue with the available item types and their particular available features.
The main window acts as an aggregator for all the interactions, providing also support for the other controls and their interactions. It offers access to the main contextual menu, to the tag recognition and visualization, offers the data context through binding against the Session singleton and, because binding is propagated to the children controls, allows subsequent binding context to all of them.
In the figure from the left, the Breiny application was started and a new Brainstorming session is created. In this specific case the Brainstorming Items Repository didn’t already contain Brainstorming Items, thus none of these appeared on the application screen. But the users may have prior access to other devices that can interface with this thesis proposed system (iPhone, iPad or Adobe Air client) as mentioned in Architecture chapter and send they initial suggestions. If so, their Brainstorming items will show on the newly created Brainstorming Session. (more…)
My thesis proposes several approaches as result of the research questions. Their validity of each approach should be verified by allowing users to interact with a prototype built to fulfill all the proposed solutions. Hence, the final users will prove the correctness of this thesis approaches. The implemented prototype will be a brainstorming supporting application.
Because my thesis studies brainstorming on a tabletop, I will choose the Microsoft Surface as the underlying hardware because it supports multitouch and multiuser experience. As development environment I will use Visual Studio 2010 and the programming will be done in C# with User Interface (UI) built in Windows Presentation Foundation (WPF) framework.
In this article I will start with a brief introduction in general details of the baseline system and the technologies implied in the creation of the prototype system. I will continue with the presentation of the architecture of the actual implemented system, which tries to follow the proposed one; the differences between the proposed system and the actual implementation will be underlined and the choices motivated. The article will continue with the class diagrams of the prototype and will end with the presentation of the User Interface, allowing a clear view over the features and the way the prototype performs in achieving the goals of the previous concept presentation. (more…)
This article presents the system architecture and the underlying decisions that were taken to achieve the above requirements. The system requires a repository for sessions, a repository for Brainstorming items and a native Windows application for achieving the baseline of desired features. Thus, a distributed client – server architecture is considered, exemplified below.
Session repository architecture
The above schema represents the Session repository architecture. Its main purpose is to offer two main features:
- communication over network
- means for storing, retrieving and sharing the sessions
The server and the native application can be mixed together, but with limited sharing options. Because of this reason, we will use an already created system which not only that offers through its network module of API (Application Programming Interface) both means of storing and retrieving of the session data, but also allows advanced features of sharing the data in an already familiar interface.
Brainstorming items repository architecture
The architecture for the Brainstorming items repository is similar with the one for Session repository, but the most important feature it brings is the possibility to interface with other devices, not only with the tabletop.
Another reason for creating this secondary architecture is the fact that it easily enables an offline scenario, in which the repository server is not available because of the network availability issues. Nevertheless, the mobile devices can connect to this repository through creating a local wireless network without the need to be connected to the internet. Another advantage is that, because it’s small complexity, can (but it is not required to) be installed on the same tabletop. The offline scenario is not covered in this paper, but rather proposed as future work.
We already covered the proposed general usage scenario and particular use cases of a Brainstorming system. In order to evaluate their implementation, in this section the application requirements will be presented. They are split into functional and non-functional ones. The first refers to all the features needed to be implemented, while the second category refers to performance, extensibility and scalability of the system.
|FR.1||The system will allow multiple users to interact with the system simultaneously|
|FR.2||The system will allow multiple devices to interact with the Microsoft Surface simultaneously|
|FR.3||The users will be able to send ideas to the tabletop using mobile clients|
|FR.4||The users will be able to generate and input ideas in the system|
|FR.5||The Contributors will be able to input Brainstorming items directly on the tabletop|
|FR.6||The Contributors will be able to Group Brainstorming items|
|FR.7||The Contributors will be able to Edit Brainstorming items|
|FR.8||The Contributors will be able to Relate Brainstorming items|
|FR.9||The Contributors will be able to Print Screen|
|FR.10||The Contributors will be able to promote Twitter or Flickr items to Brainstorming items|
|FR.11||The Session Master will be able to login into its BSCW account|
|FR.12||The Session Master will be able to Save a Brainstorming Session|
|FR.13||The Session Master will be able to Load a Brainstorming Session|
|FR.14||The Session Master will be able to set Session Goal|
|FR.15||The Session Master will be able to start the application using his Tag|
|FR.16||The Session Master will be able to search on Flickr using a keyword|
|FR.17||The Session Master will be able to search on Twitter using a keyword|
|NR.1||The application will be designed to be extensible and maintainable|
|NR.2||The application will be highly responsive|
|NR.3||The application will be reliable|
|NR.4||The application will use responsibly hardware resources|
Now let’s move to the Architecture of the application!
Because of the specifics of the brainstorming activities, there are two sets of features that must or can be implemented: the first ones which regulate the brainstorming activity as a set of rules; the second ones add productivity to the process, enabling additional scenarios that improve the user experience. In the first phase of requirements elicitation for such a system, the Microsoft Surface was chosen as a direct, face to face collaboration. For enabling the sharing of Brainstorming session, BSCW system was chosen as it enables shared workspace paradigm, allowing easier collaboration and cooperation among users with common interests.
Detailed usages of typical brainstorming and idea generation tools were presented in Related work. What all this papers have in common are few scenarios which are very important for the idea generation process:
- Creation of ideas and inserting them into the system (e.g. Creating text items with brainstormed ideas)
- Editing (e.g. Correcting, improving and augment with graphical elements the already existing items)
- Grouping through visual clues (e.g. creating logically grouping structures with the aid of visual elements)
- Relating (e.g. creating a tree-like relation that gives a structured form to the ideas generated through the process towards a specific goal)
A typical user scenario is presented to clarify the use and features provided by the system.
Anton, Kai and Vladimir are three students developing applications in Fraunhofer FIT under the supervision of a research assistant, Greg. They set up a meeting in the User Experience lab, where the Microsoft Surface is, and set as their goal to create a new system for helping people with Alzheimer.
Because Greg is the coordinator of the brainstorming session, he uses its card to login into his BSCW system which will be later used as a repository and central sharing hub for the outcome of this session. Then he inputs the text “Alzheimer project” into the central control on the application, allowing all of them to keep focus on their goal. They start adding to the Surface the ideas that they have, as the brainstorming process is about quantity. When they feel that a small dose of inspiration is required, they use the application’s feature of searching in Twitter or Flickr for fresh ideas or hints. Since they work on the same device, the awareness of each one presence is complete, conversations start over some interesting features, raising more and more ideas for their project.
After the main part of the brainstorming is finished and all ideas are inputed on Microsoft Surface application, the next phase starts – reviewing their ideas and correct the small mistakes that they made in typing. Anton also quickly adds some more visual hints to ideas, like arrows pointing to the text or underlines some important keywords. Kai enjoys the dictation feature of the application to quickly add some more information to some ideas that he feels that aren’t very well explained.
With all ideas correct and complete, they go into the next phase of requirements elicitation for their application start grouping them into functional and non-functional features for their system, using the background colors to create visual hints. This enables them to have a quick overview on the ideas they gathered and relate them to a specific part of their software planning.
The nest step in the process is to link the ideas into a tree like structure, as all good software developers do, identifying the dependencies between the ideas. This will prove useful later, when they will create the necessary UML diagrams for the software.
Since they will go afterwards to their own workstations and they need to review and create the appropriate documents for their project, Greg uses the saving feature of the application, which will create a folder in his Workspace on the BSCW system. In it he will later find a screenshot of the application’s screen to have a quick visual of the session, the entire session metadata saved as XML for later use in other applications and a collection of all ideas saved as HTML documents.
Because all work needs to be shared and the session repository is on the BSCW system, Greg can share his session with his colleagues by inviting them to the session directory. Thus, not only that all of them will have access to the session, but they can choose later to present the outcome of their work to their Professor, by using the Load session capability of the Microsoft Surface application.
An additional scenario can be used with this application, if there is time for Brainstorming session preparation:
If they have time to prepare for the meeting, some rough ideas may have been conceived before and they want to share with each other. Because using the Microsoft Surface is not possible with prior reservation, they may use the additional clients of the system to record their initial ideas: Adobe Air application (available for Windows, Linux and Apple) and an iPad and iPhone client for the mobile devices. Anton and Vladimir use the Adobe Air client to quickly create and send their ideas to the Surface default user, where the brainstorming items will be recorded and made available in the next Brainstorming session. Kai uses its iPhone to do the same, as he gets its best ideas while commuting.
Next let’s see how the requirements look like!