Uwe Malinowski (1) and Kumiyo Nakakoji (2) (3)
Software Research Associates Inc.
Software Engineering Lab.
1-1-1 Hirakawa-cho, Chiyoda-ku
Tokyo 102, Japan
Kumiyo Nakakoji
Siemens Corporate R&D;
ZFE ST SN 5
D 81730 M�nchen, Germany
+49 (89) 636-2969
malinow@zfe.siemens.de
User interface design and end-user adaptation during the use of the system should be viewed as an ongoing collaborative design process among interface designers and end-users. Existing approaches have focused on the two activities separately and paid little attention to integration of the two by supporting their asynchronous collaboration over a long period of time throughout the evolution of the interface design. Our knowledge-based domain-oriented user interface design environments serve both as design media and as communication media among interface designers and end-users. An embedded computational critiquing mechanism not only identifies possible problematic situations in a design for user interface designers and end-users but also facilitates asynchronous communication among stakeholders. The presentation of critiquing messages often triggers designers and end-users to articulate design rationale by describing how they responded to the critiques. The recorded design rationale mediates collaboration among end- users and user interface designers during the end-user adaptation and redesign of the interface by providing background context for a design decision.
Issues of concern to user interface development are twofold: (1) knowledge necessary for
interface design is distributed among interface designers and end-users; and (2) design tasks are
ill-defined and open-ended by nature.
First, design knowledge necessary for interface design is distributed [11]. Designing a user
interface requires two types of knowledge: knowledge about user interface design and knowledge
about the application domain. Initially, there is a symmetry of ignorance between interface
designers and end-users in which (1) the understanding required to solve the design problem is
distributed, and (2) there is no common language that allows the stakeholders to communicate
their understanding to each other [11]. Thus, not only the thin spread of application domain
knowledge among user interface designers [2] but also little understanding about computer
technology among end-users [3] causes communication breakdowns between end-users and
interface designers [5].
Approaches have been made to improve collaboration among interface designers and end-users.
A participatory design approach encourages end-users to have a stake in the user interface
design process from the very beginning [3]. Another approach is to encourage interface (system)
designers to participate in end-users' work settings [12]. Although we acknowledge the
importance of such face-to-face collaboration among interface designers and end-users, a
challenge we see is that interface designers are not available all the time during the use of the
system. Once the system is released, designers are rarely assigned to maintenance and it is
difficult for end-users to contact designers directly in order to communicate their requests. End-
users and interface designers need to collaborate asynchronously over a long period of time
throughout the life-cycle of a system.
Second, user interface design is an ill-defined and open-ended problem-solving task.
Understanding a design problem and solving it are intertwined tasks; one cannot completely
specify the problem before starting to solve it [23, 24]. Starting with a vague and incomplete
problem description, people sketch out a partial solution. By seeing the partial solution, they
identify portions of the problem that have not yet been understood, gain an understanding of the
problem, and then refine the solution [25]. By iterating this process, understanding of the problem
gradually emerges. Design problems are open-ended in that the knowledge necessary for a
design can never be completely articulated a priori [23]. Traditional knowledge acquisition
approaches that aim at capturing expert knowledge will not work because some part of expert
knowledge always remains tacit - people know more than they can tell [21].
Thus, end-users cannot foresee the many different ways in which they will use the system.
Furthermore, the system might change the way tasks are performed, and changes to the
environment might make a different design necessary. Specific needs always emerge during the
system usage, not due to sloppy requirements analysis but due to the very nature of design tasks.
In response to this, ways to enable the end-user to adapt the user interface have been proposed
[15].
To cope with these challenges, we present our approach to view user interface design as an
evolving collaborative design process by interface designers and end-users of the system over a
long period of time [9]. The process starts with the development of a user interface by user
interface designers while collaborating with end-users. During the use of the system, end-users
should be enabled to customize the system to their specific needs, which cannot be determined a
priori but emerges only during use. After a while, user interface designers should be able to
redesign the interface because end-users may have introduced inconsistencies and inefficient
modifications during the adaptation.
Knowledge-based, domain-oriented design environments that facilitate asynchronous
collaboration among different types of stakeholders in various design domains have been studied
[7]. A computational critiquing component of such an interface design environment [8] supports
the interface design and end-user adaptation as an evolutionary design process. The embedded
critiquing component:
In this paper, we first describe a computational critiquing mechanism embedded in a domain-
oriented user interface design environment as a framework of our approach. We then present a
scenario to illustrate how the critics support user interface development as a collaborative
evolutionary design process using the domain of traffic management systems as an object-to-
think-with. We describe the prototype system and discuss necessary improvements.
Domain-oriented knowledge-based design environments that cope with the issues discussed in
the previous section have been studied in various domains, including kitchen floor plan designs
and network design [7]. We have applied the design environment approach to user interface
design. In the application domain of integrated traffic management systems, for example, the
system provides knowledge not only about interface design (e.g., a guideline in designing a pop-
up menu) but also about the domain (e.g., necessary functions for a train station).
Figure 1 illustrates a cycle of design, user adaptation, and subsequent redesign as a process
model for user interface design supported with a design environment. The system is initially
designed by interface designers in collaboration with end-users. Although not shown in Figure 1,
this collaboration is very important for both design process and result. The application is equipped
with mechanisms that allow end-users to adapt the interface during the use of the system. By this
means, the interface "gradually evolves." Finally, the system is occasionally "redesigned" by
interface designers in collaboration with the end-users to reorganize changes made by the end-
users that might have introduced inconsistencies.
FIGURE 1:User Interface design as an evolutionary process.
Critiquing is the exchange of a reasoned opinion about a product or action, which triggers further
reflection on or changes to the artifact being designed [8]. Critiquing has been found to be an
effective and natural method people use in collaborative problem solving [10, 17]. The critiquing
mechanism embedded in a design environment helps people engage in design activities by:
The first two aspects of the mechanism support user interface designers and end-users to exploit
the system's stored knowledge by delivering the task-relevant knowledge even when they do not
know about the existence of the knowledge, or when they do not realize the need to know the
design knowledge. A part of such knowledge can be accumulated through the third aspect of the
mechanism: design rationale.
Previous efforts [8, 14] have reported that embedded computational critiquing mechanisms in
design environments have proven to be effective in supporting user interface designers by
providing application-domain knowledge while developing a user interface.
We also use this critiquing mechanism to support end-users to adapt their interface to their needs
by providing user interface design knowledge. Computer-aided adaptation has been proposed to
enable users to adapt the user interface [13]. Experiments with prototypes [16] showed that
enabling is not enough - end-users need support during the adaptation process.
We have identified that the critiquing approach is an effective knowledge-elicitation technique by
motivating people to articulate design knowledge [19]. In the study of critiquing components in
various design environments, we have observed that users often articulated why they violated the
principles when being critiqued. This information was accumulated as design rationale. For
example, in the study of the KID kitchen design environment [18], when a subject used a double-
bowl sink when designing a kitchen for a single-person household, and KID presented a critiquing
message saying that the subject should use a single-bowl sink if the design was for a small family,
the subject articulated: "Well, ... Do you think a college student will do dishes every day? I don't
think so... I wanna stack up dishes in one sink and I need another one."
In the study of a critiquing mechanism of another system called VDDE (Voice-Dialogue Design
System) [1], when the system informed subjects that a design violated a certain design guideline,
they stated that another conflicting rule was more important, therefore, they had to violate the
indicated guideline.
The above evidence demonstrates that critiquing mechanisms are effective methods to elicit
knowledge from users and encourage them to articulate design rationale.
Design rationale plays a crucial role in supporting long-term asynchronous communication among
stakeholders in design tasks [6]. Design rationale provides descriptions and explanations for why
a certain design decision is being made [6, 26]. Because the context of certain design decisions is
not always explicit, explicitly articulated design rationale that describes why a design decision is
made with regard to which design intention helps people to have a better understanding of the
design. In user interface design, for example, design rationale recorded by interface designers will
help end-users adapting the user interface to understand why the user interface has been
designed that specific way. A ration-ale for adaptation recorded by end-users will help user
interface designers to redesign the interface. This rationale can later be translated into knowledge
usable by the critiquing component. Thus, triggered by critiquing mechanisms, the system serves
as communication media that supports collaboration between user interface designers and end-
users. Figure 2 illustrates this process.
FIGURE 2:Asynchronous Long-term Communications
THE SCENARIO.
In this section, we present a scenario to illustrate our approach in the domain of user interface
design for a traffic management system to allow traffic managers to monitor and influence the
traffic flow on streets as well as that of railway trains. An interface design environment for the
domain is provided for user interface designers. It consists of a user interface toolkit, a critiquing
component and an argumentation component. The scenario focuses on how the order of
functions in pop-up menus has been refined by designers and by end-users throughout the design
life cycle, supported by the system's critiquing.
The user interface is required to show the graphical representation of the objects, including
streets, intersections, railway tracks, and stations. Figure 3 illustrates an example interface
consisting of several streets and railroad lines, two intersections with the main street ("IS-1" and
"IS-2") and two railway stations ("ST-A" and "ST-B"). The required functionality for each of the
objects is depicted in Figure 4.
FIGURE 3:User Interface for Traffic Management.
FIGURE 4:Required Functionality
Using the given requirements, interface designers start designing pop-up menus for the
functionality of the objects under consideration. Since no order criterion is specified, they choose
to organize the pop-up menus alphabetically. However, Display Video is placed in the last position
in the menu for the intersection IS-1 in order to keep the consistency among objects of the same
type, i.e., with IS-2, which does not need the functionality because video cameras are not installed
in IS-2 (Figure 5).
FIGURE 5:Menus after the initial design.
The critiquing component analyzes the design. The rule "Consistency between pop-up menus"
fires, saying that the pop-up menus for stations and intersections are not consistent. Having
chosen alphabetical order and consistency between pop-up menus for objects of the same type
as the ordering criteria, the designers do not understand why the system complains about this. So
the designer decides to inspect the related argumentation, which describes why this rule fired in
the current situation. The argumentation component explains that in this domain, the functions
Schedule Screen and Traffic Light are semantically analogous to end-users (see Figure 9).
FIGURE 9:Argumentations for Designers.
This leads the designer to exchange the positions of Schedule Screen and Display Video on the
menus for ST-A and ST-B so that Schedule Screen and Traffic Lights appear in the same position
in the menus (Figure 6). The designers record this design rationale in the argumentation
component to describe why they switched the positions of Schedule Screen and Display Video.
The design is now released to end-users for real use.
FIGURE 6:Menus after Redesign Based on Critiquing.
After using the system for some time, one of the users thinks that the order of the functions in the
pop-up menus is inadequate. In a discussion with colleagues, the decision is made to change the
order of the items in the pop-up menus according to the frequency of use (Figure 7).
FIGURE 7:Menus after Adaptation by the End-
users.
When end-users make this arrangement by using adaptation mechanisms provided by the design
environment, the system presents a critiquing message complaining about the lack of consistency
between pop-up menus, as mentioned above. The user presses the explain button to get an
explanation of why the critiquing message is significant. The argumentation component explains
that consistent positions of functions in pop-up menus support learning and reduce search time
and error rate (Figure 10).
FIGURE 10:Argumentation for End-users.
With this explanation, the end-users understand that it is principally good to have consistent pop-
up menus and that this rule must have been the reason for the user interface designer to set up
the menus as they were. However, after discussing this conflict between the users' needs and the
rule from the interface guidelines, the users decide to change the order as planned (Figure 7)
conscious of the violation of the rule. The users document this decision in the argumentation tool,
explaining that it seems to be more important for them that the menus fit their needs defined by
their tasks than the user interface guidelines (Figure 11).
FIGURE 11:Design-Rationale recorded by End-user.
The system is used in the changed setup for some time before a new version is developed that
includes a redesign of the user interface. The user interface designers examine the adaptations
made by end-users during the usage of the system.
Inspecting the adaptation of the pop-up menus, the designers wonder why the users might have
adapted the interface in a way that clearly violates user interface guidelines, although they must
have been informed by the critiquing component about the rule concerning semantic consistency.
The designers examine the argumentation available for this design decision. After reading the
explanation given by the end-users, the designers understand that the end-users think that
usability increases if the frequency of usage is reflected in the order of the menu items. This was
not part of the initial requirements, nor had end-users detected this before using the system on a
regular basis.
Therefore, the designers decide to design the pop-up menus in a partly consistent way, so that
the menus for one type of objects is consistent, but not necessarily consistent with the menus of
different objects by changing the order of functions in the menu for IS-1 and IS-2. This design also
allows consideration of the frequency of usage of the functions (Figure 7). The solution is
discussed with and accepted by the end-users who made and explained the adaptation of the
pop-up menus.
FIGURE 8: Menus after Redesign based on Adaptation.
Besides informing the user interface designer about the application domain, the domain
knowledge implemented by the end-users in the argumentation tool is used in a second way. A
knowledge engineer analyzes all the information and implements it in the knowledge base of the
critiquing component. In this example, the frequency of usages of the functions is added to the
knowledge base (Figure 11).
FIGURE 12:Knowledg_Base Update by a Knowledge Engineer based on the Design
Rationale of the End-user
The scenario illustrates how the embedded computational critiques support the evolution of the
user interface design by:
Our emphasis in using critiquing mechanisms in a design environment is on providing related
argumentation and explanation of why the critiquing rule is significant, rather than on simply
notifying users or designers of possible problematic situations. Different stakeholders need
different descriptions for the same critiquing rule. That is, critics for user interface design are
related to both user interface guidelines and domain semantics. In the scenario a critique fired
using these two types of knowledge:
For example, in the scenario, the same critiquing rule fired to both the end-user and the designer:
"pop-up menus need to be ordered semantically consistent." User interface designers needed to
know which operations are semantically corresponding to each other in this specific domain. End-
users needed to know what "semantic consistency" is and when and why it is important in user
interface design.
Thus, the critiquing mechanisms need to combine both domain knowledge and user interface
design knowledge. Our implementation separates the two types of knowledge by using the notion
of "domain-distinction," which is further described below. By integrating these two types of
knowledge-bases rather than using a single knowledge-base for "user interface design for traffic
management systems," the same user interface guideline knowledge can be combined with
another application domain knowledge to produce a domain-oriented design environment.
The current prototype of a user interface design environment for integrated traffic control systems
is implemented on a Macintosh computer using MCL. The prototype has been built on top of
Agentsheets, a substrate for building visual design environments [22]. Agentsheets supports the
creation of galleries of design units that can be used to build visual representations in a work area.
In this prototype system, tools for adding, erasing, and moving objects can be selected from a
toolbar. The user interface objects are presented in the gallery and can be copied into the artifact.
Currently, the application-specific objects include streets, intersections, railroad lines, bridges, and
stations. Additionally, pull-down and pop-up menus are available (Figure 3).
FIGURE 13: User Interface Design Environment with Critiquing Message
Pane
The critiquing component consists of a critiquing manager and a set of rules. The manager
decides which rules are applicable in the current situation, based on the action of the designer
and on the type of the objects involved in the current design activity. The rules themselves are of
the form
if condition then reaction
with condition describing a situation that is an error or a problem that requires a warning. Reaction
defines the message that will be given to the designer.
The critiquing component monitors the actions of the designer. Every time an interface object is
placed, moved, or deleted, or the definition of a menu is changed, the critiquing manager
identifies all rules that are applicable in this situation. The conditions of all rules are inspected.
The messages defined in the rules where the condition is true are displayed in the message
window. This means that all information is delivered to the designer immediately as a response to
the design activity. Additionally, the designer can select a "critique complete design" function. This
is necessary because some conditions can be checked in a meaningful way only when the design
is finished; otherwise, they would deliver the same information after each design step. An example
involves the completeness rules (see below).
The messages are presented in a message window. In this window, the "explain" button gives
access to the explanations of rules. Additionally, rules can be disabled for the current situation
("reject" button), allowing the designer to define an exception situation. A rule disabled for a
specific situation will not be checked in the future for this part of the design, but remains active for
the rest of the artifact. Pressing the "where" button highlights all objects related to any currently
displayed message. Selecting a single message highlights only the objects related to this
message.
The set of currently implemented rules covers different groups of rules. Some rules represent
user interface design principles, like a maximum number of items in a menu or the maximum
depth of cascaded menus. The second group of rules checks the completeness of the design
according to a given specification. This currently covers the required objects and required
functions for the system and for the individual objects. Another group of rules represents the
consistency requirements, for example, that functions with the same name always appear in the
same location in all pop-up menus. Finally, a group of rules that represent design knowledge that
is strictly application domain specific was implemented. These rules describe the topological
relationship of user interface objects that are allowed in this domain.
As mentioned above, our prototype system integrates two knowledge-bases: user interface
guideline knowledge and application domain knowledge. Figure 14 illustrates an applicability
condition. It checks whether the operations in the same position of two different menus (menu-A
and menu-B) are semantically consistent. Note that the function "menu-consistency" is
independent from the application domain. Providing another "domain-map" will check semantic
consistency in a different domain.
FIGURE 14: An Applicability Condition Rule for Menu Consistency
In order to link critiquing rules with argumentation, we use the notion of domain-distinctions (see
Figures 9, 10, and 12). The argumentation-base is structured as a network of nodes consisting of
issues, answers, and arguments based on IBIS [4]. An issue-answer pair represents a design
decision in terms of function, structure, or behavior at various levels of abstraction. In our system,
issue-answer pairs and arguments are associated with domain-distinctions. Whereas the domain-
distinctions come from the vocabulary of user interface design, the arguments are part of the
application domain vocabulary. Each domain-distinction is related to a LISP function that checks
the current interface design. For example, "Semantic-correspondence (menu-A menu-B)" is
related to two predicates: first to determine which functions (operations) are semantically
corresponding (i.e., domain knowledge), and second, to determine whether semantically
corresponding operations such as "schedule screen" and "traffic lights" occur in different positions
in different pop-up menus.
Keywords:
Usability Engineering, Collaborative Design, Design Rationale, User Interface Design
Environments, Critiquing Systems, End-user Adaptation, Process Control.
Introduction
FRAMEWORK:
EMBEDDED CRITICS IN A DESIGN ENVIRONMENT
(1) Initial Requirements
(2) Initial Design by Interface Designers
(3) System's Critique of the Initial Design
(4) End-users' Adaptation and the System's Critiquing
(5) Redesign by Designers -
Informed by the Recorded Argumentation
DISCUSSION OF THE SCENARIO
THE PROTOTYPE
