PARALLEL DESIGN IN THE CLASSROOM

Saila Ovaska and Kari-Jouko Räihä

University of Tampere, Department of Computer Science

P.O. Box 607, FIN-33101 Tampere, Finland

+358 31-2157058, +358 31-2156952

ov@cs.uta.fi, kjr@cs.uta.fi

© ACM

Abstract

To be able to compare the various solutions analytically, we made a list of all the good or interesting features that came up in the solutions. We have tried to include all features that are at least worth further consideration. Notably bad design features have been omitted from this analysis. As noted in [4], composing such a list is not straightforward. The list we have used is given as an APPENDIX.

Single novice designs contained from 9 to 24 good features; the expert designs scored 16, 20 and 26. Altogether the expert designers introduced 51 design features, which shows that the designs were radically different. The first novice group scored even higher than the experts, 59 features, while the second group produced 49 design features.

THE REMOTE CD PLAYER ASSIGNMENT

The task as described in the INTERCHI'93 proceedings is to design a remote control panel for a hypothetical CD-player with a jukebox-like functionality and over 10000 discs. The system operates like a central database, from which the songs are played via a broad band network to a set of speakers at the user's workstation. It is possible to select songs by different search criteria, and play them.

The functionality description as given to the expert designers [3] has been left vague by Nielsen and Desurvire; they claim that in practice no detailed functionality descriptions are available in the first place. As the task description allowed various interpretations of the task, also an extensive functionality can be planned. For exam- ple, saving play lists and loading them into use, managing time and cost, and sup- porting a wider variety of search combinations will make the user more satisfied with the system. Some of these options (like finding a song with just a phrase of its lyrics) may be hard to implement, but nevertheless, we have included them as desirable design features in our analysis, since they came up in some proposed solutions.

ANALYSIS OF THE DESIGN SOLUTIONS

Our updated list of design features contains 69 design ideas - a considerable number in view of the relatively limited functionality. Some design choices are mutually exclusive, so it is impossible for any single design to incorporate all the features in the list. The novice groups of 1994 introduced only two additional good ideas into the list; the design space seems to be saturating.

How did the designs fare? The variance was high in both groups. For the expert designs, the scores (number of features appearing in the designs) were 16, 20, and 26. For the student designs, the scores varied between 9 and 24 in 1993, and between 11 and 24 in 1994. It is interesting that quite a few student designs scored 20 or more, a good result. On the other hand, 9 is a really low score - in some cases our feature list contains all possible design choices, so any design will automatically score a few points.

The first conclusion to be drawn from this is the one familiar from other contexts in usability engineering: never trust a single opinion. Some interesting questions are:

1. how many parallel designs should be carried out to produce a large percentage of the useful design ideas; and
2. how much does the experience of the designers affect this number.

The above figure shows the expected number of design features produced by a given number of (groups of) designers. First of all, we observe that the designs produced by the experts were radically different. The slope of the curve is steep, and taken together, the designs of the three experts exhibit 51 of the 69 features. On the other hand, the novices were thus able to come up with 18 good ideas that did not appear in any of the expert designs. Inversely, the experts suggested 5 ideas that did not appear in any student designs.

It is remarkable that the expected value for just two novices is greater than that for a single expert! From there on the novice curves grow slowly, as could be expected. For the 1993 course, the expected value of 51 features for three experts is matched when the number of novices exceeds 16.

It is striking that the curve for the 1994 course grows slower than the one for the 1993 course. One explanation is the smaller number of groups. Just one or two outstanding groups can affect the expected number of design features considerably. For a comparison of the two courses, it would make sense to exclude a few of the best and the worst solutions.

There is, however, also another explanation. Experience with the 1993 course had given us a firm understanding of good design ideas (just you wait... [5]). Although we, of course, tried not to guide the students in any particular direction, it is plausible that our view of the problem affected their solutions, which were developed in four stages and discussed with the course assistant after every stage. This suggests the importance of really doing the designs independently of each other.

As a final note, although they had to work hard, the students enjoyed the problem a lot and appreciated seeing (and criticizing...) the expert solutions at the end of the course.

ACKNOWLEDGEMENTS

References