Abstract
Isis is a multimedia authoring tool for children, where
videos, photos, drawings, texts, sounds and
cartoons are treated as electronic building blocks (time boxes).
Adopting a metaphor of simple building blocks, Isis allows children
to create complex time-space multimedia stories by
stacking and arranging "time boxes" on the screen.
The algorithms within Isis are based on symbolic temporal
constraints, such as "co-start", "co-end", "co-occur",
"meet".
Keywords
Multimedia, educational applications, home applications,
entertainment applications, multimedia authoring.
Introduction
In recent years, research activities on
uses of computers in schools, in particular
in K-12 education, have increased greatly.
The notion that children learn by constructing their own
knowledge is highly popular among educational theorists.
Children ought to be active, not passive, in the learning process.
They ought to be doing something, not merely watching it.
Multimedia technologies
offer children the opportunities of learning "actively" by
allowing them to
construct knowledge as interactive multimedia documents
(e.g. multimedia stories).
Isis is an interactive multimedia document building
tool for children.
Isis has been developed
as part of the Home Health-Care Prototype System
[ 4,5 ]
for children with leukemia,
which IBM has developed jointly with
New England Medical Center.
When a child is on chemotherapy, the drugs can lower the blood
counts over time
and make the child anemic, causing the child to feel fatigued.
The child may not want to engage in activities,
although once engaged, the child will usually be able to perform
because the distraction will be able to raise them out of the
fatigue state.
Medical providers recommend that children
under chemotherapy avoid passive activities such as watching television
and reading books, but
be encouraged to do something active and creative.
.
Our aim is to foster creativity in children
and motivate them to do what they can enjoy.
Children can create
a story by connecting
multimedia objects using relationships between them.
The multimedia objects are
new building blocks.
These building blocks not only occupy space, but also occupy time,
so we call them time-boxes.
As original building blocks can be stacked
together, the time-boxes
can start together, end
together, or occur together.
As original building blocks
can be put side by side, time-boxes can meet one another.
.
Isis is an implementation of the Hyperstory model
of combining time, space and asynchrony in a multimedia document
[ 3 ].
The Isis model is unique in that the construction principle is based
on temporal constraints, or temporal relationships,
rather than on hard-wired time points, thus
providing flexibilities.
Furthermore, from the
temporal constraints spatial constraints are derived,
thus simplifying an otherwise complicated problem of obtaining a
time-space layout of a multimedia document.
The model also allows the representation of
asynchronous
control, such as "branch" or "call",
in a multimedia document (a hyperstory).
.
The current implementation of the Isis system
has been used to provide the multimedia substrate
for the Home Health-Care Prototype System.
We took its simpler subset to give to the children,
focusing on the concept of time and space.
It is under usability tests currently, and
early empirical findings suggest that it has positive impact on children.
.
It is our aim to make a system such as Isis
available as an authoring tool
in schools and in homes for children as well as for adults
who are non-computer professionals.
Children can make their music videos and build their
talking story books;
students may prepare multimedia term-papers;
professionals may prepare multimedia presentations;
parents may create multimedia stories for their children.
Temporal Constraints for Temporal Layout
Multimedia authoring can be viewed as a process of ordering,
or integrating, multimedia objects.
We call a set of multimedia objects to which a certain ordering
has been given a story.
Ordering of objects with respect to
a story may be done both in the temporal dimension
and in the spatial dimension.
.
Reasoning about time has been the focus of much of research activities
within Artificial Intelligence, and
many formalisms have been proposed for temporal reasoning
[ 1, 2 ].
Most notably, James Allen's interval algebra
[ 1 ], where he
defined 13 possible relationships between a pair of intervals,
has received much attention in the AI community
for its simplicity.
One of the requirements in authoring with multimedia objects
is the ability to deal with metric information, such as duration.
Note that an Allen-style interval algebra does not offer
a convenient mechanism for dealing with metric information.
A representation of time to provide metric information,
while allowing uncertainties in the representation,
has been proposed in [ 2 ].
As a number of time intervals are related by temporal relationships,
where each time interval is bounded by its
minimum duration and the maximum duration,
it has been shown that
the time intervals and their relationships
can be conveniently represented as a graph to which a shortest
paths algorithm can be applied. If the relationships are consistent,
the algorithm provides two sets of answers: a set of earliest
possible times and a set of latest possible times.
.
We adopt a representation
that takes intervals as primitives as in [1 ]
and describe a multimedia story using relationships between
its components (multimedia objects).
To reflect our requirements
for representing metric information as well as
uncertainties, we associate with each object a
time interval with its
minimum and maximum bounds. For instance, a video object
may be played as fast as 60 frames/second,
and as slow as 10 frames/second.
Given a set of objects and the corresponding
temporal constraints, we then obtain a temporal layout
by applying a shortest path finding algorithm as
in [ 2 ], and obtain
two sets of possible solutions: the earliest possible times and the
latest possible times.
But the simple subset we have chosen for children assume a special case
where there is only one length for a time interval.
Creating a Multimedia Story
(Scanned image of paragraph with symbols not renderable by HTML)
(Scanned image of paragraph with symbols not renderable by HTML)
Spatial Layout
With a small number of objects having a simple temporal ordering
on them, placing them on the screen may be a straightforward task.
But with a large number of objects with a complicated
temporal ordering, it may become an involved
process to find a suitable spatial placement,
especially if the process is to be iterated.
Without losing generality, we assume that
all our objects are or visible.
Depending on where on the time line
the objects fall and with which objects they may overlap on the
time line, the location and the amount of space they may occupy
on the screen may vary.
As such, finding a spatial layout
is a problem that may be characterized as an XYT problem,
XY being two dimensions that define a plane and T being the
temporal dimension.
.
Given temporal information in a story,
we identify temporal
cliques
such that a clique consists of a number of objects
where there is a shared
time point over which all the objects overlap.
As such,
a temporal clique can be treated as if it were on an XY plane with
no knowledge of time.
The problem of finding a spatial layout, then,
becomes a number of XY problems, which is easier to solve.
Once temporal cliques are found,
the user can work with one clique at a time,
placing the objects on
the screen without worrying about their temporal dimension.
Acknowledgements
The author is grateful to
Ho Soo Lee for his help in
constraint satisfaction algorithms,
H. Ellozy, W. Kellogg, A. Khorasani, J. Koenemann, R. Schloss, and
L. Tetzlaff,
for their help in improving the
usability of the system, S. Greene and J. Koenemann
for their valuable comments on the user interfaces of Isis,
and J. Backman for being such an inspiring participant in the
usability test and for his useful comments
on the interfaces.
.References
- J. Allen,
Maintaining Knowledge about Temporal Intervals,
CACM, Vol. 26, No. 11, 1983.
- R. Dechter, I. Meiri and J. Pearl,
Temporal Constraint Networks,
Artificial Intelligence, 49, 1991.
- Kim, Michelle, Hyperstories: Combining Time, Space and
Asynchrony in Multimedia Documents, IBM Research Report RC 19277, 1993.
- Kim, M., Schloss, R., Tetzlaff, L.,
Home Health Care Support System,
CHI'95.
- Kim, Michelle, Guardian: A Knowledge-Based Home Health_care System
for Children with Leukemia, IBM Research Report RC 19858, 1994.
