Shinji Nabeshima*, Shinichirou Yamamoto*, Kiyoshi Agusa*, Toshio Taguchi**
We have developed a new input device, named MEMO-PEN. The MEMO-PEN is an ordinary ball point pen except for a capability of memorizing what it draws in itself. We can hold the handwriting in computers as it has been, and we can treat it as image data for favorite application. The pen is carrying a small CCD close to its tip to capture a series of not whole but partial snapshots in its memory. This paper presents a brief description of the MEMO-PEN system and an experiment on reconstructing a whole image from the partial snapshots. We also show that we can satisfactorily work out a whole image for 80% of more than 50 examples.
Popular input devices at present are keyboards, mice, and pen-based
input device. But, each device has various limitations as
follows. On a keyboard, novices waste a lot of time to learn it.
Furthermore, we can not expect smaller in size because of the
limitation of our finger size. On a mouse, which is suitable for
drawing figures and pointing, when a good idea occurs, it is difficult
to put the idea into computer immediately with it. On pen-based input,
which is adopted in electronic notebooks, it never satisfies the
two contradictory demands, to be bigger for easy reading and to be
smaller for portability, because the display for output and the
digitizer for input are combined into one device.
In order to overcome these limitations, we propose a new system, named
the MEMO-PEN system. The system consists of two components; the
MEMO-PEN, which is a ball point pen with a CCD close to its tip and
memorizes a series of partial snapshots of the handwriting captured by
the CCD; the image reconstruction software, which reproduces the whole
handwriting from the partial snapshots. This capability frees us from
the limitations of materials for writing, e.g. tablet and touch panel.
We can write down our new idea, whenever it occurs, on whatever
materials, e.g. paper, wall, box, and even our palm.
Since the usage of the pen just conforms to the familiar style of
thinking and writing, which we call "Paper and Pen."
Without any doubt, novices can use the system immediately.
As another candidate of input devices, some may propose scanners,
which capture a whole image of handwriting at once. It has same
capability of putting the image into computers easily. But, the
MEMO-PEN system has an important advantage in that the system can
reconstruct dynamic images. We can playback the sequence of the
handwriting, from which the process of thinking is guessed.
The MEMO-PEN has lenses, a CCD, a stress sensor, a micro computer, a
memory, and a battery. Figure 1 shows the MEMO-PEN.
FIGURE 1: Structure Model of the MEMO-PEN.
When we write letters and/or figures with the MEMO-PEN, the snapshots
of handwriting are captured by the CCD and stored as locus data in the
memory at every fixed sampling time. Simultaneously, the stress
applied on a ball point pen is detected by a stress sensor and its
value is stored in the memory, too.
Locus and stress data are transmitted to a computer through the
interface. But, because the CCD is close to the tip of the pen and its
scope is narrow, locus data are not whole images of handwriting but
partial ones. Figure 2 shows examples of the partial
image. A whole image is reproduced from the partial ones by a
reconstruction software.
FIGURE 2: Examples of Partial Image.
Because the MEMO-PEN itself moves on paper, it is impossible to detect
the absolute position of the tip of the pen on paper. But, two
successive partial images have common locus, since the sampling time
is short enough that they can overlap each other.
Therefore we can find the position of a partial image relative to the
previous one. We reconstruct a whole image by combining the partial
images according to the positions.
We have developed the prototype of the MEMO-PEN. Figure 3 shows our
current prototype. Our current prototype
is a tethered version, and the pen is separated from a battery and a
memory. The pen is 200mm in length and 70g in weight, and uses
1M-byte RAM-CARD as the memory. The pen can memorize the handwriting
drawn for about 15 minutes.
FIGURE 3: Prototype of the MEMO-PEN.
We are designing the next prototype, an untethered version, which
includes a memory and a battery in a body. The next prototype will be
about 170mm in length, about 35g in weight, and have 500K-byte
memory.
We have made an experiment on reconstructing a whole image from
partial images. We have written 50 samples of handwriting with the
MEMO-PEN, and in them we have reconstructed the whole images of 40
samples satisfactorily.
We have written the following samples. The sorts of
handwriting are alphabets, Japanese characters(KANJI), formulas, and
symbols. The sizes of characters are 1-2cm. The numbers of
characters are about 5 characters par line and 1-4 lines.
It takes about 10-60sec to write each sample. The sampling time
is 10 snapshots/sec. The number of snapshots in each sample is
100-600. It takes our reconstruction software 45-420sec to
reproduce each whole image.
Figure 4 shows an example of the whole images.
FIGURE 4: A Reconstructed Whole Image (Formula).
We have proposed the MEMO-PEN as a new input device. It can memorize
what it draws in itself. The capability frees us from the limitations
of materials for writing, e.g. tablet and touch panel. We can
reconstruct the handwriting in computers as it has been.
We have made an experiment on reconstructing a whole image from
partial snapshots captured by the MEMO-PEN. We have satisfactorily
reconstructed the whole images of 40 samples in 50 ones.
We have some problems on our current reconstruction algorithm. First,
we can not detect a long straight line because its locus have no
visual change. On this problem, we extrapolate the positions of the
tip of the pen from the motion of the pen just before the locus
provides no visual change. Second, we have not dealt with the
distortion when the pen is held at different angles. But it does not
seem to cause failures of reconstruction in our experiment.
Jenny Preece, Yvonne Rogers, Helen Sharp, David Benyon,
Simon Holland, Tom Carey.(1994).Human-Computer Interaction,
211-236.
Introduction
THE MEMO-PEN SYSTEM
EXPERIMENT
CONCLUSION
References
