The story goes that the current QWERTY keyboard layout was designed some 150 years ago specifically to slow down typists enough to keep the mechanical keys of 19th century typewriters from jamming. Whether this is true or not, the fact is that the keyboard represents the bottleneck between the human mind and the computer. With computers gaining more and more processing power and becoming ubiquitous in our society, the keyboard bottleneck becomes ever less acceptable. You could argue that this ancient method of communicating with machines is one of the greatest detriments to productivity we are facing today.

As a result, there have been many efforts to develop alternate ways of communicating with computers. The mouse was invented to provide an easier way to navigate a new generation of graphical user interfaces. Handwriting recognition was pursued to provide a more natural way to enter data for people not familiar with computers and keyboards. Voice recognition was seen as a way to simply speak to a computer instead of typing or writing.

Well, the keyboard is still here. And that is because making handwriting and voice recognition work turned out to be more complex than anticipated. Suffice it to say that much progress has been made, but that neither technology is quite there yet.

The primary problem is that people working on these new technologies have spent most of their time getting the basics to work. With handwriting recognition, that is a set of algorithms that result in reasonably accurate recognition under a variety of circumstances. The same applies to voice recognition. The latest voice and handwriting recognition products running on the latest hardware will yield satisfactory results to a person willing to learn how to use the product and play by its rules.

Where the picture breaks down is in how these new technologies are used and applied. Recognizing handwriting alone is not enough. The software must also work in the environments we generally use, and it must provide easy and natural ways to edit and correct our work. The same applies to voice recognition. Reasonable accuracy in interpreting commands or even dictation is of little value when corrections and editing still require a mouse and a keyboard.

In addition, these alternate input/editing technologies have generally been treated as separate projects. Handwriting recognition tries to do it all: recognition, editing, and correction. Voice recognition likewise tries to do it all with voice. Thus, in addition to a lack of truly functional editing tools in each technology, each also seeks to be the be-all-end-all in replacing the keyboard.

As far as I am concerned each of these alternative input technologies has some advantages and some limitations. Dictating to a computer can result in faster data entry than typing or writing. But using verbal commands to edit, tap, point, and drag is not optimal. A pen is superior for that kind of work. What we have here are two technologies with complementary strengths and weaknesses that have the potential to be combined to form a revolutionary way of interacting with computers.

One such approach I have seen comes from Natural Input Solutions Inc. of Canada. They recognized this dilemma and developed the prototype of a hybrid solution. After analyzing each aspect of existing handwriting and voice recognition technologies they isolated the respective advantages and disadvantages and created their Natural Input project, a solution that combines the best of pen and the voice technologies into a system that is more than the sum of its parts. It is not easy to describe Natural Input in a sentence or two, but even a casual review of the approach reveals great potential for a revolutionary way of communicating with a machine. Not only does Natural Input employ the respective strengths of two alternate input technologies, it also has a number of pen-based editing and correction features that are more intuitive than anything I have seen to date.

This does not mean that Natural Input will take the world by storm within the next year or so. A good deal of work remains to be done. However, the ideas, and research that went into this project/product seem very valuable and we hope to see a commercial product on the market sometime soon.

Contact: Sean Maxted at s.maxted@sympatico.ca