Off Course-On Target

World Languages: Going Colorblind?

"Maps courtesy of www.theodora.com/maps used with permission"

In case you find it as interesting as I did, here is the list (according to www.geographic.org) of the top ten most popular languages in the world. See the map above and this full list of the top 50 languages.

1. Mandarin	(1120 Million)
2. English	(480M)
3. Spanish	(332M)
4. Arabic	(235M)
5. Bengali	(189M)
6. Hindi	(182M)
7. Russian	(180M)
8. Portuguese	(170M)
9. Japanese         10. German	(125M) (98M)

** note this list was updated Oct.1 to correct errors in list tabulation that omitted Spanish!  Apologies for confusion and thanks Kean for correcting!

As is often the case with these kinds of lists and large scale estimations, we need to keep in mind that there are many different ways of counting and measuring things, such as the amount of people who speak a given language. Do you only count the first or native language?  Do you group dialects together?  What constitutes fluency? To prove my point, I've picked two other equally interesting lists of language by population that you can find here and here. No doubt there are many more to be found by simply searching online.

What these lists and maps don't show us is how these languages are spreading throughout the world based on both the movement of people from their country of birth to other countries for work or through marriage for example, as well as the spread of languages via people learning new ones, organizations setting up offices, factories, and other operations around the world, etc. 

In the next 25 years or so, especially with the augmentation and acceleration of language-based technology and translation, I wonder if we are headed for a world that is "colorblind" when it comes to languages, and the need for these maps becomes somewhat obsolete?

Lurching Towards More Natural Interfaces

In “Coming to our Senses?”  I introduced the topic of finding ways to make better use of our human senses to interact and interface with tools, technology, and machines, and even between ourselves. The LiveScribe Pulse digital pen and paper provides a good example of more “natural” and sense-based ways of interacting, communicating, and connecting.

For more of an overview of this new product, read my article  “LiveScribe Pulse Digital Pen + Paper”. This relatively new addition to digital pen and paper offers exciting capabilities, such as the ability to create sketches and drawings and embed audio explanations or sounds within them, or the ability to write in one language and have the words spoken through the speakers in another language.  Its combination of sight, sound, sketching, and writing reminds me that we are making progress, however slow and lurching, towards more natural interfaces and interactions and in particular, those which are and sensory or sense-based.  In addition, the Pulse pen lets you create some fascinating new "mashups" by mixing and matching written words with sketches, and drawings with speech and sounds.

Deafening Silence Coming to an End?

When you stop and think about it, isn’t it rather odd that we have been working this long with computers and other forms of capturing our thoughts and ideas without much effective use of sounds and speech?  We have the ability to capture audio so clearly and so easily, and yet we are only just now seeing an expansion of its application through podcasts, text to speech, speech to text, and voice recognition.  I suspect we will look back in a few years and view this time as akin to the days when movies were silent or before before audio recording and playback were possible.  Our children will surely ask how we could have ever put up with “silent” notes, sketches and drawings and to have managed to live without the ability to just create whatever interfaces and interactions we wanted on the fly.

In light of this, I’m hoping this article and others to follow, will make you aware of and encourage you to try for yourself, some of the additional new capabilities this digital pen puts into our hands.

Draw Your Own Interface

One of the best new features recently introduced by digital pens such as the Pulse and the Fly Pen is the ability to “draw your own interface”.

I’m fascinated by this overall capability for combining writing, drawing and programming. The Pulse pen lets you create your own interfaces anywhere you want (on digital paper) by just drawing them. For example, you can draw a cross symbol, which the Pulse pen understands for navigation of the pens various features.  You can then use this simple graphic interface to scroll through the programs on the pen, playback, and recording of audio, etc.

Programs can also be developed for the Pulse pen that enable the creation of any kind of interaction between the “ink” and sounds.  For example, a demo has you create a piano by drawing an eight note keyboard, which you can then “play” by touching the pen on the virtual key on the paper and hearing whichever note you touch through the speaker in the pen or via headphones that you can plug into the end of the pen. 

Another powerful capability lets you to write in one language and convert it to spoken words in another.

These examples of "draw your own interface" are limited and minor at this point to be sure, but just start to imagine what this opens up as we begin to draw our own interfaces.  And not just for literal human-computer interfaces but the ability to make anything we draw come to life, draw and play our own instruments, draw and listen to our own stories, etc.  Try out some of these new capabilities yourself.

My Very Own Purple Crayon at Last?!

These capabilities are about the closest thing yet to making one of my all-time favorite children’s stories, Harold and the Purple Crayon become a reality!  If you don’t know this 50-year-old classic by Crockett Johnson, then you should.  It is an absolute "must read "in my book.

In the story, whatever Harold draws with his purple crayon becomes real.  He wants to go for a walk so he draws himself a sidewalk.  He’s hungry, so he draws an apple tree and then adds a ferocious dragon to protect it for him. So scary is the dragon that Harold’s hands start shaking, causing the crayon to create water and waves, so Harold needs to draw a sailboat to climb into...and you get the idea. 

I encourage you to stop right now, and imagine you have this ability to have whatever you draw come to life and interact with you. What do you come up with for some other practical uses?  What does your wild imagination conjure up?   I suspect that you will soon have a list of amazing and truly exciting possibilities, and I hope you will take a few minutes to post them to your own site or share them here.

Natural Interfaces at Last?

These recent advances with digital pen, paper, and sound provide yet another significant marker on our journey towards truly “natural” and sense- based interfaces. I believe that there are some characteristics that are essential and relatively timeless when it comes to anything we humans are going to use effectively to communicate, live, learn, and work together.  Sketching and drawing will continue to be some of the primary means we use to communicate and will be highly effective ways of capturing, sharing, and explaining our thoughts and ideas. What could be more “natural” to us as humans than drawing, writing, and talking?  So too will the spoken and written word, acting, gesturing, storytelling, feeling and forming things with our hands,  and so on. 

Anything and everything that can be done to capitalize and leverage our very human senses and natural ways of interacting, communicating and connecting will be a move in the right direction. Stay tuned for many more postings I’ll be doing on this overall topic and trend towards more natural sense-based interfaces.

Coming to our Senses

If you have been kind enough to listen to me or read some of my entries on OCOT or my other writings, you may have correctly concluded that I suffer from a severe case of Attention Deficit Disorder (ADD). Contrary to popular opinion (and even some “experts”), ADD is not about a LACK of “attention” but rather, an excess of it! 

And Now a Word from My Symptom

Those of us with ADD are unable to pay attention to just ONE thing at a time and are instead forced to pay attention to every sensory input all of the time. We try to make sense of things when EVERY thing is demanding our focus. All of our sensory inputs are dealt with equally and simultaneously. 

What we can’t do, for example, is turn down the volume on every sound except the one we want to focus on. We can't tune out the movements we see around us or choose not feel every input through our skin. In other words, we can only pay attention to EVERYTHING, all at the same time and all at equal levels.

I’m one of the fortunate few with ADD who, long before I knew what it even was, began to develop  coping skills and techniques to survive with these traits. They didn’t seem like anything special to me.  For the longest time, I just assumed everyone else was having the same experience.  And somehow I managed to start turning these “bugs” into features that have enabled me to be as successful as I am and to do what I do. If everything demands your attention and you find everything interesting, then life is very full!

If you are interested in learning more about ADD, look at some of these  alternative descriptions.  And on the topic of attention itself, I know of no better source than that of my dear friend and fellow ADDer Marcia Conner and her Ageless Learner site’s section on Attention and on How Adults Learn.

But now, back to the topic at hand (see what I mean?!)

It All ADDs Up

So why are my ADD tendencies relevant here?  Because its a way to introduce a meta theme that I'd like to address on OCOT. Just as my own natural sensory interface presents challenges, so too are the challenges of developing human/machine sense-based interfaces. Over a series of postings, I'd like to address:

• The use of our human senses as ways through which we communicate and interact with machines, computers, and technology, and ourselves. 
• Other interrelated topics both individually and as a collective whole. 

I’ll do my best to stay "on target" as I wander around "off course" and divide my pursuit of this trend toward more natural and sense-based interfaces into several separate posts. I see it playing a critical role in the future I am both designing and imagining.  

The Missing Link?

Interestingly, this whole topic of natural or sense-based interfaces does not seem to have any common singular term or definition. Or perhaps I just have not found it and some of you can point me to it?  I’m aware of categories such as:

• HCI  Human Computer Interaction/interface
• HMI Human Machine Interface/interaction
• User Interface

But these all seem to be differentiated based on interactions with specific types of technology, computers only, or segmented into individual human senses, such as touch with the use of haptic interfaces and technology with tactile feedback . And I can find definitions of HCI, HMI such as this one from Wikipedia that describe them as:

“...the 'layer' that separates a human that is operating a machine from the machine itself.”

However, my interest is in the larger meta theme of using all human senses and eliminating the “separation layer” between machines and ourselves so that they are invisible and we can directly connect and communicate through our senses. I’m want to ensure that we are taking the more holistic and inclusive view of the use of ALL of our senses, not only because we have so many that are being underused, but also because it is often the combination of multiple senses that make up our best experiences and interactions.

Making Sense of and Better Use of Our Senses

So here’s what I’m going to do. I’ll address the meta theme of sense-based and more natural interfaces through a variety of posts, podcasts, and perhaps some interviews over the next few months. I’ll alternate between the big picture view of the use of senses in general and the use of them collectively, then look at the use of individual senses such as touch and feel, audio, visual, and so on. I will also cover the use of combinations of senses for better, more innovative designs and solutions for interacting with technology...and each other. Add to this the coverage of tools, technology, and examples of real world applications of sensory-based and natural interfaces and I think we’ll have a lot of fun and learn a lot in the process. These posts will be intermixed with other topics on a regular basis and spread over many months for as long as my and your interest continues to be high.

Oh!  Look at that cool blue butterfly that just flew by!  Hmm, I wonder what kind of plane that is flying overhead?  Now what was I saying?  Oh yeah, senses, interfaces, human interaction... 

Hang on, fasten your seatbelts low and tight, we’re about to take off on what should prove to be a fun and very "off course", very "on target" sensory journey!