Back in 2000 I had the honor of writing a vision paper for ASTD and the National Governor's Association called "Into the Future". I included some of my thoughts and excitement at the time about the notion of augmenting our eyes by having additional layers of information overlaid on top of what we are "really" seeing. I described things like being able to walk around the ruins of an ancient city and be able to see drawings, photos and video of what it would have looked like hundreds of years before. In the years since my excitement has continued to grow and I’ve written more and more articles on this and all other forms of augmenting our “cognitive capacities” and there have been more and more advances on transforming this vision into reality.
Google has just released an update on their work on “Project Glass” and some new prototypes they are testing with wider audiences now. One of their new examples can be seen in these photos for a pair of frames that sit above your eyes and provide information by looking up or another pair of more traditional glasses where the information is projected onto the clear glass lenses.
Nick Bolton provides a more comprehensive overview of this type of augmented reality in his NYT “Bits” article “Google begins testing its Augmented Reality glasses”. Personally I’m more excited and interested in future models Nick also notes of wearable and imbedded technology such as the contact lenses pictured here. I’ve posted previous articles with more of my thoughts on these such as this one on “Augmented Reality in Contact Lenses”.
Personally I will also welcome and be first in line for things like embedded ear buds that would give me additional sound and control to augment what I am hearing and a similar form of embedded microphones to augment my speech and vocal output. Begin to imagine having a camera, earphones and microphone literally “on you” or “in you” at all times?!
I’m pretty confident that groups of bright people all over the world will continue to develop every better augmented reality technology at an exponential rate and in all cases I think the greater challenge befalls the rest of us to begin figuring out how to use these well and how to deal with the inevitable pitfalls and more negative consequences all great innovations and inventions bring with them.
For example how will we deal with this significant additional increase in the information at our avail? In some of the more negative initial reactions to Google’s update some have already begun to worry about the likelihood that our eyes will be “augmented” with even more marketing images and this is a legitimate and inevitable concern.
However these new forms of information delivery and consumption are inevitable IMHO and so let’s just get over it and get on with it! WE get to chose how to use these new tools and information influx. As Clay Shirky put it so well; “There’s no such thing as information overload, just filter failure”. While there is no question we have lots of progress to make in overcoming “filter failure”, we ARE making progress and it seems to me that failure is the root of most learning so it is all good.
So let’s continue to develop better ways of filtering information in the Snowflake Effect tradition of getting just the right information for just the right person at just the right time in just the right context, and so on. We certainly have as much more more learning and creative thinking to do in coming up with new habits, conventions and techniques that will ensure that exponential innovation also leads to exponentially better living and learning!
Let’s start thinking about what an incredible way of augmenting our learning these kinds of “glasses” can be by enabling us to add additional information to what we are seeing before us. And note how Google’s Project Glass also includes a wearable camera. This makes taking pictures and video more instantly available anywhere, anytime and more transparently for us in that we would always have a camera available to capture serendipitous moments and memories by simply blinking our eyes or some other simple gesture. While there is always the “scary” part of how this can and will be used, this capability will also let us continue to live more IN life’s moments, enabling us to learn and live life more fully in the process.
It will take us many years to develop new conventions for all these kinds of new technology as well such as how do we best ‘interface” with them for things like gesture based control with our eyes. How do we learn to use these to improve our connections with others and not interfere? Gee, if you think people’s behavior with the use of cell phones and texting in the middle of a conversation are a problem just imagine what it will be like when we don’t even have to look away or down to see that new call coming in!!
As usual, it all comes back to and down to each of us snowflakes. WE are the ones who are responsible for our own behaviors and need to learn how to use and control the tools and technology we are surrounded by such that they increase our connections and conversations with the other special snowflakes around us.
I for one could not be more excited by both the new technology arriving daily and the new challenges and learning that lies before us and in many ways is already here. The best way I know of to counter FUD (Fear, Uncertainty & Doubt) is to run TOWARDS it rather than away from it by increasing our curiosity and learning of things we don’t know.
Note: I am much better at starting things than finishing them and have this bad habit with starting postings that never get posted. This one has been sitting in my drafts folder for several months now and long overdue to finally get to you so here it is. The iPad announcement has since come and gone and will help fuel some of what I'm looking for but as you'll read I'm still optimistic and holding out for a future covered in "digital goo". - Wayne
Seems I’m on a bit of a print sprint of late (see previous List with a Twist posting for example), though I’m surely spending more time in front of a screen than ever. Today’s update is about the future of magazines and an update on my long standing quest and wait for the arrival of true digital paper or digital “goo”.
I LOVE Magazines!
I read a lot of magazines—more than one a day—on a very wide range of topics. And I’m one of those silly people who reads every page of the magazine, including all the ads. So as digitally inclined and geeky as I may be, let’s get this out of the way up front: I LOVE magazines.
Things I love (and want to retain) about magazines:
Focused topical areas
Relative, informative, and clever (sometimes) advertisements
Great glossy color photos, graphics
Serendipitous discovery of cool new ideas, products, etc.
Reading anywhere I’m sitting, having it in my hand/lap rather than on my desk
Just the right size/heft for handheld/lap reading
Easy on the eyes for extended reading
Cleared for take off and landing times! (reading anywhere such as planes)
No cords or batteries
However, as with any love or relationship, here are a few things I don’t like so much and would like to see improved:
Physical delivery format limits me to mail subscriptions, magazine stands, etc.
Weight. On my last flight 28 lbs/12.7kg of my bags were magazines!
Wastefulness. Apparently 15% sale of magazines in bookstands is considered high!
Static images only
Read only
Requires good external lighting
No easy search/find capabilities, especially across issues
No connections to related items of interest
No cut, copy, paste, share, keep (I used to have boxes full of torn-out magazine pages)
No disctionary or lookup
No easy way to archive them
Can I do some of this reading and resolve some of these problems on my laptop, tabletPC, smartphone, eReader? Yes, but not well, and with no where near the same reading experience of real magazines.
I Love Books Too
I don’t equate magazines at all with books, though books too are another love. Many of the items on my like/don’t-like list apply to books as well, but my Kindle has resolved most of my don’t-like list items and it has increased my book reading quantity and quality. I’ve still got some key items on my wish list for the next eBooks version such as a touch screen interface, handwritten notes, etc., but the ease of getting new books no matter where I am, access to so many (including almost all of the classics for free), linked note-taking and having my entire library with me all the time (including manuals for all my boat equipment) has been one of the best experiences in the past few years.
And for me, Kindle passes the most important test of all for book reading—I can lose myself in the story. By this I mean I am unaware that I’m reading, paper or otherwise and I'm in the story or in my thoughts as I read! Much of this is due to the very paper-like screen technology (E Ink in the case of Kindle and many other eReaders) as well as its baby bear (not too big, not too small) size and weight.
eMagazines?
So far, eReaders and tablets don’t work too well with magazines…at least not for me. I’ve tried quite a few and continue to try more of them—both software and hardware based. But with magazines, I’m looking for a different experience: color is critical, so is size of the images (larger), and yet not larger in weight and size in my hand/lap. However, I’m seeing some help on the horizon for my magazine reading and in the short term (next year), I think some of the new digital formats could provide some near-term solutions.
Check out the video below for something we’ll almost assuredly see in the next few months, and which has some promise. If you’re like me, you’ll need to look beyond the content of this example (Sports Illustrated), but I do like much of what I see in the demo (thanks to Dan Pink for the tip). Peter Kafka also has a posting on this: “Game On: Time Inc. Shows Off a Tabletized Sports Illustrated”
I am especially intrigued by the interface and the nice mix of old and new with such things as retaining all the familiarity of the old, a print- and paper-based magazine, blended with new capabilities, such as richer content with video and audio, search, rearrangement, sharing, and linking. This video is well worth 3 minutes of your time. Have a look and see what you come away with?
Note that pretty much everything you see in this demo, and what I’m most intrigued by are the changes in content and interface features. However just as important as these will be the issues of the hardware and the physical attributes of what comes next.
Here is another example of a different set of perspectives on what the future of eMagazines might look like. It’s worth watching to see what these researchers and developers assessed to be some of the essential elements of the magazine reading experience which we want to retain and what we want to avoid.
I’ve been a tabletPC champion and user for over 10 years (bought one of the first Apple Newtons too!) and still believe that tabletPC-based features, especially screen-based features such as multi-touch, “flippable”, handwriting recognition and input, and so on will become standard features on all laptops and all screens for that matter. At this point in time, tabletPCs are the best and almost only choice for any digital magazine reading I do, but the benefits I get from magazines are not there. Much of the problem is due to the physical limitations of the device, too thick, heavy, and clunky. And there is still no help with subscriptions, integrated note taking, integration of content across devices, common formats, ease of access, and very limited content beyond what is in the print versions.
It is no coincidence then that there is so much rumor and hype surrounding things like the upcoming “iPad” and rumblings from other hardware sources. It may also bode well that some of those in the publishing world are taking a more future-oriented view of these developments, such as is demonstrated by Time Inc. in the video above and by this more recent announcement ofWired's upcoming version for the iPad. Adobe is also rumored to be developing a publishing tool and magazine reader for tablet devices. And here is a recent list of "5 Things That Will Make E-Readers Better in 2010". At the very least, 2010 promises to be a significant year for more options that may help deliver some of my wish list items for magazine-type reading.
Digital Goo: The Only Book or Magazine I Ever Need?
As excited and optimistic as I am that we will see all of the above, and more over the coming year, I’m holding out for a completely different development—digital goo. This is what I’ve been asking for and evangelizing about for more than 20 years. Digital goo will be the advent of “real” digital paper, virtual paper or whatever we come to call it, and more importantly, will be a major shift in the way we think of, access, create, and consume content.
Here is an excerpt from an old paper that will give you an idea of what I have in mind for “digital goo”. I wrote this back in the 1990’s, so you’ll have to pardon the black and white and television orientation, but I think you’ll get the idea. It is as simple as it is profound, and yes, I’m still waiting.
… imagine a substance at the molecular level where each molecule is a tiny sphere where one half of its surface is black and the other half white. Each black and white sphere can easily be controlled by electrical input so that either the white or the black side is facing up. Now, mix this substance into paint or wood pulp or plastic and you suddenly have the ability to make ANY surface digital and capable of displaying ANY image you’d like with almost infinite resolution.
You can see the possibilities. Imagine binding a few pages of digital paper together to create truly digital books! Oh, and note that the images on this digital paper can move, so suddenly that you can be watching “TV (the content) in a book.” Just throw in some other colors with those black and white molecular spheres and we’ve got color screens literally EVERYWHERE! An equally frightening and exciting vision for most of us, I suspect. Based on what I’ve been privileged to see in research labs for the past few years, the introduction of this technology into the marketplace can’t be far off.
Therefore, I’m hoping that the next iteration of eBooks will consist of a few pages of real digital paper bound into the only book/magazine/paper I’ll ever need. These pages will be made by adding some “digital goo” to the regular production of paper, or paint, producing sheets of relatively ordinary looking and feeling paper which are now essentially “just” digital surfaces capable of displaying any content at near infinite resolution including video and animations. This same digital paper could also accept inputs such as touch and handwriting.
For me at least, I still want multiple pages, and I think we may see a renaissance of books as a format and form factor in this regard. Maybe it is just a function of my rather severe ADD affliction, which results in my flipping back and forth through multiple pages of a magazine that I’m reading. I bring most of my laptops to their knees because at any one time, I usually have more than 30 tabs running in my browser, and more than 10 other apps running at the same time. But I think we all have versions of this problem, and having multiple pages, both for just larger display areas (think centerfolds and flip outs) as well as speed of access, means that this will become a very common and highly desirable feature.
Research is going on to support much of this and I’m sure that like most things, it will be here sooner than we expect and also take longer than we expect for us to take advantage of.
Peter Kafka in another related posting on Condé Nast’s Offering for Apple’s Mystery Tablet: Wired Magazine finished with the following paragraph that seems just so apropos for the world of exponential change we are living in and a fitting end to this post:
“But all of this assumes that consumers, who’ve shown no inclination to pay for this stuff on the Web, will be willing to pay for it once it appears on devices no one owns yet. We’ll find out soon enough.”
Here’s a list of innovations that really caught my attention as this list takes a refreshingly human and personal look at the future and also manages to live up to this headline of being both profound and innovative. Well worth your time to read though I’ll warn you in advance that there are lots of links and each one is worth following and using up even more of your valuable time and attention.
Notice how many of the innovations featured in this list are already happening and add to my What if the impossible isn’t? list. As you reflect upon what you read here I think you will see how this adds to my conviction around The Snowflake Effect and that our future is a giant snowstorm of extreme mass customization for every person every day by making everything deeply personal and unique. Read on and see what you think?
By Jeremy Hsu TechNewsDaily 29 January 2010 1:11 PM ET
Today's world looks increasingly like the future. Robots work factory assembly lines and fight alongside human warriors on the battlefield, while tiny computers assist in everything from driving cars to flying airplanes. Surgeons use the latest technological tools to accomplish incredible feats, and researchers push the frontiers of medicine with bioengineering. Science fiction stories about cloning and resurrecting extinct animals look increasingly like relevant cautionary tales.
Thought control is definitely on my What if the impossible isn’t? or WITITT? list and here is a recent addition to the list of examples.
I suspect we will do our usual underestimation on this whole area of brain direct or thought controlled interfaces in that we will underestimate how long it will take to mainstream but more so we will underestimate the impact it will have. In any case while it is early in the development don't forget that history shows clearly that adoption and most every other type of change increased exponentially. We therefore need to be thinking of things much sooner than in the past so we we will be ready to adopt and leverage these new enablers as they emerge from the lab and head increasingly quickly towards everyday use by the rest of us.
Consider this latest example and start pondering how we will use this to interact with and control our environments and interactions with information, technology and each other.
Winter Olympics to Demo Thought-Controlled Lighting
Along with the figure skating, ice hockey and snowboarding, another event will compete for attention at the Winter Olympics in Canada this month.
A Canadian company has created what it calls the “largest thought-controlled computing installation.” It’s an experiment that lets visitors to the Olympics use their brainwaves to control the lights at three major landmarks in Canada, including Niagara Falls.
“When people put on the headsets and find themselves increasing the brightness of the lights by just thinking about it, you can almost see their brains explode,” says Trevor Coleman, chief operating officer for InteraXon, the company that has created this installation.
As consumers get more comfortable with going beyond the keyboard and the mouse to interact with their computers, companies are looking for alternate ways to make the experience better. Already, touch and voice recognition have become a major part of the user interface in smartphones, and harnessing brainwaves or other biological data is slowly emerging as a third option, especially in gaming.
What caught my eye (ear?) was not so much the on screen piano this provides but the clever use, or more correctly re-use and re-purposing of video and timelines. This piano example by itself is a bit of a novelty and one which wears of soon. However, start to imagine how you could use this for other applications? Think about a new level of interactivity that anyone can add to any of the increasing volume of video that is being posted and a new way of creating and re-using/re-purposing video content. Try using this on a touch enabled screen such as your iPhone or Android.
Perhaps it helps to think of this along the lines of "scanning" rather than needing to recreate from scratch, to almost instantly capture that which already exists and use this as the raw components to create whatever YOU have in mind.
As the fidelity of audio and video continues to increase the gap between real/original and simulated/recorded lessens to the point of being indistinguishable.
For me, this is yet another example of how we are transforming from consumers to creators and to a society of "ProSumers" the likes of which McLuhan, Nevitt and the Tofflers so presciently described back in the 70's. And of course I also see this as more examples of how we are also becoming a society of "Tearners" (Teacher/Learners)
Give this video piano example a try and see if you start to hear the future as you play?
As I sit here typing on my keyboard and clicking my mouse, I find myself desperately hoping we may finally be moving beyond our narrow view of what form interfaces between humans and technology can take to one where we can be more creative and innovative...and effective!
Seems to me that today, almost every interface is limited to no more than a variation on more button pushing! Computers, for example, come with keyboards and mice, and we even carry this metaphor of button-pushing onto the screen with “virtual buttons” that we have to "push". And even smaller devices, such as mobile phones, are designed with button-driven interfaces.
Recently we’ve transferred button-pushing over to “touch” screens on phones and we think that this is just "oh so cool' and innovative. Give me a break! Most examples don’t even work as well as a real button! Ever tried to type more than a word or two on a touch screen keyboard?
Enough with the buttons already!!
There is a glimmer of hope, however, in the world of interfaces! Some of the new phones from Samsung (currently only available in China) use Immersion’s VibeTonz technology, a tactile feedback that simulates the feeling of pushing a mechanical button even though the surface is completely flat and does not move. It is the latest example of a new breed of haptic technologies that do for the sense of touch what high definition displays and surround sound do for eyes and ears. (see links at end for more on haptic technologies)
But I believe that the lack of feedback in touch screen devices will severely limit their success and adoption, and that someday we will shake our heads when we remember how we lived “out of touch” for so long without the use of touch in interfaces.
Hope on the Horizon?
But surely we can be more creative and innovative with our interactions and interfaces than limiting every solution to simply a variation on more button pushing! Currently we are seeing innovative uses of touch screens that provide a better interface and interaction with our digital world. Applications on the iPhone and other smartphones, for example, let you flick or flip things around, such as a set of digital photos or stacks of music CD artwork. This use of “gestures” as a way of interacting has great promise and recent examples, such as the Wii controller and some of the multi-person, multi-touch interfaces on digital surfaces are showing great promise.
Since touch works in both directions—as feedback to us and feedback from us—I'm delighted to see the use of haptic interfaces with the Wii controller, which generates physical feedback to your hands as you strike a tennis ball or golf ball, for example, and simultaneously uses your physical input to provide velocity and angle as you swing a golf club and the resultant force applied to the ball. Airplane and car simulation game controllers (joy sticks, steering wheel, brake & gas pedals) also provide similar haptic feedback. Flight simulators for training pilots and amusement park “virtual reality rides” show us how truly real, natural, and effective touch interfaces can be.
Let’s Stop Flapping and Start Flying!
But these incremental improvements, like touch screen buttons and limited haptic feedback, also serve to continue our "flapping" behavior of mimicking physical button pushing. Our tendency to focus on copying what worked well in the past, such as designing planes that mimic flapping wings on birds or pushing buttons for machines, is in fact holding us back from truly "flying"! If you are not familiar with my use of this metaphor of "flapping vs. flying", take a few minutes to read or listen to my story on “Confusing Flapping with Flying”.
To start truly "flying", that is developing truly innovative solutions, I think we need to focus on the use of touch and tactile feedback as some of the essential elements of successful interactions and interfaces. For example, the ability for us to feel and touch any object even though it is not “really there” and similarly for our technology to acquire the sense of feel and touch.
Less Virtual, More Real
Early applications of virtual reality used to seem pretty cool but now they seem, as my daughter liked to say to me, “Oh so yesterday, Dad!”. These early efforts failed because they were too virtual and too far from reality. Today, we can see clearly how much of a difference it makes when, for example, motion, sound ,and color is added to movies. More recently ,the addition of some 3D, high definition, and high fidelity has narrowed this "reality gap" to our eyes and ears. And virtual reality applications required all that extraneous equipment, such as headsets, goggles, helmets, gloves and even full body suits, to provide the feedback to or from us, which severely separate or distanced us from the reality of the situation.
So, what if we moved towards being more real and more natural? What if we had digital models with haptic interfaces that let us feel, handle, turn, assemble and disassemble them? How about applying full haptic feedback to a holographic images? Imagine if holographs provided not just stunning visual 3D reality, but also came with haptic feedback to your hands, so that you could feel their edges, curves, and shapes?
Digital Clay: Digital Prototyping gets Real
Moving along the reality scale, we already have very functional and bidirectional digital prototyping via 3D scanners and 3D printers. Together these can both create digital models from existing objects (input) as well create new fully-functional physical objects (output). Such digital prototyping can all be done BEFORE we start using large amounts of materials and energy to produce the “real things” ** and before we use the energy and resources to create these physical entities. This capability may well turn out to be one of the single greatest ways of conserving resources and energy and bringing some much needed practical reality to the “green” movement.
Leaping (and lurching) further forward, wouldn’t it be great have true virtual digital prototyping that gave us an experience equal to that of having the real thing in our hands? How about if we could move beyond digital clay to “real clay” where haptic sensors and generators were built-in at the nano level or could be more like the holographic + haptic models I mentioned previously. With this digital clay in your hands, you could now create any shapes and objects you like "by hand" or you could add some "clay" to your 3D printed models or you could use 3D printing to create a whole model out of digital clay and use this as the starting point of your design work or testing.
AND, now how about if we also hook up some of that digital clay “backwards”, so that rather than limiting the feedback to one direction—from our hands to the clay— it could also move from the clay to our hands?!
Say what, Wayne?
Well, imagine you are on a team whose members reside in many different locations. Your team is designing some physical object, such as a new smart phone. Rather than “just” having a 3D printer create the latest design iteration of the phone on your desk as a “solid”, what if every team member had one made out of digital clay and whenever you changed the phone's shape or moved some buttons, for example, the other phones at the other locations changed accordingly. You each would have a working version.
THAT sounds a lot more real and a lot less virtual to me, and it's all possible by adding the use of our sense of touch to our interactions with technology.
** BTW, this example starts to show how increasingly strained the concept of “real” is becoming and what I see as another trend towards the closing and ultimate elimination of the “reality gap”. But I’ll expand more upon this in a future posting.
eSkin
The integration of sensory feedback and control within technology provides another look at touch working in reverse (so to speak), and it is equally important and fascinating. One recent example is the progress being made with “e-Skin” for robots. This new material is the first that combines two key characteristics:
The ability to stretch like rubber
The ability to conduct electricity like metal into a single substance
They’ve accomplished this with carbon nanotubes and used it to allow robots or machines to “feel” heat, pressure, and texture. I was equally intrigued with e-skin's potential to to stretch into larger sizes and onto curved non-flat shapes. I can imagine how this could lead to having displays in your pocket that you can pull out, unroll, and stretch into a large scale screen for viewing, or applications as "skin" grafts for burn victims.
Hiding Everything but the Benefits
Some have suggested that Jules Verne completely missed the computer and all its technology in his futuristic books such as 2000 Leagues Under the Sea, and Around the World in Eighty Days. But I think Verne got it completely right by NOT focusing on or imagining computers and technology. He simply assumed that the compute power was just “there” behind the scenes. I loved the way he kept the focus on the fantastic capabilities and the grand adventures we would have if such capabilities existed. Who cares how or what enables them?
So it is my belief and my hope that we are finally seeing technology dissolve into the background and into ourselves so that the focus can be on people, challenges, solutions, and great adventures in life. I know I'm signed up for the pursuit of joy and as such I welcome and champion any and all movement towards more natural interfaces and the use of all of our human senses to communicate and work more effectively with each other and with our supporting cast of machines and technology.
In “Coming to our Senses?” I introduced the topic of finding ways to make better use of our human senses to interact and interface with tools, technology, and machines, and even between ourselves. The LiveScribe Pulse digital pen and paper provides a good example of more “natural” and sense-based ways of interacting, communicating, and connecting.
For more of an overview of this new product, read my article “LiveScribe Pulse Digital Pen + Paper”. This relatively new addition to digital pen and paper offers exciting capabilities, such as the ability to create sketches and drawings and embed audio explanations or sounds within them, or the ability to write in one language and have the words spoken through the speakers in another language. Its combination of sight, sound, sketching, and writing reminds me that we are making progress, however slow and lurching, towards more natural interfaces and interactions and in particular, those which are and sensory or sense-based. In addition, the Pulse pen lets you create some fascinating new "mashups" by mixing and matching written words with sketches, and drawings with speech and sounds.
Deafening Silence Coming to an End?
When you stop and think about it, isn’t it rather odd that we have been working this long with computers and other forms of capturing our thoughts and ideas without much effective use of sounds and speech? We have the ability to capture audio so clearly and so easily, and yet we are only just now seeing an expansion of its application through podcasts, text to speech, speech to text, and voice recognition. I suspect we will look back in a few years and view this time as akin to the days when movies were silent or before before audio recording and playback were possible. Our children will surely ask how we could have ever put up with “silent” notes, sketches and drawings and to have managed to live without the ability to just create whatever interfaces and interactions we wanted on the fly.
In light of this, I’m hoping this article and others to follow, will make you aware of and encourage you to try for yourself, some of the additional new capabilities this digital pen puts into our hands.
Draw Your Own Interface
One of the best new features recently introduced by digital pens such as the Pulse and the Fly Pen is the ability to “draw your own interface”.
I’m fascinated by this overall capability for combining writing, drawing and programming. The Pulse pen lets you create your own interfaces anywhere you want (on digital paper) by just drawing them. For example, you can draw a cross symbol, which the Pulse pen understands for navigation of the pens various features. You can then use this simple graphic interface to scroll through the programs on the pen, playback, and recording of audio, etc.
Programs can also be developed for the Pulse pen that enable the creation of any kind of interaction between the “ink” and sounds. For example, a demo has you create a piano by drawing an eight note keyboard, which you can then “play” by touching the pen on the virtual key on the paper and hearing whichever note you touch through the speaker in the pen or via headphones that you can plug into the end of the pen.
Another powerful capability lets you to write in one language and convert it to spoken words in another.
These examples of "draw your own interface" are limited and minor at this point to be sure, but just start to imagine what this opens up as we begin to draw our own interfaces. And not just for literal human-computer interfaces but the ability to make anything we draw come to life, draw and play our own instruments, draw and listen to our own stories, etc. Try out some of these new capabilities yourself.
My Very Own Purple Crayon at Last?!
These capabilities are about the closest thing yet to making one of my all-time favorite children’s stories, Harold and the Purple Crayon become a reality! If you don’t know this 50-year-old classic by Crockett Johnson, then you should. It is an absolute "must read "in my book.
In the story, whatever Harold draws with his purple crayon becomes real. He wants to go for a walk so he draws himself a sidewalk. He’s hungry, so he draws an apple tree and then adds a ferocious dragon to protect it for him. So scary is the dragon that Harold’s hands start shaking, causing the crayon to create water and waves, so Harold needs to draw a sailboat to climb into...and you get the idea.
I encourage you to stop right now, and imagine you have this ability to have whatever you draw come to life and interact with you. What do you come up with for some other practical uses? What does your wild imagination conjure up? I suspect that you will soon have a list of amazing and truly exciting possibilities, and I hope you will take a few minutes to post them to your own site or share them here.
Natural Interfaces at Last?
These recent advances with digital pen, paper, and sound provide yet another significant marker on our journey towards truly “natural” and sense- based interfaces. I believe that there are some characteristics that are essential and relatively timeless when it comes to anything we humans are going to use effectively to communicate, live, learn, and work together. Sketching and drawing will continue to be some of the primary means we use to communicate and will be highly effective ways of capturing, sharing, and explaining our thoughts and ideas. What could be more “natural” to us as humans than drawing, writing, and talking? So too will the spoken and written word, acting, gesturing, storytelling, feeling and forming things with our hands, and so on.
Anything and everything that can be done to capitalize and leverage our very human senses and natural ways of interacting, communicating and connecting will be a move in the right direction. Stay tuned for many more postings I’ll be doing on this overall topic and trend towards more natural sense-based interfaces.
If you have been kind enough to listen to me or read some of my entries on OCOT or my other writings, you may have correctly concluded that I suffer from a severe case of Attention Deficit Disorder (ADD). Contrary to popular opinion (and even some “experts”), ADD is not about a LACK of “attention” but rather, an excess of it!
And Now a Word from My Symptom
Those of us with ADD are unable to pay attention to just ONE thing at a time and are instead forced to pay attention to every sensory input all of the time. We try to make sense of things when EVERY thing is demanding our focus. All of our sensory inputs are dealt with equally and simultaneously.
What we can’t do, for example, is turn down the volume on every sound except the one we want to focus on. We can't tune out the movements we see around us or choose not feel every input through our skin. In other words, we can only pay attention to EVERYTHING, all at the same time and all at equal levels.
I’m one of the fortunate few with ADD who, long before I knew what it even was, began to develop coping skills and techniques to survive with these traits. They didn’t seem like anything special to me. For the longest time, I just assumed everyone else was having the same experience. And somehow I managed to start turning these “bugs” into features that have enabled me to be as successful as I am and to do what I do. If everything demands your attention and you find everything interesting, then life is very full!
If you are interested in learning more about ADD, look at some of these alternative descriptions. And on the topic of attention itself, I know of no better source than that of my dear friend and fellow ADDer Marcia Conner and her Ageless Learner site’s section on Attention and on How Adults Learn.
But now, back to the topic at hand (see what I mean?!)
It All ADDs Up
So why are my ADD tendencies relevant here? Because its a way to introduce a meta theme that I'd like to address on OCOT. Just as my own natural sensory interface presents challenges, so too are the challenges of developing human/machine sense-based interfaces. Over a series of postings, I'd like to address:
The use of our human senses as ways through which we communicate and interact with machines, computers, and technology, and ourselves.
Other interrelated topics both individually and as a collective whole.
I’ll do my best to stay "on target" as I wander around "off course" and divide my pursuit of this trend toward more natural and sense-based interfaces into several separate posts. I see it playing a critical role in the future I am both designing and imagining.
The Missing Link?
Interestingly, this whole topic of natural or sense-based interfaces does not seem to have any common singular term or definition. Or perhaps I just have not found it and some of you can point me to it? I’m aware of categories such as:
HCI Human Computer Interaction/interface
HMI Human Machine Interface/interaction
User Interface
But these all seem to be differentiated based on interactions with specific types of technology, computers only, or segmented into individual human senses, such as touch with the use of haptic interfaces and technology with tactile feedback . And I can find definitions of HCI, HMI such as this one from Wikipedia that describe them as:
“...the 'layer' that separates a human that is operating a machine from the machine itself.”
However, my interest is in the larger meta theme of using all human senses and eliminating the “separation layer” between machines and ourselves so that they are invisible and we can directly connect and communicate through our senses. I’m want to ensure that we are taking the more holistic and inclusive view of the use of ALL of our senses, not only because we have so many that are being underused, but also because it is often the combination of multiple senses that make up our best experiences and interactions.
Making Sense of and Better Use of Our Senses
So here’s what I’m going to do. I’ll address the meta theme of sense-based and more natural interfaces through a variety of posts, podcasts, and perhaps some interviews over the next few months. I’ll alternate between the big picture view of the use of senses in general and the use of them collectively, then look at the use of individual senses such as touch and feel, audio, visual, and so on. I will also cover the use of combinations of senses for better, more innovative designs and solutions for interacting with technology...and each other. Add to this the coverage of tools, technology, and examples of real world applications of sensory-based and natural interfaces and I think we’ll have a lot of fun and learn a lot in the process. These posts will be intermixed with other topics on a regular basis and spread over many months for as long as my and your interest continues to be high.
Oh! Look at that cool blue butterfly that just flew by! Hmm, I wonder what kind of plane that is flying overhead? Now what was I saying? Oh yeah, senses, interfaces, human interaction...
Hang on, fasten your seatbelts low and tight, we’re about to take off on what should prove to be a fun and very "off course", very "on target" sensory journey!
Google has just released an update on their work on “Project Glass” and some new prototypes they are testing with wider audiences now. One of their new examples can be seen in these photos for a pair of frames that sit above your eyes and provide information by looking up or another pair of more traditional glasses where the information is projected onto the clear glass lenses.
For example how will we deal with this significant additional increase in the information at our avail? In some of the more negative initial reactions to Google’s update some have already begun to worry about the likelihood that our eyes will be “augmented” with even more marketing images and this is a legitimate and inevitable concern. 
Recent Comments