Off Course-On Target

Lurching Towards More Natural Interfaces

In “Coming to our Senses?”  I introduced the topic of finding ways to make better use of our human senses to interact and interface with tools, technology, and machines, and even between ourselves. The LiveScribe Pulse digital pen and paper provides a good example of more “natural” and sense-based ways of interacting, communicating, and connecting.

For more of an overview of this new product, read my article  “LiveScribe Pulse Digital Pen + Paper”. This relatively new addition to digital pen and paper offers exciting capabilities, such as the ability to create sketches and drawings and embed audio explanations or sounds within them, or the ability to write in one language and have the words spoken through the speakers in another language.  Its combination of sight, sound, sketching, and writing reminds me that we are making progress, however slow and lurching, towards more natural interfaces and interactions and in particular, those which are and sensory or sense-based.  In addition, the Pulse pen lets you create some fascinating new "mashups" by mixing and matching written words with sketches, and drawings with speech and sounds.

Deafening Silence Coming to an End?

When you stop and think about it, isn’t it rather odd that we have been working this long with computers and other forms of capturing our thoughts and ideas without much effective use of sounds and speech?  We have the ability to capture audio so clearly and so easily, and yet we are only just now seeing an expansion of its application through podcasts, text to speech, speech to text, and voice recognition.  I suspect we will look back in a few years and view this time as akin to the days when movies were silent or before before audio recording and playback were possible.  Our children will surely ask how we could have ever put up with “silent” notes, sketches and drawings and to have managed to live without the ability to just create whatever interfaces and interactions we wanted on the fly.

In light of this, I’m hoping this article and others to follow, will make you aware of and encourage you to try for yourself, some of the additional new capabilities this digital pen puts into our hands.

Draw Your Own Interface

One of the best new features recently introduced by digital pens such as the Pulse and the Fly Pen is the ability to “draw your own interface”.

I’m fascinated by this overall capability for combining writing, drawing and programming. The Pulse pen lets you create your own interfaces anywhere you want (on digital paper) by just drawing them. For example, you can draw a cross symbol, which the Pulse pen understands for navigation of the pens various features.  You can then use this simple graphic interface to scroll through the programs on the pen, playback, and recording of audio, etc.

Programs can also be developed for the Pulse pen that enable the creation of any kind of interaction between the “ink” and sounds.  For example, a demo has you create a piano by drawing an eight note keyboard, which you can then “play” by touching the pen on the virtual key on the paper and hearing whichever note you touch through the speaker in the pen or via headphones that you can plug into the end of the pen. 

Another powerful capability lets you to write in one language and convert it to spoken words in another.

These examples of "draw your own interface" are limited and minor at this point to be sure, but just start to imagine what this opens up as we begin to draw our own interfaces.  And not just for literal human-computer interfaces but the ability to make anything we draw come to life, draw and play our own instruments, draw and listen to our own stories, etc.  Try out some of these new capabilities yourself.

My Very Own Purple Crayon at Last?!

These capabilities are about the closest thing yet to making one of my all-time favorite children’s stories, Harold and the Purple Crayon become a reality!  If you don’t know this 50-year-old classic by Crockett Johnson, then you should.  It is an absolute "must read "in my book.

In the story, whatever Harold draws with his purple crayon becomes real.  He wants to go for a walk so he draws himself a sidewalk.  He’s hungry, so he draws an apple tree and then adds a ferocious dragon to protect it for him. So scary is the dragon that Harold’s hands start shaking, causing the crayon to create water and waves, so Harold needs to draw a sailboat to climb into...and you get the idea. 

I encourage you to stop right now, and imagine you have this ability to have whatever you draw come to life and interact with you. What do you come up with for some other practical uses?  What does your wild imagination conjure up?   I suspect that you will soon have a list of amazing and truly exciting possibilities, and I hope you will take a few minutes to post them to your own site or share them here.

Natural Interfaces at Last?

These recent advances with digital pen, paper, and sound provide yet another significant marker on our journey towards truly “natural” and sense- based interfaces. I believe that there are some characteristics that are essential and relatively timeless when it comes to anything we humans are going to use effectively to communicate, live, learn, and work together.  Sketching and drawing will continue to be some of the primary means we use to communicate and will be highly effective ways of capturing, sharing, and explaining our thoughts and ideas. What could be more “natural” to us as humans than drawing, writing, and talking?  So too will the spoken and written word, acting, gesturing, storytelling, feeling and forming things with our hands,  and so on. 

Anything and everything that can be done to capitalize and leverage our very human senses and natural ways of interacting, communicating and connecting will be a move in the right direction. Stay tuned for many more postings I’ll be doing on this overall topic and trend towards more natural sense-based interfaces.

Coming to our Senses

If you have been kind enough to listen to me or read some of my entries on OCOT or my other writings, you may have correctly concluded that I suffer from a severe case of Attention Deficit Disorder (ADD). Contrary to popular opinion (and even some “experts”), ADD is not about a LACK of “attention” but rather, an excess of it! 

And Now a Word from My Symptom

Those of us with ADD are unable to pay attention to just ONE thing at a time and are instead forced to pay attention to every sensory input all of the time. We try to make sense of things when EVERY thing is demanding our focus. All of our sensory inputs are dealt with equally and simultaneously. 

What we can’t do, for example, is turn down the volume on every sound except the one we want to focus on. We can't tune out the movements we see around us or choose not feel every input through our skin. In other words, we can only pay attention to EVERYTHING, all at the same time and all at equal levels.

I’m one of the fortunate few with ADD who, long before I knew what it even was, began to develop  coping skills and techniques to survive with these traits. They didn’t seem like anything special to me.  For the longest time, I just assumed everyone else was having the same experience.  And somehow I managed to start turning these “bugs” into features that have enabled me to be as successful as I am and to do what I do. If everything demands your attention and you find everything interesting, then life is very full!

If you are interested in learning more about ADD, look at some of these  alternative descriptions.  And on the topic of attention itself, I know of no better source than that of my dear friend and fellow ADDer Marcia Conner and her Ageless Learner site’s section on Attention and on How Adults Learn.

But now, back to the topic at hand (see what I mean?!)

It All ADDs Up

So why are my ADD tendencies relevant here?  Because its a way to introduce a meta theme that I'd like to address on OCOT. Just as my own natural sensory interface presents challenges, so too are the challenges of developing human/machine sense-based interfaces. Over a series of postings, I'd like to address:

• The use of our human senses as ways through which we communicate and interact with machines, computers, and technology, and ourselves. 
• Other interrelated topics both individually and as a collective whole. 

I’ll do my best to stay "on target" as I wander around "off course" and divide my pursuit of this trend toward more natural and sense-based interfaces into several separate posts. I see it playing a critical role in the future I am both designing and imagining.  

The Missing Link?

Interestingly, this whole topic of natural or sense-based interfaces does not seem to have any common singular term or definition. Or perhaps I just have not found it and some of you can point me to it?  I’m aware of categories such as:

• HCI  Human Computer Interaction/interface
• HMI Human Machine Interface/interaction
• User Interface

But these all seem to be differentiated based on interactions with specific types of technology, computers only, or segmented into individual human senses, such as touch with the use of haptic interfaces and technology with tactile feedback . And I can find definitions of HCI, HMI such as this one from Wikipedia that describe them as:

“...the 'layer' that separates a human that is operating a machine from the machine itself.”

However, my interest is in the larger meta theme of using all human senses and eliminating the “separation layer” between machines and ourselves so that they are invisible and we can directly connect and communicate through our senses. I’m want to ensure that we are taking the more holistic and inclusive view of the use of ALL of our senses, not only because we have so many that are being underused, but also because it is often the combination of multiple senses that make up our best experiences and interactions.

Making Sense of and Better Use of Our Senses

So here’s what I’m going to do. I’ll address the meta theme of sense-based and more natural interfaces through a variety of posts, podcasts, and perhaps some interviews over the next few months. I’ll alternate between the big picture view of the use of senses in general and the use of them collectively, then look at the use of individual senses such as touch and feel, audio, visual, and so on. I will also cover the use of combinations of senses for better, more innovative designs and solutions for interacting with technology...and each other. Add to this the coverage of tools, technology, and examples of real world applications of sensory-based and natural interfaces and I think we’ll have a lot of fun and learn a lot in the process. These posts will be intermixed with other topics on a regular basis and spread over many months for as long as my and your interest continues to be high.

Oh!  Look at that cool blue butterfly that just flew by!  Hmm, I wonder what kind of plane that is flying overhead?  Now what was I saying?  Oh yeah, senses, interfaces, human interaction... 

Hang on, fasten your seatbelts low and tight, we’re about to take off on what should prove to be a fun and very "off course", very "on target" sensory journey!

Experts...like you and me?

In previous postings, I’ve pointed out that we're living in a world of exponential change and that this is affecting all of us in many different ways. You can find my thoughts and references on this with a quick search on exponential change on OCOT as well as my overall article and podcast. 

To continue with this theme, today I'd like to explore how this world of exponential change has affected our concept of expertise and experts. Has it changed as well? What do you do when you need some assistance and there are no experts to be found based on our traditional definitions? 

The rapid introduction of technologies such as entire new programing languages, new software and hardware; or on the social side, things like social networking, un-conference event models, and so on, challenge our understanding of expertise,  and our traditional models and techniques for obtaining expertise and for finding those who have it.

While it is often true that there are very few completely new ideas and so many very new things are based on very old ideas, previous work, models and products, it still seems as if we are seeing more that are truly new and different. Being so new, we have no Ph.D programs,  certification programs, or other credentials in place, and in many cases, not very much experience or expertise at all because these technologies  and models are new to everyone. It helps in part that large scale adoption of anything new typically takes 10 or more years (see this “Long Nose” post by Erik Duval for more on this) but it often takes even longer for us to establish the traditional forms of denoting expertise and creating the experts themselves. What do we do in the meantime and even more so what do we do as the rate of change trends ever upwards?  Will our traditional models continue to work at all? For example:

• What does this mean for “boards” or other committees of experts for an organization, company, association?  If these are becoming less effective what is the better choice?
• In the absence of any “formal” credentials, how to you find and deploy individual experts and consultants? 
• Is there a value to a “panel of experts” at a conference?  If so, who should be on the panel?
• How does our understanding of new more contrarian models such as “wisdom of the crowds” and the “Blink” approach affect our definition and use of experts?

You get the idea, and I’m sure you can come up with your own list as well. 

Finding Solutions

I quite like and often quote Einstein’s observation that:

"We can't solve problems by using the same kind of thinking we used when we created them."

More often though, I think our problem is that we are restricted by our thinking which as been shaped by the previous paradigm and so we search for solutions to the new paradigm from within the old. As we seek to find and develop solutions to this new challenge of identifying and finding expertise and experts, we can take advantage of the way other things are growing exponentially.  For example:

• The degree to which these technologies can help us find and connect with others who are focused on the new “thing” we are interested in.  There are usually many more experts and more expertise out there than we assume or were previously able to find.
• The increased exposure more and more of us are having to new products, programs and ideas and therefore pursuing our curiosity with them thus taking the first steps towards developing more expertise.
• The increased access and availability for learning about and experimenting with these new ideas, products and services.  This is especially true in the form of the more informal learning we are increasingly doing through our daily forms of (re)search, reading, listening and otherwise consuming the exponential growth of “content” available to us.

Experts "R" Us!

So how do we find the right experts and expertise at the right time?  First, we need to get better at finding expertise that often goes unnoticed or remains unknown to us.  If you don’t know it’s there, then it essentially doesn’t exist!  In my experience, we often fail to find experts because it doesn’t appear in most people’s job description or title.  Have you ever had the experience of meeting someone AFTER a project was over (typically in an informal setting) and realizing that they have this incredible expertise and experience? Have you caught yourself saying something like “Where were you when I needed you?”  And worst of all, did you find that they were right under your proverbial nose, right next door or even already working for or with you?

I’ve long advocated that we need to work to dramatically scale up the amount of metadata we have for each of us. By this I mean the enormous list of individual attributes that define and describe us as unique individuals—deeply detailed metadata inventories of our skills, competencies, experiences, aptitudes, attitudes, knowledge and so on—and have it all being dynamically updated on a constant basis as we acquire new skills, have new experiences as well as those we lose from lack of use.  Just imagine how valuable it would be to find just the right person at just the right time when you need to add someone to a project team or have a question you need answered or an idea to test out.

Exponential Growth of Experts?

And perhaps most of all, we need to consider the potential of developing more expertise within each and every one of our exponentially expanding world population (currently 6.7 Billion and counting!).  We need to learn to see and find the expertise in all of us, both collectively and individually. By collective, I am thinking of the growing awareness and understanding of the latent expertise that lies within a large group of people. This idea is nicely captured in James Surowiecki’s book “Wisdom of the Crowds”.  This is increasingly happening as we become more and more connected to more and more people and things. 

Individually I am thinking that if something is very new, then an “expert” could be defined as whomever has the most experience—often the individuals who have been early adopters, high risk takers, and those who jumped in from the beginning and have the most experiential learning as a result.  If you believe, as do I, that every one of us has some degree of expertise and our numbers and our amount of expertise is growing at an ever faster rate, then therein lies much of the solution to the problem of finding experts for the exponential growth of new ideas, products, and services. It all sounds very “loopy” (consider the source!), but it seems to me that we can indeed invert Einstein's observation and best find solutions for a new paradigm from within the NEW paradigm.

We probably need to get much better at learning in real time as we jump in and try out these new products, services and ideas.  We need to get better at doing a lot of post-mortems to analyze what worked, what didn’t would be smart.  Getting over our apparent reluctance to admit to or review failures, and instead learn from what doesn’t work well. If increasingly more things are very new and we have less and less experience with them, should we not expect to be in an almost constant state of exploration and discovery and expect “failure” to be more common than success? 

I don’t mean "failure"in a negative sense, nor do I mean that failure should become the goal. Rather, learning from what doesn’t work seems in my experience to be at least as valuable or more so than that which does. We’ve long known about the value of “practice” but usually separate and distinguish this from “the real thing” and regard practice as what we do to get ready for the real performance.  Perhaps we now live in a world where ALL we do is practice and strive for continuous improvement.

But I also believe in the old adage that “the more things change the more they stay the same”, and so we will continue to have the more traditional types of experts who are deeply specialized.  They will remain highly valuable and sought after.  However, given the enormity of the number of problems we have to solve, both large societal ones and small daily ones, and the fact that these are growing exponentially, we need, as they say,
"all the help we can get."  And I for one simply can’t imagine how we can be effective in solving problems through linear approaches, such as continuing to depend upon our traditional model and definition of expertise. 

As we start to live and work in this world of exponential change, we need to think exponentially as well. We need develop solutions and models which scale exponentially. Rethinking our definition of expertise and experts to be on more of a sliding scale and where we assume that each and every one of us have valuable expertise to offer seems to me the best direction to head as we search for more experts like you and me!

Designing Your Own Future!

Carrying the rather magnanimous title of “Strategic Futurist”,  I try to always clarify that I have no crystal ball, no special powers for prediction of the future (nor any other special powers for that matter!), but I do believe that “The best way to predict the future is to invent it.” as Alan Kay said.  However, I have become somewhat skilled at discovering and observing trends and patterns that I believe will help us to shape and design our futures to be bright and positive ones.  Today I’d like to highlight some recent examples that add credibility to this model of designing your own future, and I for one could not be happier with the progress and trend. 

It seems that the topic of the future is particularly popular of late.  In my recent posting “Inventing the Future?” I wrote about the fascinating new joint venture between Jane Congeal and the Institute for the Future called Superstructure, "the world’s first massively multiplier forecasting game" which is about to start September 2008 . With a tag line of “Superstructure! Play the game, invent the future”, you can see why I was immediately attracted.

Rather than fatalistically accepting that the future as something that happens to us, I believe that we can both individually and collectively invent and design our own!  I also see this as part of using serendipity as a serious strategy for success.  More on that later.

Web of Fate; NOT!

Mark Alert recently (July 16, 2008) did a good job of highlighting several of these new examples of what I would see as “designing your own future” in his e-Clippings: Learning As Art blog post called Forecasting the Future:  Prediction Markets & MMOG's.  In particular, check out the Web of Fate(Wof).  Despite the title, I see WoF as serendipitous and by design rather than fatalistic. It is, in their own words:

“.. a social experiment that harnesses the collective intelligence of the web to visualize and uncover hidden relationships among future and historical events."

They do this by providing everyone (yes you too!) the opportunity to add their own thoughts in a variety of ways. You can make your own predictions, quote a prediction from someone else you agree with, set a future goal, or create a time capsule.  Web of Fate also offers another way of looking at this with their “Semantic Future: explore the meaning of everything”,  accomplished by their semantic analysis engine that extracts meaning from historical, present and future events and draw connections among them.

Since I'm a very visual thinker, I particularly liked the way that WoF is synthesizing all of “our” input to create the Web of Fate visualizations.  Given enough input, I think these will go a long way towards helping us spot trends and patterns that I find so enlightening to the process of designing your own future. 

World Mural Project:  Youth is Not Always Wasted on the Young!

But I’ve saved the best for last.  Chris Dawkins, another blogger I recommend, wrote a terrific summary of the “Intel World Mural project - just what will the next 40 years bring?”.  Chris is behind the Ziff Davis blog on Education and I’ve written more about him and his blog in my previous post “Scan, scan, scan as fast as you can”. I recommend that you subscribe to his feed, if you don’t already.

But the treat today is to direct your attention to the World Mural Project, which is part of Intel’s celebration of their 40th anniversary as a company (July 18, 2008) when they will unveil what they describe as:

“...one of the world’s largest web based digital murals designed by youth.  More than 530 young people from 21 countries around the globe share their vision on what computers will allow us to accomplish in the next 40 years.”

To see for yourself, just click the “View Mural” link on the World Mural site. I hope you will have as much fun as I did exploring the various visions from the young contributors.  The interface is fun and whimsical, and it allows you to move around to different countries and regions and then click on the individual contributions.

Chris pulled out this quote from Intel spokesperson Agnes Kwan, which gives a great glimpse into the future that some of these “kids” see:

"Some of the themes that the kids have expressed are: having multi-function, wearable computers, achieving a greener environment on Earth, greatly improved healthcare, vastly improving our brain power, experiencing virtual education, communicating through holograms, teletransport, living more relaxed, peaceful lives, and living on Mars.”

Also worth checking out is the Intel Computer Clubhouse through which the mural project was developed.  This network is an after-school community-based technology learning program that enables youth in underserved communities to acquire tools necessary for personal and professional success.  The Clubhouses were an ideal way for youth to collaborate on the mural and create a dynamic and unified piece of art that represented a global world view.

I think this is also a great example of another trend that I highlighted in my post “Future Sources of Innovation, Discovery and Design” where I see more and more innovation and ideas coming from the underdeveloped and underserved groups and regions of the world (many of them located inside otherwise very “developed” regions and countries, I might add!).  In any case, a very good use of your time would be to check out both the World Mural project itself and the Computer Clubhouse project.  Consider getting involved yourself as a volunteer, mentor or trainer in these projects.

Who better to involve directly in designing our future than those who are going to spend the most time in it and who likely have entirely new and fresh ideas and perspectives than those of us who may be more often shaped by the past?

I particularly liked the line and the link found in the Web of Fate site:  “The Solution:  You have the solution”    OK, OK, so it’s a form and you know I subscribe to the “all forms must die” attitude, but in this case the potential for the benefits of your input outweigh the format and I’d strongly encourage you to add yours to the collection and check back regularly to see how our future is shaping up!

Make This Personal!

I hope you too find some of these new examples inspiring and insightful and that you will check them out.  However, all of these are acting at a very high level. Most change occurs much closer to the ground and with more specific applications.   Therefore perhaps most important and valuable is the potential for YOU to adopt some of these examples to your own work and areas of interest.  What is to stop you from creating your own game along the lines of Superstruct to drill down on designing the future on a much more focused level of your own industry, profession, hobby or other areas that you are passionate about?  It “only” takes a good initial premise or scenario to cause some of that incendiary “Spontaneous Creative Combustion” in your own organization. How could you involve younger minds in helping to see and shape the future of these worlds you are a part of?  How could you instigate an outbreak of input and feedback within your groups and utilize the semantic analysis from something like the Web of Fate to spot trends and directions that will help you direct and design the future?

At the risk of beating this theme to death, let me restate that I am NOT interested nor optimistic about these examples I’ve highlighted for their predictive capabilities, but rather for their assistance in helping each and every one of us to play a part in designing a future that is both bright for all and is one we can be proud of and part of.

But most importantly I’d like to know how do YOU see the future shaping up?    You know where I stand but how do you see it?  Bright or dark?  Is the future fatalistic of serendipitous?  Done to you or something you have a hand in?

Cool Tools: LiveScribe Pulse Digital Pen and Paper

I truly believe that every one of us (yes, that includes YOU!) has cool tools that we use on an everyday basis and stories about how we use these tools and how we came to find and use them. I'd like to hear about yours.

This week’s Cool Tool, the new Pulse digital pen from LiveScribe, is still undergoing my evaluation since I’ve only had it about a month now, but it has already lived up to its predecessor, the now discontinued io2 digital pen from Logitech, which I nominated as a previous Cool Tool.  I can see that this new Pulse pen has much more to offer, so I thought it worthy of bringing to your attention. 

Some of the more significant new features that the Pulse has over its predecessors are:

• The addition of audio recording & playback
• The ability to draw your own interfaces
• The option for additional programs and features via a developers API

These features intrigue me and seem to have so much potential that I’m going to address them in a separate posting.

I got this new digital pen (thanks Kelly!) in part to replace, as I already mentioned, my beloved and previous Cool Tool, the io2 digital pen from Logitech. For reasons that still elude me and which Logitech seems to deny and not want to discuss, the io2 has been discontinued. Logitech is not providing new drivers for the io2 so that it can work on Vista, so I’ve been having to use an old XP machine to keep my notes. Plus, the digital paper is no longer available and my supply is almost out. 

I continue to champion and use digital pens and paper because I still find that writing and sketching on paper is the best option in many situations and ensures that I always have a digital version for better sharing of my notes and sketches and for the assurance of a backup copy and long term storage.  The new Pulse pen from Livescribe has proven to provide all these features so it makes for a good replacement, but what makes it worthy of an additional Cool Tools posting is that it adds some additional new capabilities which have truly surprised and excited me with their potential as I’ll explain shortly.

About the LiveScribe Pulse Pen

OK, let’s get the info about the pen itself out of the way so we can get into what and why I think this is so “Cool” and worthy your consideration. I’ll let you do your own reading of the many reviews about the Pulse pen, but I can recommend that you check out these reviews in Engadget by Ross Rubin:

• Switched On: Livescribe's hot recording artist seeks mass appeal (Part 1)
• Switched On: Livescribe's hot recording artist seeks mass appeal (Part 2)
• Switched On: More options for getting from scribble to screen which also covers two other new digital pens and approaches

You can also check out the interactive demo and overview on the Livescribe site, which has links to other articles and reviews by PC Magazine, USA Today, etc.

Not surprisingly (read on), the Pulse pen is based on Anoto digital paper technology, which utilizes an almost invisible micro grid of small unique digital dots covering each piece of digital paper that is then recognized by the infrared camera in the end of the pen.

The list of items that come with or are inside the pen itself include: a rechargeable battery, internal microphone and speaker, USB port, an LED screen (currently quite underused I think), and a set of ear buds and microphone. 

The included PC desktop software (Windows only at this time, Mac version said to be in development) copies all your written and audio content whenever you plug the pen into the USB cradle to recharge and synchronize with all your previous notes and recordings.  Since this content is now stored as files, you can view, share, and edit them further. From within the Livescribe desktop, you have the option to upload your files (text only, audio only or both) to your own online space for either your own access or for sharing this with whomever you choose to grant access.

 

The digital paper comes in a variety of formats, including a very nice leather bound “Moleskin”-like version which I like the most so far.  Each notebook also contains a set of additional capabilities, which are printed inside the covers or at the bottom of each page. These include an impressive calculator with full trig, financial, and scientific functions, as well as audio controls, book marking and a full keyboard. 

The pen itself comes in 2 versions with either 1MB or 2MB of internal storage. Each MB is capable of holding about 100 hours of audio (at the lowest setting), as well as all your written notes and any additional programs you choose to install. I have the 2MB version and  after taking both audio and "written" notes regularly and frequently for more than a month, it still has over 1MB available.  You can delete what is in the pen’s internal memory every time you put the pen in its USB cradle and synchronize it with your PC.

The Man Behind the Curtain/Pen

Almost as interesting as the pen is the man behind it all, Jim Marggraff.  Until last year, Jim was the CEO of Anoto, the Swedish company that developed the technology used by the vast majority of digital pens (I think), including of course this latest Pulse pen from LiveScribe. At 48, Marggraff has already achieved tech world fame and fortune as the inventor of the LeapPad and the FLY Pentop. 

I can highly recommend the Wired article from back in November 2005 called  “LeapFrog’s Wild Ride” that will tell you the interesting story about Marggraff’s journey from being the CEO of Anoto to building the fastest growing toy company in history, Leapfrog and the Fly Pen.

The Fly Pen grows up?

I was an early customer of the Fly Pen back in 2005 and in many ways, I see the new Pulse pen as a “grown up” version of the Fly Pen that was aimed more at children and students, though I should note that there is now a newer version of the Fly Pen that is called the “Fusion”, which has a lot of improvements over the original. 

I was very disappointed when the Fly Pen was not a breakout success, since I thought it had so much potential. I bought several of these for friends to play with and gave them to several who had young children. But for a variety of reasons, it just didn’t catch on with either adults or kids. After experimenting with it and also after talking with others, my take was that the Fly Pen and its interface was just a bit too complex and non-intuitive. Most people who tried it never got past that critical first few minutes and figured out how to use it and unleash its power. But for me the potential is still HUGE and these are just part of the steps along the way to eventual success, as is usually the case with truly revolutionary technologies and inventions. Hence my delight when I found that the Fly Pen had in many ways evolved into this new Pulse pen.

New Mashups: Highly Annotated Notes & Sketches

Of all the many things that are capturing my curiosity with this new Pulse pen, I must say that the connection between writing and audio is at the top of the list. With the Pulse pen, you have the option to easily record audio (just turn on record with a tap of the pen) with the built-in microphone and then listen either through the surprisingly good built-in speaker or plugging in the supplied ear buds and microphone. This latter accessory is full stereo BTW and provides what Livescribe hyperbolically describes as “3D sound”.  

But for me the real power lies in the combination of controls for finding and playing the audio, and the ability to create mashups of written notes and accompanying audio. This has already had a significant effect on the way I take notes. Because I now know that the full audio of the conversation I’m having with someone in a meeting is being recorded, I can and do take much more condensed notes, simply jotting down the key words, phrases, or just marking a key point in the conversation. What I can do afterwards is touch the pen to any word or mark in my paper notes and start hearing the audio that was present at that time. It's very cool and useful. Taking notes in meetings, lectures, and presentations is a great application of this Cool Tool.

I’m also finding myself writing my own notes differently.  I now often turn on the record feature as I’m jotting down some of my thoughts and ideas and then talk while I’m writing to provide more details in my notes, reducing the amount of writing I need to do. 

These added audio capabilities have also enabled me to be much more productive when my hands are otherwise busy, such as when I’m driving. In these cases I use the “pen” as more of a digital audio recorder. In this situation, I just use the pen and paper to help with later access by making a few marks or one word notes for reference. The recording function also lets me capture external sounds that are going on around me that are related to what I’m writing about and want to share, such as music, birds singing, street noises, etc.

Perhaps even more valuable is its ability to create voice-annotated interactive sketches and drawings. You can talk while you are drawing to explain details, symbols, and background.  It gives you the ability to synchronize locations on the drawing with the audio so you can create a drawing whereby you can tap on a spot and hear the audio explanation that goes with it. Simple yet powerful. 

For example I recently had the opportunity to tour the Mayan ruins of Copán in Honduras. I was able to record the guide talking while I took some written notes and made some sketches of the layout of the ancient Mayan city. I could also go back and add my own audio explanation to my sketches from additional details I had learned from reading plaques and from spending a few hours in the museum.  The result is a series of annotated sketches I can share with others or refer back to and help out with those “senior moments” when I can’t remember details from this fascinating tour of the Mayan ruins.

On my wish list for future releases is the ability to manipulate and edit the tech and audio portions independently. For instance, it would be much better if I could take a piece of audio, such as a recording of the guide in the Copán ruins, and trigger it to play when I tap a spot on a sketch I create later. This would be much like the way we can currently move and manipulate text and photos in blog entries and documents.

I’ve got the 2MB version, which gives me the ability to capture up to 200 hours of audio. This capability encourages me to turn on the audio recording feature and add audio to almost all my notes, a response that is similar to the “if in doubt shoot” approach many have adopted since the advent of digital cameras where there is little or no cost to taking more pictures, and you might just get that perfect shot!

On the down side, I guess this also leads to a society full of people who appear to be “talking to themselves” with the advent of hands-free mobile phones with wireless Bluetooth head sets, Skype calls, audio interfaces, and now digital audio note taking.  So PLEASE practice and encourage considerate audio etiquette whenever you are “talking to your technology”!  Microphones are typically very sensitive and do NOT require yelling to be understood!

Share More

LiveScribe enables you to share your work as an attachment to an e-mail, a posting on Facebook, or for downloading as a PDF. The company is also hosting what they call “Pencasts” that let you put your written plus audio notes up on the web.  This could be quite a useful feature for capturing, sharing and reusing lecture notes (and the lecture itself!) or presentations and meetings.  I expect this will lead to some interesting additions to blogs, web sites, and social sites such as Facebook and MySpace.

Here for example is a Calculus 191 lecture on the Livescribe site. While the recording is not the best quality (note: you can change the setting of audio recording quality), I rather like that it takes only the push of one button on the pen to turn it on and one tap of the pen to instantly start recording everything you write and any audio happening at the same time. What matters is that it works and is simple and easy for anyone to do. 

Create Your Own Digital Paper

One other key advancement I’m anxiously awaiting is the ability to print your own “digital paper”. This capability has been mentioned several times by Marggraff and on the LiveScribe site, so I’ll keep watching.  As you come to understand what you can do with digital pen and paper, you’ll start to see almost infinite and exciting possibilities. For example, you could create personal or professional “business cards” that would include either digital places to write on or the ability to understand commands, such as “email this” or a small calculator or...

Key to the Future: Extending the Pen with Software

In my estimation, the highest potential for the future of the Pulse pen is the ability to create additional programs and applications which utilize the pen’s capabilities. One tantalizing example is a demo of a translator function where you can write in one language and have it be translated and “spoken'” via the speakers in another language! A demo included with the pen has the ability to translate from English to Spanish, Mandarin, Arabic or Swedish for a limited set of 20 words, such as hello, goodbye, coffee, chocolate, beer, and numbers from 1 to 10.  It has been great fun to try out even this very limited set of words with some of the people who only speak Spanish here in El Salvador. Just start to imagine the potential this has when there is a much larger vocabulary and selection of languages!  For example, you could be in a taxi giving directions and write down what you want to say and then have the pen “speak to” the driver. Or you could hand the pen and paper to someone and have their writing translated for you, so you can communicate with them despite language differences. We’re still a ways from the “Bablefish” translator of Douglas Adam’s imagination but the feature of the Pulse pen definitely elevates it to the Cool Tool status in my view.

I’m fascinated by this overall capability for combining writing, drawing and programming. The Pulse pen lets you create your own interfaces anywhere you want (on digital paper) by just drawing them. For example, you can draw a cross symbol, which the Pulse pen understands for navigation. You can then use this simple graphic interface to scroll through the programs on the pen, playback of the audio, etc. For example, a demo has you create a piano by drawing an eight note keyboard which you can then “play” by touching the pen on the virtual key on the paper and hearing whatever note you touch through the speaker in the pen or via headphones you can plug into the end of the pen.

While these demos are just limited examples, the potential of being able to have hand drawings and symbols create interfaces and trigger actions offers truly unlimited potential and power in my mind.

I should note that these abilities for hand drawn interfaces and even the piano and translation demos were in the original Fly Pen and they generated most of my initial excitement. I’m therefore hoping that both Livescribe and especially external individual developers will be bringing us a flood of additional software programs that we can add to our digital pens and paper to extend their functionality and use more.  This is where the developer’s kit and API are so critical for the future of this pen.

On the Livescribe web site there is what appears to me to be a reasonably complete set of tools, including a “content development kit” (CDK) and a software developers kit (SDK), which allows for others to create add-on programs and games. Just as was the case with operating systems initially and now the web, this ability to extend the basic functionality and open it up for independent extensibility will be key to determining the future success of the Pulse pen and others to follow. Initially, just as is the case with games, this will be depend upon the degree to which independent developers take up the charge to create add-ons and applications that extend the capabilities and applications of this new technology. But over time, my greatest hope is that this capability to “draw your own" interface and programs will enable any of us, regardless of our programming skills, to create just what we need. 

The Pen Ever Mightier than the Sword?

As I re-read what I’ve written above, it doesn’t sound like much to be able to capture handwritten notes, sketches and audio.  But as with most major value propositions, I often find that the simplest things turn out to be the most valuable and powerful. While I think this new Pulse pen is but one example, this ability to both capture, find, control, and replay the mixture of text, images, and sound in a very simple and intuitive way—pen and paper—is extremely compelling and powerful.

I hope some of these new capabilities of digital pen and paper ignite your imagination as much as it does mine and that you’ll look into these digital pens and paper.  As I said previously, the success of this technology will depend in part on the degree to which we see the development of add-on applications and capabilities.  However, it will ultimately depend upon the degree to which all of us see the potential of digital pen, paper, and audio and begin to adapt, adopt, and apply it to innovative and creative uses.

What do you think about digital pens and paper?  Are you using any yourself?  Would it make it to your Cool Tools list?  It usually takes months or years of regular use for something to make it to my  Cool Tools list, but the Pulse pen is definitely already there.  Free your imagination and I’m sure you can already start coming up with many other exciting ways to apply and extend these new capabilities of pen and paper, along with the mashups you can create by adding audio into the mix. I’ll be back with more wandering and pondering about digital pen and paper and especially the combination of audio, writing, and what I see as the evolution toward truly natural interfaces. Hope you’ll stay tuned!

***********

Have  "Cool Tools" you use and want to share?  Send me your favorites with the following information:

• Name of the tool and source for getting one. 
• What does it do? 
• How did you come to discover this? 
• How do you use it? 
• Why is it on your top ten "Cool Tools I Use" list?