There have been
some really interesting developments and concepts in the human-computer
interface area recently. This is one of my favorite areas
of technology to play with. It’s just plain fun and I can’t
think of another area where the future is available for several years
before it hits the mainstream market. Plus, there’s just something exciting about trying to bridge the virtual and the real. I own a
3D glove, VR goggles, and the first commercial multi-touch
interface. If we had the room I’d love to get a virtual reality ball to play with 
3D movement tracking a la what Johnny Lee is doing with the Wii
hardware is just insanely cool, and I have to think that it will find a
useful niche in human-computer interactions somewhere–even if only in
games.
Some of these ideas are fun and some are clunky, but I like playing
with all of them because I think it shows you where things are headed
long-term–once the kinks are worked out. Witness the multi-touch
interface called iGesture I bought like 6 years ago and the current raging success of the iPhone/iPod Touch.
Of course, SOFTWARE user interface innovations are much easier and
cheaper to make as they rely solely on good design taste. It’s
actually funny to me that Apple’s eventual insane profitability and Web
2.0 have transformed the computer industry from a bunch of nerds who
like keyboard shortcuts and command lines into ar-teests who discuss
minimalism and color palettes as much as any interior designer. There
could easily be a Web Design Star TV Show.
"Ohhhh my, look at that menu, is that actually just TEXT? We’ll
need to get some rounded buttons in here to soften things up, STAT.
Oh, a grey background… no he DIDN’T. Somebody spilled 1998 all OVER
this place!"
But aesthetics aside, there have actually been some great user interface concepts to come out of the Apple/Web 2.0 era.
Here are some of my favorite user interface innovations I’ve seen lately.
Natural Language Processing
The Mozilla bookmarking and history concept video
demonstrates looking for activity history using queries such as
"yesterday, office, around 2 pm, eric
sent" The natural language search is a really nice touch whenever it’s
implemented, and one that
could be useful in many more areas than it currently is used–life
streams definitely seem like an ideal area here. In fact, really good
text indexing like Google does will usually suffice in this area. It
reminds me a lot of how (why) I use Gmail (I’m looking for an email
sent to Jessica last year, regarding taxes? Gmail can do that), and
why I have come to loathe
Outlook in comparison. When we were designing our business
intelligence application we quickly realized that our users loved how
easy it was to run a report for yesterday, last month, etc, rather than
having to spell out dates. People
like the ability to interact as they would speak to another human, but
it assumes a certain level of intelligence on the side of the engine.
Natural language search also opens the door to using voice as an interface, which I’m really high on as well (see Jott).
Voice is really a terrific low-effort way for humans and computers to
interact, heinous uses of IVR systems have just numbed us to this fact.
Visual Item Recognition
The human mind really is amazing at image recognition, and that’s rarely used. Mozilla’s
new concept videos use thumbnails to display search results and history. When I first looked at the concept video,
I reflexively thought that visual search results couldn’t possibly work
better than text. However, after further thought, I think that if you
incorporated real-time zooming and
panning, a la the iPod touch interface, it would actually be a great
way to cover a much bigger swath of search results than textual results
like Google’s allow, relatively quickly. Photosynth technology, which
allows you to start from the 50,000 foot view of thousands of documents
and zoom into specific high-resolution areas would be very helpful here
as well.
Network Graphs
Being able to graph your network is just plain old fun. I almost
think it’s hard to grok a network without seeing it. If you use
Twitter but haven’t checked out www.tweetwheel.com
yet you should go do it–it graphs your Twitter network and it’s
interesting to see how many people you follow also follow other people
you’re following. Networks which grow exponentially, like social networks, are inherently
hard to grasp–displaying them in a visual format like this makes it
easier.
OpenLink Software has created a nifty Linked Data add-on for FireFox which shows you the data behind the page in a data graph, exposing
the relationships between concepts in the document and other data. It would be really cool to work an RDF relationship
browser into an interface as polished as this Mozilla concept. I’m envisioning
that in the middle of browsing a page, I could zoom out to a network graph with that page at the
center, and getting a visual thumbnail cloud of all the pages and
concepts that the page links to. That would be just groovy.
Mobile Interfaces
The iPhone/iPod Touch has done a great service
to humanity by showing us that intuitive and easy to use interfaces are
indeed possible on a small piece of screen real estate. The touch
interface and accelerometers have also shown us that there doesn’t HAVE to be a
barrier between the phsyical world and the digital, and in fact the
more you blur that line the easier a device is to use.
As I mentioned in a previous post,
I think that all user
authentication will eventually be done via your mobile device. Log
into a Web site on your laptop and you will be prompted to authenticate
via thumbprint on your mobile device. You shouldn’t need to have any
kind of biometric input device duplicated on your desktop or laptop.
To death with USB dongles, I say, let’s converge the important stuff in
our mobile devices and be done with it.
Because of the limited real estate on a mobile device, touch
screens are almost a necessity. Some people still favor a phsyical
keyboard, but I’ve used a Blackberry and an iPod Touch, and I have BIG 
hands,
and I think those people are just reluctant to change. Apple’s
auto-complete based on QWERTY works really well. Actually, though,
probably the fastest way to do text input right now is via fold-out
keyboards. I had a Nokia 6800 with a fold-out keyboard once, and it
was really cool. However, I would place my bets on mobile devices
eventually
projecting a full-sized keyboard onto the surface they’re sitting on
and doing heavy-duty text input that way.
But for now, I think Apple has done a commendable job with mobile
text input. Touch screens also allow multi-touch interfaces, which
brings me to…
Multi-Touch Interfaces
Multi-touch
interfaces are still in their infancy, but already they have completely
obliterated every other type of interface, especially in the mobile
space.
One of the nice things about multi-touch interfaces is that they
allow you to drag things off of the screen to access hidden
functionality. Aza Raskin uses this to great effect in the FireFox
Mobile concept video to
hide browser functionality offscreen while viewing a page, leaving more
screen real estate available for content display. I love the idea of
not having to look at my options until I’m done with the task at hand.
Gestures
Intuitive, self-teachable gestures are definitely
where touch interfaces are going. The iPhone uses these and when you
pick it up you don’t even realize you’re issuing a command to the
computer. You have to think about pressing Ctrl + to zoom in, but
pinching your fingers on the screen is intuitive and natural. The
iGesture pad can learn new gestures, so once it’s configured you can
pretty much manipulate things on the screen Minority Report-style
(albeit without the nice display).
It’ll be interesting to see how this space evolves and which of
these ideas go mainstream. I’m taking apart Wii controllers while I
wait
UPDATE: Coincidentally, I just saw this really cool article along similar lines, and they have a few interesting interfaces that I hadn’t seen before.
