1,072 Posts served
4,471 Conversations started
How do you represent "icon" as an icon?
By Clay Breshears (Intel) (75 posts) on March 26, 2008 at 3:19 pm
If you were illiterate, how would you interact with a computer? Some folks at Microsoft have been addressing that issue with a project in Bangalore, India. A Computerworld article describes some of the ideas and problems that a team of researchers have experienced with trying to allow users that cannot read or write (including about 50% of the Indian population) the opportunity to interact with computers. Obviously, it has to be pictures for the interface and there will be cultural differences that will require different representations for the same idea.
Using gestures, rather than a mouse and keyboard, would be a natural method for folks to interact with the keyboard. Researchers from two Fraunhofer Institutes demonstrated how hand waving and finger pointing could be used to operate a computer. Cameras watch the user's gestures and interpret the intention. Here again, certain gestures don't translate across all cultural boundaries, so we might see a system that needs to first learn a user's intentions.
All this is really cool work. It is certainly more useful (and less scary) than having Braille keys on a drive-up ATM. However, as I got to thinking about it more, I began to wonder what someone that couldn't read or write might do if they had easy access to a computer. I thought about all writing and working with numbers I did every day. That would be out.
I decided that it would have to be something visual. Something to do with pictures or movies. My mind quickly ran to the trailer for the latest Bollywood feature, slide shows of paintings by van Gogh or Renoir, and travelogues of distant countries. Unfortunately, my mind just as quickly ran to examples of still and moving pictures on the Internet that tend to be rated by a letter between "W" and "Y".
Since I can't seem to get my mind out of the gutter, what kinds of content would you think anyone that can't read or write would want to "use" on a computer? And what kind of icon (picture) would you use to represent that content?
Categories: Uncategorized
Comments (3) 
By Will Pearson on March 28th, 2008 at 8:16 am
Hi,
This is interesting and it could probably have wider applications than just the illiterate. The labels that are used in the mostly natural language interfaces that we use today can often be ambiguous. In turn this leads to users having to spend time learning what the labels mean and an increase in user error. Pictures are often less ambiguous than natural language and replacing natural language with pictures might lead to a more efficient and less error prone user interface that users have to spend less time learning how to use.
I agree with the point about the need to develop different icons for different communities; however, this can be equally true for natural language as different language communities exist throughout the world and ultimately we all have our own individual idiolects. The fact that this isn't an issue in UI design at the moment is more to do with the software industry not taking any notice of these linguistic differences than it is to do with their lack of existence.
From a software engineering viewpoint the fact that there are differences in communication across communities is interesting. It gives a new purpose to the traditional n-tier design in that it is pretty easy to create different user interfaces based around different needs if the code handling the UI presentation is separated from the rest of the code. It would be possible to create sets of DLL's for each community and swap them out at runtime based on the type of user that was using the system at that time.
All this makes me quite excited, and of course we'll need the compute power and cores to make it all happen :-).
Will
By Clay Breshears (Intel) on March 28th, 2008 at 10:25 am
One of my colleagues pointed out that some visual representation system like this would be part of the One Laptop Per Child program. Even in the US, there would be a need since there is an alarming amount of illiteracy in this country (1998 report put 30 percent of the adults in Mississippi as completely illiterate).
Your first comments remind me of an acquaintance that was saying that they had an easy time traveling in Europe. I can't remember which system they used, but they could easiliy communicate via sign language to deaf natives that knew the same system. So, there is an example where everyone in a world-wide community can share the same communication methods.
It's going to be a little harder for symbols to translate, I think. I watch anime and there are many things that go right over my head due to the differences in culture. I suppose that the skull and crossbones is pretty universal for danger and colors have come to some degree of universal acceptance for things like danger (red) and caution (yellow). When traveling in Brazil a couple of years ago, my companion had a picture book that was intended to be used by tourists to get across ideas like "where to go to eat chicken" or "where is the toilet." (It even had icons of heavy construction equipment. I guess the publishers figured tourists would be the ones needing to rent a bulldozer.) But, would a gun icon mean the same to an American inner city youth as it would to someone that has not even seen a gun?
By Will Pearson on March 29th, 2008 at 11:19 am
Hi Clay,
Yes, I agree with your comments about the difficulty in translating symbols. The variety of symbols is good for reducing ambiguity but it also can lead to situations where communication fails entirely because someone doesn't recognise a symbol.
One way to reduce these failiures might be to create symbols in a way that only uses features common to all instances of the concept someone is trying to communicate. There is some evidence from psychology that people recognise objects based on the features they consist of. Using common features should increase the likelyhood of people recognising a symbol; however, there would still be the problem of those people who haven't encountered that concept before.
I wonder if bulldozers look the same around the world.
Will