Topic
Games such as Second Life are immersive 3D experiences for their users, but users with physical disabilities are often excluded from participating in the fun due to limitations in the interface. My topic will be to create a "guide bot" for users which will handle the motion planning aspect of game play, providing audio cues so that a blind user could follow it.
The guide-bot can be instructed to find a certain location or to find another avatar and it will plan a course to that goal, leading the user via audio cues. If the user strays off course, the guide bot will track them down and re-plan a new path toward the goal.
Motivation
The motivation of this project is to use motion planning and good interface design to make Second Life more accessible to users with disabilities.
Prior Work
Although I found several games that produced audio cues designed for the blind, I did not find any commercial 3D games (MMORPGs, first person shooters, etc.) with blind accessibility:
http://www.audiogames.net/playcenter/index_exp.php
Most games are limited to simple puzzles:
http://gambit.mit.edu/loadgame/index.php#audiodyssey
Goals
The main goals of this project will be:
- Create a guide-bot that can be commanded to find a destination or other avatars to talk to
- Create audio cues to be able to lead a user
- Design interface for interacting with the game with limited keyboard interaction
Subtasks
- Create a guide-bot that can be commanded to find a destination or other avatars to talk to
- Determine how guide-bot will identify destinations and avatars
- Perform motion planning to reach these goals and take the path
- Determine heuristic for if the user has strayed to far from the path, and then be able to follow them, gain their attention and re-plan new path
- Create audio cues to be able to lead a user
- Provide audio instructions to guide users (for example "follow me straight ahead", "turn right", "slow down", etc)
- Make audio take advantage of stereo speakers
- Provide simple interface for selecting goals
- Additional Subtasks that I may not be able to get to, but would be nice:
- Would it be possible to utilize audio-to-text technology (or vice versa) to make the interface even more accessible?
- Could I make use of surround sound to provide an even more immersive audio cues?
Power Point Presentation
October 31 2007 Update
Since my program will be done in Second Life, the first Second Life homework is the first building block of my project (a motion controller), and the second homework will be a second step that must be taken to in order to to create a guide bot in Second Life (a path planner). This second part is due on November 12, 2007 and I will be beginning on it shortly.
In addition, I have begun work on investigating incorporating sound into Second Life. There are two approaches that I may take.
- The Second Life client is now open source. I could create a modified client which incorporates text to audio technology. I could present navigation options as chat or Instant Message spoken by my guide bot and then have the client translate all of the chat to audio (or chat on a specific channel to avoid confusion).
- Use a script which incorporates text to audio technology directly into a script executed by my guide bot. This has the advantage of avoiding two code bases to work with.
So far I have:
- Gone to http://secondlifegrid.net/programs/open_source and downloaded the open source code base
- Read over the documentation some to try to figure out if this would be a good approach
- Did some searching on integration of text to audio technology in the Linden Scripting Language and found this http://blogs.electricsheepcompany.com/christian/?p=80 which indicates that it can be done
Admittedly this isn't a lot of work done so far, but I plan to work on it a lot in the upcoming weeks, including working on the Second Life Homework 2 which will later be incorporated into my project.
Final Results
Below are links to code, source and PowerPoint/Word documents describing my results:
Final Project Results Presentation
Accessible Second Life Client (Source) - This is just a diff from the basic source code once integrated into Visual C++ 2005 Express Edition. Some changes were made from the v1.18.4.3 source downloaded from hereto get it to work with this version of Visual Studio as described here.
Accessible Second Life Client (Executable) - This may need additional DLLs (I have included all the ones I think you'll need but SecondLife makes use of so many different packages plus with the addition of the Microsoft Speech API it is difficult to make sure everything is there).
libSecondLife MotionController (Source) - This is just the files I added to the libSecondLife v0.3 codebase which can be found here.
libsecondlife Guidebot Executable- The executable and all DLLS needed to run the guide bot executable