Main content

Alert message

Voice Navigation (aka Talking Brick)

"Voice Navigation" turned out to be the very first enhancement to the MINDSTORMS® environment requested by one of our blind students. Within seconds of picking up a demo robot at our very first robotics meeting, he asked, "How do I turn on the sound?". I totally mistook the question he asked, thinking he was asking how to turn on sound output from the NXT. This demo program didn't output sound, so I said "It doesn't play sounds." But that wasn't his question. "How do I turn it on and navigate the menus?" he asked more forcefully this time. "Well, you can't", I answered. "At least not yet".

With a bit of NXC programming, we ended up with a usable "voice navigation" system. It is far from what it could be, but much better than what we had. Our first attempt which worked well for practice and our first competition looked like this: 
nxt brick showing display

The display above contains the following information:

  • Line 1 - TSBVI - "Texas School of the Blind and Visually Impaired"
  • Line 1 - 8004mV - 8.004 Volts. Battery Power indicator.
  • Line 3 - left - announce - pressing the left triangular gray button announces what program you are about to run
  • Line 4 - right - next - cycle to the next program in the set of programs are in memory ready to run. Programs are stored in a "ring", where the last program wraps back to the first program.
  • Line 5 - square - start - the orange square button is the program start button
  • Line 7 - Pgm: 1 blueQ - 1 is the current program number, and the one that will run if the "start" button is pressed. "blueQ" is the name of the program for the (sighted) mentors to read. The talking brick only speaks letters and numbers. In this case it would announce "ONE". The small memory (256 KB Flash Memory) on the NXT do not allow longer names to be recorded and stored there. You quickly run out of available memory. The newer EV3 will remove this restriction (16 MB Flash Memory plus and a microSDHC slot).

This worked well enough in practice and also reasonably well at our first qualifying tournament. It must have since we were co-champions at that tournament, but we did have some trouble hearing the program numbers being announced from the NXT. The NXT does not have a large speaker, and the sound can easily be drowned out by the crowd noise. After discussing this in the team post game analysis (our post mortem), we decided to add large numbers to the display which could be read by our vision impaired students.

That display is shown below: 
display showing #1 filling the screen

Operationally, it has the same button interface as version 1 of the voice navigation had. The display now is geared towards the visually impaired student and has a huge number or letter in the display. In the example above a "1". The program name is in the upper right corner, here "yelQ" (For those of you who know the &qout;Senior Solutions" game, this is for the "Yellow Quilt Mission".) The upper left hand corner also has the program number. It should match the large print display and is there for debugging purposes. The lower right corner has "8239" which is the battery state in milliVolts.

One other oddity remains in using this interface. It is easy to turn on, simply find and press the square (orange) button. But how do I turn it off? We found the only reliable way to do this is to have only one true program on the NXT, the voice navigation (or launcher program). Thus, to start a program from an NXT which is off, we simply press the square button, wait for the NXT startup sound, then press the square twice. This starts the voice navigation. Now you can select which program to run and start it.

When it comes to turning off the NXT you do this specific sequence:

  1. Press the (gray) rectangular button 6 or more times.
  2. Press the (orange) square button once
  3. Press the (gray) rectangular button twice
  4. Press the (orange) square button once

After the press of the square button in step (2), you may be off already. In that case, continuing with steps (3) and (4) won't matter. But, if you aren't off at step (2), continuing with steps (3) and (4) will turn the NXT off. It's a bit of a pain at first, but it is a easily learnable sequence that will become second nature very quickly.