It's an historic week for AI. Ann Johnson, unable to talk for 18 years after a stroke, answered her husband's question about the possibilities for the Toronto Blue Jays, "Anything's possible," her avatar said, based on an AI program reading her brain waves, synthesizing her voice, and talking. Ann Belluck has the amazing story in the NY Times,  well worth the click. Or click on the picture for the video

 

The system worked by reading her brain waves,  translating her thought patterns into text, and then using a previously created synthesized version of her own voice to speak the words aloud.

 The avatar even reproduced some of her desired facial expressions.

I believe the programming took years of work. Like most machine learning programs, it started with a massive amount of data from tests by Ms. Johnson. I surmise the researchers annotated the data based on what she was able to slowly communicate. This is supervised learning like the early speech to text programs.

This is one of the great achievements of machine learning. 

 Written by Dave Burstein, with editing suggestions from Claude

Here's a Claude summary of the Times article article and the video, lightly edited.

- Ann Johnson lost the ability to speak after a devastating stroke at age 30. She was a teacher, coach and mother at the time.

- In a new study, researchers implanted electrodes in Johnson's brain to decode signals as she silently tried to say sentences. AI translated these into text, computer speech, and facial expressions on an avatar.

- This breakthrough could help restore communication for people who have lost speech from strokes, ALS, etc. It allowed Johnson to "speak" at a rate of 78 words per minute.

 This allowed the AI system to learn through supervised learning, similar to early speech recognition programs.

The system built a synthesized voice resembling Ms. Johnson's own voice, based on recordings of her speaking prior to the stroke. In the video, this voice emanates from the avatar shown on the computer screen as it has a conversation with Ms. Johnson's husband. The back-and-forth conversation demonstrates the remarkable achievement of this brain-computer interface and AI system in restoring Ms. Johnson's ability to communicate.

Hearing a computer voice similar to her own was emotional. Johnson and her husband even had a back-and-forth conversation using the avatar.

- The technology is advancing quickly but still imperfect. It misses one word in four. Johnson has been persistent in overcoming challenges since her stroke. This research helps her feel like her old talkative self again.