You are here

You are here

Brain-machine interface to reach beyond health, game developers

John P. Mello Jr. Freelance writer
Research in the medical field and the arrival of virtual reality hardware in the gaming realm have brought brain-machine interface tech closer to reality.

Controlling machines with the human brain the way brain controls the human body has been the stuff of science fiction for decades. But recent research in the medical field and the arrival of virtual reality hardware in the gaming realm have brought the technology closer to reality. For the immediate future, medicine and gaming will be the areas offering the most opportunity for BMI developers, but it won't be much longer before other industries follow.

Among those areas will be lifestyle applications: "Developers are building for games now, but there is a lot developers can get into in lifestyle applications," says Tej Tadi, founder and CEO of MindMaze, which makes BMI products for both medical and gaming applications. For example, when the brain senses the temperature in a room is too cold, an app could automatically turn up a smart thermostat.

Before the marriage of brains and machines can make it to a broad audience of developers, the technology needs to be perfected. Medicine is a good field in which to do that, which is why MindMaze began developing its BMI there. "You go through a rigorous process building for medical applications," Tadi explains. Once MindMaze gained some success in medicine, it decided to broaden the reach of its technology to a consumer market—and one where more developers could get their hands on it—by forming a spin-off company focused on virtual reality gaming.

The spin-off, MindLeap, uses MindMaze's medical-grade neurotechnology engine, which predicts brain and muscle activity to trigger real-time, immersive virtual and augmented reality experiences. Dry electrodes detect brainwaves and muscle movement, and a camera captures motion information.

Near real-time connection gets real

An important consideration for a system like MindLeap's is how fast it reacts to a gamer's thoughts and movements. MindLeap marks the quickest reaction time to date, with near-millisecond synchronization, the company says. Before year's end, the company expects to have its neuro engine available to game developers via a software development kit (SDK). Gamers will see the tech via its own head-mounted display (HMD), which includes special neuro goggles and 3D motion cameras that allow a gamer to not only see their bodies or avatars in their virtual worlds but also trigger events.

MindLeap appears to be hoping to gather a large cadre of developers for its neuro-powered, immersive virtual and augmented system with plans to support major gaming consoles, such as Xbox and PlayStation, as well as the leading mobile platforms Android and iOS.

"MindMaze puts your brain into the game," Tadi said, when announcing MindLeap at the Game Developers Conference in San Francisco in March. "Never before have neuroscience, virtual reality, augmented reality and 3D full-body motion-capture come together in a games system.

"Gamers will be able to see, feel and experience virtual gameplay with absolutely no delay or need for controllers," he continued. "The technology will enable game developers to deliver responsive virtual games experiences, further immersing gamers into the game play and creating opportunities for deeper levels of engagement with consumers."

The pledge to cut gamers loose from their bondage to game controllers is appealing to developers who want to create truly immersive virtual experiences for their offerings. "The use of traditional game controller devices removes the gamers from the experience somewhat because the action is unnatural," explains Christine Arrington, senior analyst for games at IHS Technology. "Shooting with a gamepad or even a [PlayStation] Move-type device is a barrier to full immersion. Controlling interactions directly with a BMI can serve to remove that barrier."

"Ditching the controller is the Holy Grail for VR and AR," adds Eric Smith, an analyst with the digital consumer practice at Strategy Analytics.

While controlling objects in a game with one's mind has been demonstrated before, this is the first system that allows a player to insert his or her body in the virtual environment and act in it in real time, notes Lewis Ward, a research director for gaming at IDC. "That is unique and could be very important over time," he says. "This is a new UI for interacting with computing devices. That has the potential to be revolutionary."

Interface tech has game face on beyond gaming

That revolutionary potential will reach beyond games. In the defense sector, for instance, the US Defense Advanced Research Projects Agency (DARPA) is working on a "cortical modem" that allows images to be sent directly to the brain, removing the need for devices like virtual reality glasses. In addition, BMIs would add a new dimension to multitasking. "You could be physically doing an activity but your thoughts could be gaining control of something else happening at the same time," Tadi says.

There's also a potential "Iron Man" application for the technology. The military has been working on exoskeleton technology that could be enhanced with a BMI. It uses an articulated steel frame designed to move as a human within it moves.

Once BMI moves beyond sharing control of objects, the technology could move into learning, which will open it up to educational developers. "Any motor-specific skill learning can always be tuned in by BMI because you're tapping into error correction in the brain," Tadi says. "So you'll be able to learn differently. You'll be getting real-time feedback, and it won't be a one-size-fits-all approach. The feedback is going to be specific to your skill and aptitude, which is a good way to perfect any new skill."

Bleeding-edge tech to make the jump, but with bumps

As promising as BMI could be for developers, some bumps can be expected in the road ahead because new technologies never work as smoothly as they do in labs, product announcements, and demos. "The bridge between medical and consumer technology will take some time to build," IHS Technology's Arrington says. "Often, in technology transfers from one industry to another, there are unforeseen obstacles that will come up and delay or even prevent the transfer from happening.

"No matter how much a technology developer thinks they have covered every contingency," Arrington adds. "Application developers will find a use they didn't account for and there will be more work needed."

Keep learning

Read more articles about: App Dev & TestingApp Dev