Audio version:
The path of technological innovation has been one of improving efficiency, speed and effectiveness of machines and humans. Whether steam engines, factory machines, medical devices or computers, there has been a steady march of technological innovations simplifying the way humans control machines and the way our machines communicate to us. Making machines easier to control to achieve desired results has been a steady and driving force in technological innovation.
Perhaps the first innovation in the user interface technologies was the development of IBM punch cards that replaced the need to program computers by punching a keypad or throwing switches on a programming panel. Punch cards gave the users the opportunity to store data and manipulate programs for easy loading into computers. My college days were spent assembling and sorting stacks of IBM punch cards that stored my programs and data for class projects. The technology worked well for computers that were like oracles, housed in large buildings and manned by students who offered your punch cards to the machine with reverence.
Next came teletype interfaces that eliminated the need for punch cards. You could type your program directly into the computer, which would store the program in memory. We still use keyboards today as learning how to type is now a skill taught in elementary school. However, the limitations of keyboards are well known.
The next revolution in user interface was the computer mouse. Originally conceived by Doug Engelbart of the Stanford Research Center in 1968[i], Apple was the first to use the computer mouse in a consumer product, introducing the mouse as a key innovation of the early Macintosh computers. The mouse provided a new way of interfacing with computers that soon lead to all new applications, programs, and furthered the mainstreaming of computers in society. With the mouse came the graphical user interface. The GUI combines visual impact with the tactile manipulation enabled by the mouse. The mouse and GUI made computers accessible to the average person, including children. Development of the graphical user interface by Apple and Microsoft dramatically increased the usage of personal computers and the number of applications to which this technology was applied. This lead to significant increases in productivity across all industries. The impact of GUI technologies is an example of how an innovation in human-machine interface technologies can have a dramatic and far-reaching impact on the economy and the way we live our lives.
In industry, machine-to-machine interface protocols soon enabled computers to control other machines, enabling computers to take the place of human operators. Computer numerical control (CNC) machines revolutionized manufacturing. The development of the program logic controller or PLCs extended the role of computers to controlling large-scale systems, such as factories. The result was automated factories that increased productivity, reduced errors and lowered the cost of manufacturing.
Next came web browsers for interfacing with the Internet. The impact of the Web browser is hard to summarize, given that is now the ubiquitous interface to the world via the Internet. It is no exaggeration to say that web browsers enabled the Internet to become what it is today.
Soon came touch screen displays that enabled users to interact almost directly with the computer by touching items on the screen. Manipulating objects on a display through touches, drags, drops, and swipes was even more intuitive than the mouse. Touch screens also enabled new applications of computers in smaller packages, most notably smartphones, smart watches, and similar ubiquitous computing systems that are now part of mainstream society around the globe.
Most recently, voice interface technologies have entered the commercial marketplace with Amazon’s Echo, Google Home, and Apple’s HomePod, as well as voice recognition systems in smartphones and automobiles. Significant investments in improving voice recognition and inferring the intention of humans have spurred a new growth of technology applications and increased our dependence on technology in our daily lives.
We can expect graphical user interface and voice recognition technologies to improve with time, such as using artificial intelligence to infer intent and provide more effective and interactive responses. However, the innovation trajectory of these technologies can be expected to follow the trends of previous user interface technologies in which technological breakthroughs and dramatic improvements in the interfaces become infrequent. Soon the limits of touch and voice interactions with machines will fall short of what is required for new technological applications and devices. Those needs will then be satisfied by brain-machine interface technologies.
I believe that brain-machine interface technologies are likely to be the next revolution in man’s means for controlling machines. The direction and pace of technology has made this type of user interface all but inevitable. The demand for better interfaces with computers, machines and the Internet are driving developers to look for better machine interface options.
Also, the time is ripe for brain-machine interface technologies due to the confluence of many factors. Artificial intelligence technologies are maturing and leveraging the impressive processing power of small supercomputers at a time when consumers are increasingly comfortable with interacting with computer systems in new ways, including conversation.
Artificial intelligence, or “AI”, has matured to the point that such analyses and computer learning methods can tackle many of the technical challenges involved in interpreting human thoughts and nerve signals. In the past few years, artificial intelligence has left the laboratory and entered the realm of commercial markets. AI provides methods for handling the large amounts of noisy data that will be gathered when sensors monitor the observable features of human thought, such as nerve impulses and brain waves.
AI technologies are benefiting from the development of specialized programmable processors optimized for AI computations. The combination of advanced AI methods and AI-optimized processors have brought the cost of AI methods down to an affordable level, as well as improved the speed of processing. Also, large scale distributed computing networks provide massive computing power for offloading processing of the noisy data gathered when monitoring the measurable manifestations of thought.
The commercial success of new technologies depends on consumer acceptance and adoption. In this regard, the success of voice recognition technologies in consumer products and services demonstrates that consumers are open to accepting hands-off computer interface technologies. Consumers are snapping up products and services that allow them to use their voice to turn on lights, control appliances and obtain information via the Internet. New markets are arising as people are no longer tied to a keyboard mouse or touchscreen to perform various machine interactions. These trends should carry over to brain-machine interface technologies as consumers are now comfortable controlling computers without a touch screen, mouse or keyboard.
With each new interface, a new explosion of startups, innovative products and markets have been created. New interface technologies enable new ways for humans to use computers and for computers to integrate themselves into our lives. With each revolution, innovators and entrepreneurs create new companies that generate new wealth. Like many innovative technologies, new user interface technologies create a wave of startups and new products that arise rapidly, while the technologies are adopted throughout the marketplace.
Voice recognition user interface technologies provide a good model for how brain-machine interface technologies may evolve. Voice recognition technology has been around for a long time. I have used Nuance Dragon for over a decade and I am using it now to write this article. Over the years, the accuracy of Dragon has improved dramatically. In the past two years, voice recognition has been added to smartphones as in Apple’s Siri and Google’s Alexa technologies. Amazon’s Echo and Google Home, and Apple’s HomePod have emerged as new consumer products providing a wide range of interfaces for consumers, particularly to the Internet. Voice recognition technology is now available in a wide range of products and can be expected to continue expanding into new products, now that voice recognition accuracy has improved to the point of reliability. However, all technology innovations eventually become mundane. Voice recognition will soon be mainstream and consumers will be looking for a more private and faster machine interface.
We can anticipate that brain-machine technology will follow a similar product and market development path. At first, there will be a few specialist applications in which this type of interface is particularly helpful and the challenges of the application are manageable. Technologies for monitoring brain waves for patterns associated with particular user mental states may be turned into consumer products focusing on training, therapy, and specialized learning applications. As the basic technology elements are refined and proven to be reliable, more complex applications may enter the marketplace. Eventually, widespread consumer adoption of brain-machine interfaces can be expected as the fundamental technologies prove their effectiveness and reliability.
I anticipate that brain-machine interfaces will be popular because in addition to being fast and intuitive, brain-machine interface technologies will be as private as your thoughts, unlike voice recognition in which everyone can hear what you say to your computer. Privacy is a growing concern in the United States—consumer demand for computer controls that are inherently private may drag brain-machine interface technologies into the marketplace. Additionally, brain-machine interface technologies should be insensitive to background noise, enabling their deployment in noisy environments where voice recognition will not work.
Once the basic building blocks are in place, we can expect an explosion of applications and startup companies creating new uses for the new interface technology. We can expect that the pace of development and business creation to outpace anything seen before. In part this is due to the ever-accelerating pace of technology development and the creation of new businesses. The many potential applications for thought control of machines will pull the technology along by motivating entrepreneurs and engineers to accelerate market development. With a wide variety of problems to be solved, these unmet needs will provide the seeds for innovation, providing the fuel for inventions and new startups. After all, Bill Gates, Steve Jobs and Mark Zuckerberg have shown the world that riding the wave of new technologies is the path to great riches.
Brain-machine interface technologies are likely to have far-reaching and wide-ranging impacts on society, markets and our daily lives. Such technologies will be private, fast, and reliable, providing new ways to control our appliances, our entertainment devices, and our computers. We can expect new support and hope for the disabled and handicapped as the interface technologies provide new and more natural ways of controlling prosthetic devices. Brain-machine interface technologies will provide users with ways of controlling machines and functionalities that once were only conceived in science fiction. Armed with such new controls and functionalities, innovators and entrepreneurs will invent solutions to problems that have yet to be discovered.
[i] “Life as We Know It Turns 50” Inside View by Andy Kessler, WSJ, Dec. 2, 2018.
