The reign of the era of the smart phone touch has become the mainstream way of human-computer interaction. But for smart watches and VR devices, touch is clearly not the ideal way of interaction. Then, put forward by the Google want to completely change the way of human-computer interaction gesture Soli what black technology?
Google Project Soli
At Google I/O conference in 2015, with an agile team as the core, to the outside and the institute of universities and enterprises from all walks of life and to establish a cooperative manner, for a variety of crossover study Google ATAP team to share the new progress in the field of human-computer interaction, including Project Soli and Project Jacquard.
Project Soli is based on the radio waves reflected the interaction between the system. 5 mm wavelength of radio waves have the ability to capture the fine motor within 5 m space range. However, the technical challenge is that radio transmitters are currently on the market is too big. Project Soli goal is to radio transmitter + receiver do watch, in a quarter coin size chip integrated radio transmitting antenna and receiving antenna.
Project Soli developed a only the size of fingernails chip, can be embedded in different types of electronic products. The radar emitted through the chip, can accurately detect the user's hand movements, specific gestures can adjust the different Settings, such as volume size, zoom in, or even switch machine and so on.
Revelation enables you to operate Google black science and technology from a distance
Project Jacquard is a planted the human-computer interaction interface to textiles (clothing) of the Project. Clothing production (billions) is about smart phones (billions) of more than one hundred times, a huge market, but technical challenges which were as follows: 1, the production process can be integrated into the textile factory production process; 2, electronic circuits and chips not ironing of high temperature, not washing machine blister, not afraid of the destruction of the user in the process of wear pull.
Although Jacquard touch fiber project and Soli still stay ATAP, but at least it has a new Soli grand goal: building equipped with radar technology and consumer electronic equipment industry design language.
The Project progress in Soli
After the debut in 2015, the Project Soli gesture remote control concept have further demonstrates this year. Google I/O 2016 conference entirely by gesture control smart watches and wireless speaker product prototype first bright, smart watches is a modified LG Urbane, can sense your hand, and action. Is the furthest away from the normal time display surface, as the finger close to appear more and more features, and away from the surface it back again.
Revelation enables you to operate Google black science and technology from a distance
In addition to the LG, Soli team will this technology into a JBL sound, but here it can "see" the scope is much larger, can judge the whole hand gestures, like than "like", can represent jumped out of a tune.
Revelation enables you to operate Google black science and technology from a distance
Soli 18000 FPS radar update rate, can "see" the quite subtle movements, even the pinch of finger gently can side to budgeting. Google ATAP technical project director Ivan Poupyrev said: "by putting the mobile and fixed terminal to realize the third kind of interaction, gesture sensing brought new opportunity to completely change the man-machine interactive. This will be able to fill the existing gap, on the basis of the touch and voice control interaction adds a convenient alternative interaction scheme."
Infineon with Google ATAP hope with "Solli" radar technology to satisfy including home entertainment, mobile terminals and the Internet of things (IoT), etc., the demand of the market.
Infineon Urschitz power management and diversified market division President, said: "our goal is to create a new market standards, superb performance and new user experience, thus to build the core technology of augmented reality and the Internet of things." Although the virtual reality technology has been bring to our immersive feeling, but so far, the user cannot interact with. Google and the development of infineon 60 GHz radar applications can fill the gap, as the key technology of augmented reality.
ATAP's aim is not just to prove the possibility of this kind of product in theory, it has to do with LG, qualcomm, JBL company for cooperation, and prove to the companies that they can launch the product in the market.
How to implement gesture recognition using radar
Radar, English Radar (Radio Detection And held), launch the wireless electromagnetic waves are used to get the reflection wave to detect the distance of the target object, Angle, And the instantaneous velocity. As the antenna size and chip is extremely narrow, in the foreseeable future, more of the radar equipment will be available in the micro device, as shown in figure shows, they not only can be embedded into the wearable devices, become an important sensor of Internet of things.
Traditional radar detection investigation (left) and Project for gesture recognition in Soli mini radar (right).
Figure of the traditional radar detection investigation (left) and Project for gesture recognition in Soli mini radar (right).
Compared to other operating technology from a distance, such as body feeling camera, such as ultrasonic, radar has some natural advantage: for example, day or night, cold wind exposure, all can work normally; On volume, cost, and power consumption is lower than motion-sensing cameras such as access to; Hf radar measurement object distance can usually be accurate to the millimeter level; While the low frequency radar can be done through the wall ", "totally ignoring the presence of obstructions. These features make the radar, especially miniature radar, has a broad application prospect in the future.
The composition of the radar
By general radar transmitter, receiver, transmitting/receiving antenna, the signal processing unit, and the composition of terminal equipment. The FM transmitter by transmitting antenna will go through or amplitude modulated electromagnetic launch out; Part of the electromagnetic wave is reflected back to the receiver, after touching objects that's like the sound reflected back touch the wall; Signal processing unit analysis of received signals and extract useful information, such as the distance of the object, Angle and speed; These results were eventually real-time display on the terminal device. Traditional military radar are often equipped with mechanical control the rotation of the device is used to adjust the antenna orientation, and a new type of radar is more do adjust electronically.
To save materials and space, transmitters and receivers can usually share the same antenna, the method is alternate switch transmitter or receiver to avoid conflict. Terminal equipment is usually an object can show the location of the screen, but more in the application of mini radar is a radar to extract the physical information as input signals to such as watches, or other electronic devices. The signal processing unit is the radar real ideas and the soul, which make use of mathematical physics analysis and the computer algorithm for radar signal filtering, screening, and calculate the position of the object. On this basis, still can use the forefront of machine learning algorithms to capture the feeling of signal for body gesture recognition, etc.
Radar gesture recognition
Radar range or velocity imagine objects into an abstract. While the real objects such as the palm of your hand can be thought of as a heap of 3 d points. So has already has many points in the reflected wave signal distance and speed. At the same time, a good way to present this information is called Range - doppler mapping (Range - Dopler Map), hereinafter referred to as RDM (see chart). The horizontal axis in the RDM is speed, the longitudinal axis is distance. It can be thought of as a probability of reflected wave energy distribution or figure, every a unit value represents the reflected wave from a certain distance and speed of a particular object of reflection wave energy. Look carefully, from RDM already can see objects detection feature size! Based on RDM and time series, we can adopt the method of machine learning to identify specific energy patterns change, and then identify the gestures and movements. Before the Soli launch, Nvidia has done similar research.
Range - doppler (speed) of mapping contour said sample
Figure range - doppler (speed) of mapping contour said sample Each unit value represents the reflected wave has a collection of points corresponding to the distance and speed of the reflection of energy. This mapping can be used as a feature vector to identify gestures of machine learning.
Revelation enables you to operate Google black science and technology from a distance
The challenge for the Project Soli and solution
Google first edition invited 60 Soli technology developers, developers also came up with all sorts of strange applications, such as identification of materials, 3 d modeling, and even into the steering wheel, let your hand without leaving the steering wheel, you can control the entertainment system.
But the first edition of Soli power consumption is very amazing, actually cannot use on the dressing equipment, Project Soli technology, head of Pope Leo (Ivan Poupyrev) said, "if a certain technology can integrate in smart watches, it can be integrated into any other product." ATAP redesigned the Soli chip, further reduced its size, reduce the energy consumption, and repeatedly to optimize it. According to Soli chief and hardware engineers Hakim, Raja (Hakim Raja), said after a year of efforts, Google finally achieved with Infineon chip minimize. It is very thin, 4 antenna can provide full-duplex communication, sending and receiving radar signal, the power consumption of Soli reduced from 1.2 W to 0.054 W, with more than 22 times. Google will also be 256 x program optimization, let the Snapdragon 400, Intel Atom processor running smoothly.
Revelation enables you to operate Google black science and technology from a distance
To do such a small chips also have drawbacks. Is the design of the radar target detection in just a few miles and more long-distance flight metal objects, rather than a few inches away, the size of fingers to make only a few millimeters in gestures. It was not long ago, people don't need to worry about this level of energy consumption, no people consider such a small chip impact on signal radar.
Soli lead researcher realized into the space provided by radar signal on the computer can work with the meaning of time signal. Nothing is such a small chip is more involved with the noise of the problem, she algorithm must find needed in many noise signal. For bunching of the signal is not possible, by the chip's every signal must be captured. In other words, this is a very complex task.
Compared with the electronic technology, electronic equipment after decided to identify the user gestures respond of machine learning algorithm is relatively simple, but it is definitely not pediatrician also. For the protection of the equipment configuration touch screen screen display buttons and sliders; For equipment with physical switch, users can feel scale in the toggle switch. However, if what also have no, how to guide the user to operate?
Pope livni said, "all things have their own interface? Each switch, intelligent sprinkler or cup has its own interface? This will lead to chaos." Soli one goal is to develop a kind of easy to grasp and flexible enough to control many functions of universal design language.
Nick jillian Soli chief mechanical engineer (Nick Gillian) team has finalized basic gestures are introduced. Gestures divided into two major categories of far and near, in the distance, the user can use few gestures (the user can waving arms, similar to Microsoft access body controller); After users near equipment, Soli can identify more subtle movements.
Pope Leo says Soli "design language reference entity control". These controls are twisting (twist on the index finger, thumb like twist toothpick to play), ring (flap thumb forefinger), rub (thumb along the finger sliding).
Soli control gestures can provide is the advantage of two levels of feedback: besides can see the screen's response to gesture, also can feel your fingers. It sounds absurd, but fingers can touch each other is also a kind of "tangible" operation.
Project Soli plans for the future
The first Soli prototype is LG Watch Urbane and JBL speakers, both products will go on sale in the near future. Loudspeaker vibration sound is to rely on air, so in which to integrate a chip can detect millimeter level movement, there are some problems need to be solved; Smart watches power and interaction problems still need to be solved.
, Google said it will release 2017 beta Project Soli developers kit, including the smaller size and lower energy consumption Project Soli hardware, power consumption is only about 1/22 of the before (54 milliwatt), volume and only a third of the original model. Then, the solution can be integrated into more mobile intelligent device.
For the Project of Soli Project in the future, the development team that can do a lot.
Revelation enables you to operate Google black science and technology from a distance
The left side as the prototype, the bottom right hand corner for the final version of the ATAP projects, and gesture recognition, and radar chips, although you can better solve the problem of intelligent watch screen is too small, but certainly not only satisfied with using smart wristwatch. As team said they hope Soli chip can appear in more places, become a kind of new way of future human computer interaction. Development team believe that more is now focusing on voice control, but out of the hands of human evolution is equally important.
