A lot of media coverage of self-driving technology has focused on a handful of big companies with well-known brands: Google, Uber, Tesla, and GM. But there’s another company working on self-driving technology that might ultimately prove even more important. That company is Mobileye, an Israeli startup that was acquired by Intel in 2017.

Mobileye doesn’t have Elon Musk’s star power or Google’s billions. But it has something that’s arguably even more important: a dominant position in today’s market for advanced driver-assistance systems (ADAS). Mobileye had a very public split with Tesla back in 2016, but it continues to do business with a lot of other carmakers. Mobileye says it shipped 17.4 million systems last year, which means 17.4 million customers bought cars with Mobileye’s cameras, chips, and software.

In a Tuesday speech at the Consumer Electronics show, Mobileye CEO Amnon Shashua made clear just how big of a strategic advantage this is. He laid out Mobileye’s vision for the evolution of self-driving technology over the next five years. And he made it clear that he envisions Mobileye staying at the center of the industry.

“Level 2+”

For the last two years, we’ve touted Cadillac’s Super Cruise as the gold standard for ADAS systems. Two features make Super Cruise stand out. First, it uses a driver-facing camera to verify that the driver’s eyes are on the road. If not, the system forces the driver to take over. This feature addresses one of the biggest concerns with ADAS systems: that they could make drivers so complacent that they don’t intervene when the technology malfunctions.

Second, Cadillac has pre-mapped more than 130,000 miles of freeways in the US and Canada. The system will only engage on those roads, which makes it much less likely that the system will get confused and make a dangerous mistake.

In his Tuesday speech, Mobileye’s Shashua calls ADAS systems with high-definition maps, like Super Cruise, “Level 2+”—a small step above regular ADAS systems that are called “level 2” in the five-level SAE framework. A number of carmakers have developed similar systems. Shashua says Mobileye is supplying the technology for 70 percent of them, including systems from Nissan, Volkswagen, and BMW.

As it sells its technology to carmakers, Mobileye has bargained for access to sensor data from customer vehicles. Shashua says that Mobileye is already collecting data from Volkswagen, BMW, and Nissan vehicles. He says three other unnamed carmakers have also agreed to participate.

The scale of this program is massive. Mobileye says it is already collecting 6 million kilometers (3.7 million miles) of sensor data every day from vehicles on public roads. Mobileye expects to have more than 1 million vehicles in its European fleet by the end of 2020, and 1 million American vehicles the following year.

The company uses all this data to generate detailed, high-definition maps of the areas where the cars drive. Mobileye says it already has software that can automatically generate HD maps of roads above 45 miles per hour. The company expects to extend this capability to all roads next year. Mobileye expects to have all of Europe mapped by March, with America being fully mapped later in the year.

This is significant because gathering high-definition map data has been a major obstacle to deploying self-driving technology. In the past, companies had to build these maps by hand by paying workers to drive mapping cars along every street and then having a second group of humans hand-annotate the collected data. If Mobileye can crowdsource and automate this process, the resulting data will easily be worth billions of dollars.

Once Mobileye has assembled all this data into an HD map, it can send up-to-date map tiles back out to cars in its fleet. As a result, each of Mobileye’s partners—Volkswagen, BMW, Nissan, and others who haven’t been made public yet—will be able to offer Super Cruise-like “Level 2+” ADAS features without needing their own fleet of map-making cars.

Camera-only self-driving

Mobileye’s dominance of the ADAS market seems secure. But there’s a danger that Mobileye could get stuck in an ADAS rut. Mobileye’s current business model is to sell chips, cameras, and software to existing automakers. Yet many experts believe that the first deployments of fully self-driving vehicles will be in taxi fleets, not customer-owned vehicles.

Shashua also believes that full autonomy will come to taxi fleets first. And he’s determined not to let Mobileye be left behind. So in addition to its “Level 2+” products, the company is also working on a longer-term project to build fully self-driving technology. This week, Mobileye showed off a self-driving vehicle that can drive entirely based on 12 cameras. It had no radar, lidar, or other sensors.

An impressive video shows this Mobileye prototype driving through the chaotic streets of Jerusalem for 20 minutes. The car navigates complex intersections, merges into tightly packed lanes, and deftly avoids hitting other vehicles.

Lidar skeptics might claim vindication here, but Shashua isn’t planning to actually ship a car without radar or lidar. Instead, the camera-only car is part of Mobileye’s larger strategy for building safe self-driving systems.

Mobileye’s plan is to build two completely independent self-driving systems: one based entirely on cameras, the other based on radar and lidar. If Mobileye can prove that each individual system can travel for more than 10,000 hours between crashes, Mobileye argues, then a system with both sets of sensors should be able to travel for 100 million hours (10,000 times 10,000) without a crash. This latter figure would make Mobileye’s cars significantly safer than a human driver.

I was skeptical of this math when Mobileye first announced it two years ago, and I haven’t changed my mind since. Mobileye seems to assume that the two systems’ failure modes are statistically independent, and it’s hard to see how that could be true. It seems pretty likely that scenarios that confuse a camera-based system are more likely to confuse a lidar-based one.

Even still, redundancy is an important principle in any safety-critical system. Camera- and lidar-based systems will surely have somewhat different failure modes. So building separate self-driving stacks around different sensors and then running them in parallel should yield a margin of safety—even if Mobileye’s math is too optimistic.

Mobileye’s belief in redundancy is evident in the design of its 12-camera self-driving system. The company has assembled a suite of six different algorithms for detecting objects around the car:

• One algorithm is tuned to identify wheels and infer vehicle locations based on that. Mobileye also has a dedicated algorithm to identify car doors, since open doors are often a sign of potential safety issues.
• Another algorithm uses “visual lidar”—by comparing images from different cameras, the algorithm can infer a distance for each pixel in an image. The algorithm then uses these estimates to generate a three-dimensional point cloud like you would get from a lidar sensor. Mobileye’s software then applies standard software designed for lidar data to try to identify objects in the scene.
• A third algorithm focuses on identifying which pixels correspond to drivable roadway. Anything in the scene that isn’t part of the road is a potential obstacle, warranting extra caution.

Mobileye hopes that by processing images in many different ways, it can minimize the odds that any important object is missed. Once an object is detected, Mobileye has four separate algorithms that independently try to place it precisely in three-dimensional space.

A key question is what the system does if the different algorithms disagree; Shashua didn’t fully explain how this works in his Tuesday presentation. Still, the results seem to speak for themselves. Mobileye’s 20-minute demo showed its vehicle handling complex traffic scenarios about as well as other leading companies in the self-driving sector.

The road to full autonomy

As I mentioned before, Mobileye doesn’t ever intend to ship a fully self-driving car based only on cameras. Rather, Mobileye’s camera-only self-driving system is destined to be one piece of a more sophisticated self-driving system that Mobileye hopes to deploy in the next few years.

Mobileye is planning to separately build an autonomy stack based on lidar, radar, and other non-camera sensors. Mobileye plans to first test the systems separately and demonstrate that they achieve impressive performance on their own, then combine them into a super-system that Mobileye hopes will be much safer than either system on its own.

Shashua hopes to move fairly quickly. The company is aiming to deploy taxi fleets in three major cities—Tel Aviv, Paris, and Daegu City, South Korea—by 2022.

Shashua expects the hardware for these initial self-driving taxis to cost $10,000 to $15,000 per vehicle. By 2025, Shashua is aiming to “reduce the cost of a self-driving system below $5,000.”

Shashua’s speech convinced me that industry observers—including me—haven’t been taking Mobileye seriously enough. Mobileye’s position as the dominant ADAS provider gives it some important strategic advantages. Its existing relationships with automakers means that it has little trouble getting new automotive technology deployed in the real world. It has successfully bargained for access to sensor data from customers’ cars, giving it access to a large and valuable data set that most self-driving companies (other than carmakers) can access. And it has deep engineering expertise in building autonomous systems, as evidenced by the impressive performance of its self-driving prototypes.

Mobileye also deserves credit for its open approach to developing self-driving technology. Mobileye has developed a mathematical model for autonomous vehicle safety called Responsibility-Sensitive Safety and is working to get it adopted as an industry standard. Shashua’s Tuesday presentation provided more information about the technical architecture of its self-driving system than almost anyone else in the self-driving industry. Mobileye still has work to do to convince the public that its technology is safe, but at least the company is trying. A lot of Mobileye’s rivals have refused to explain how their systems work, giving the public no way to evaluate how safe they are.

https://arstechnica.com/?p=1640981