Skip to main content
Self-driving cars

Cars that can move without a human in the driver's seat have been in development for almost a century, but we are still far from trusting them with our lives in any circumstance

The Mercedes-Benz F 015 Luxury in Motion in Shanghai, China in May, 2015.

The Mercedes-Benz F 015 Luxury in Motion in Shanghai, China in May, 2015.

Daimler

Futurama, the biggest hit at the 1939 World's Fair, envisioned a future of forests, farms and cities connected by mega-highways that would be populated with radio-controlled cars.

Futurama exhibit at the 1939 World’s Fair in New York City.

Futurama exhibit at the 1939 World’s Fair in New York City.

Wikipedia Commons

Safety and speed: yours by 1960, promised the exhibit. Its designer, Norman Bel Geddes, would be the first but not the last to make predictions about when self-driving cars might take over daily commutes and reshape cities.

Mark Fields, CEO of Ford, said his company would have fully autonomous cars on the road by 2021. BMW's boss announced the same deadline. A GM executive said it would be 2020 or earlier. Audi CEO Rupert Stadler said 2025. Tesla's Elon Musk said fully autonomous cars would be possible by 2018, but governmental regulations would hold back its roll out. And now Musk says all 2017 Teslas will have "full self-driving hardware." Carlos Ghosn, CEO of Nissan, said 2020. Sergey Brin said Google would have autonomous cars available to the public by 2018, but that was in 2012.

"They are going to be retracting that," Krzysztof Czarnecki said of the CEOs' grand predictions. He is the principal investigator at WatCAR – the Waterloo Centre for Automotive Research – at the University of Waterloo. "They are going to say, 'What we have is what we promised.' But I believe what people think they are promising today is probably much more than what will be delivered."

When Tesla announced in October its cars "have the hardware needed for full self-driving capability," the company actually meant its cars will be able to accomplish the feat in certain situations: sunny warm days, certain roads, etc.

Autonomy is not binary. "Full autonomy" is more marketing buzz than anything else.

Self-driving Cars The most extreme ways car companies are testing self-driving technology

2:01

When will AVs be ready?

You won't be able to walk into a Ford dealership in 2021 and buy a car that drives you home. Fields said those first autonomous vehicles will be for ride-sharing fleets only – likely because they'd be extremely expensive. And when Musk says a self-driving car is possible by 2018, he defines it as a car that's safer than a human driver – which is by no means safe. And none of the executives are talking about cars that can drive through a Canadian winter, where road markings are obscured and the surface is slippery.

The disconnect lies in the definition of autonomy. Despite the faithful and breathless coverage of these announcements, what autonomous vehicles (AVs) actually are often goes unexplained and unexamined.

"A lot of people use the name 'autonomous car' for completely different things," said Eberhard Zeeb, a senior manager who works on driver-assistance systems at Mercedes-Benz. "Maybe through the press it was launched that Google or Apple or Elon Musk could build up autonomous cars very easily – I think that's probably not true."

Today, vehicles with Level 2 autonomy are as good as it gets.

These cars with Level 2 autonomy – from Mercedes-Benz, Audi, BMW, Ford, Hyundai, GM and others – can accelerate, brake and steer themselves on highways in good weather, but the driver must constantly monitor the situation, taking over at any time, without warning, when the car makes a mistake, which does happen.

Levels of autonomy

When it comes to cars there are five levels of automation as defined by the Society of Automotive Engineers (SAE).

  • Level 0 – No automation: The driver does all the work.
  • Level 1 – Driver assistance: A computer lends a hand occasionally such as with cruise control.
  • Level 2 – Partial automation: The car can help you drive. It can steer, accelerate and decelerate without the driver touching the steering wheel or pedals. Tesla’s ‘autopilot’ system is the best-known example of this. The driver can’t take his hands off the wheel and if there is a problem, the driver must immediately take over.
  • Level 3 – Conditional automation: The car is responsible for driving, monitoring the environment and making decisions. No company has a Level 3 on the market yet, but many say one will be available within five years. A human must be ready to take over if the sensors fail.
  • Level 4 – High automation: The car can drive in all situations by itself, as long as it stays in a safe, well-mapped area. This is the first level where the driver can take a nap.
  • Level 5 – Full automation: The car can go anywhere, any time, in any weather. No steering wheel, no pedals. There is no need for the driver to pay attention because he or she can’t take over control anyway.

What auto-industry CEOs are promising with their bold announcements is anything from Level 3 "eyes-off" to Level 4 "mind-off" autonomy by early in the next decade, and they haven't said how much such luxury might cost.

"We're working hard to have the first Level 3 systems on the road," Zeeb said. "Maybe in the next-generation [Mercedes] E-Class. Maybe a little earlier." He said the end of the decade is a reasonable time to expect Level 3 cars.

But even this "eyes-off" automation isn't quite the robo-car of the collective imagination. These vehicles would be limited to driving in certain places in certain weather conditions, and the driver would still have to be ready to take over on short notice.

What about Level 5, complete autonomy? "Normally we say it's not before 2030," Zeeb said. "But if you really want to go anywhere in any weather conditions, I would say it would be at least 2050 or so for full autonomy."

What's taking so long?

For one thing, autonomous vehicles are illegal in most North American jurisdictions. Cars in which the driver relinquishes control are only allowed with special testing permits and they require a human in the driver's seat should something go wrong.

Uber self-driving Ford Fusion in Pittsburgh

Uber self-driving Ford Fusion in Pittsburgh

Uber

Uber's self-driving taxis in Pittsburgh have an engineer behind the wheel as a back-up and a co-pilot in the passenger seats taking notes. Google's self-driving Lexus SUVs are limited to 25 or 35 mph and have a driver behind the wheel. As of late 2015, Google's AVs detected autonomous system failures once every 8,600 kilometres on average. In each case, the car alerted the human driver to take immediate control.

But the biggest hurdle is the technology. An AV must first see the world around it, then distinguish and recognize individual objects, predict where they might go, and how it should navigate them.

Driving in the chaos of a city – where a child could dart from between parked cars – in snow, fog or rain, and on bad roads with little or no markings is a complex task and presents a huge challenge for a machine. Researchers in Canada are working on AVs that will work not just in sunny Silicon Valley, but in snowy, colder climates as well.

The Waterloo Centre for Autonomous Research self-driving car sits idle on a test road near the University of Waterloo campus in September, 2016.

The Waterloo Centre for Automotive Research self-driving car sits idle on a test road near the University of Waterloo campus in September, 2016.

Waterloo Centre for Automotive Research

"The projects we're trying to push forward that differentiate us from the rest of the self-driving community are those focused on what we have in Canada that is different: surprisingly degraded road surfaces and lane markings," said Steve Waslander, associate professor and director of the University of Waterloo Autonomous Vehicle (WAVE) laboratory.

Researchers at Waterloo are working on several autonomous-related projects including: lane and marking detection, getting AVs to recognize and understand even half-erased markings; creating secure, hacker-resistant systems; improved methods of machine learning to make it more reliable; and teaching AVs to recognize and estimate when the road surface might change – useful for safely navigating a patch of black ice, for example.

When we arrived at the University of Waterloo, they'd just taken delivery of a Lincoln rigged with sensors and cameras to use as an autonomous test vehicle.

Self-driving cars Canadian universities developing self-driving cars for bad weather

3:00

At McMaster University, Saeid Habibi and his students are developing a system that can track and predict the movement of other vehicles, even if they are momentarily obscured, crossing in front of a truck, for example.

What is LIDAR?

LIDAR (light imaging, detection and ranging) will be a key component of self-driving cars, according to just about every major company, with the notable exception of Tesla, whose suite of "full self-driving hardware" doesn't include it.

LIDAR can "see" even in bad weather. It looks like a spinning siren, usually mounted on the roof. It works like radar, but instead of emitting radio waves it sends out laser beams which bounce back to create a 3-D image of a car's surroundings. It creates a real-time map of the car's surroundings up to 100 metres away and accurate to a couple centimetres.

The display screen inside a McMaster University test vehicle showing the image from a Lidar and a camera.

The display screen inside a McMaster University test vehicle showing the image from a Lidar and a camera.

Jordan Chittley/The Globe and Mail

Habibi says a LIDAR costs $35,000 and some research cars use four or six of them. That means adding hundreds of thousands of dollars to the price of a car. But there are companies working to bring the price down into the hundreds of dollars. Once that happens, LIDAR will still need to be augmented by radar sensors, cameras and detailed road maps. Still, the cost and size of LIDAR technology has to come down a long way before it's ready for your average compact autonomous car.

There are other issues to solve before AVs will be ready to tackle inclement weather – such as creating ultra-accurate road maps that update in real-time and figuring out how to communicate with and acknowledge human drivers, cyclists and pedestrians.

Getting to 100

"We are at the point now where, yes, we can demonstrate that radar can detect most cars, LIDAR can detect most cars, most pedestrians, signs, road markings," Waslander said. "But we don't want 'most of the time.' We want all the time. … We want to get to the point where we can trust these vehicles with our lives on a daily basis."

Habibi recalled seeing an autonomous vehicle demonstrated in 2006. When he asked why it wasn't on the market, he was told: "The sensors were more expensive than the car itself, but they could only detect pedestrians with 97-per-cent accuracy." For every 100 pedestrians in its way, it might hit three.

But what if autonomous vehicles are never 100-per-cent safe? At what point do we allow them onto roads, knowing they could injure or kill someone? When it's safer than a human driver?

The trunk on the Waterloo Centre for Autonomous Research self-driving car

The trunk on the Waterloo Centre for Automotive Research self-driving car

Jordan Chittley/The Globe and Mail

"I wouldn't see it as how much technology we need to add," said Sebastian Fischmeister, associate professor at WatCAR. "I see it more as the behaviour of drivers, drivers accepting the risk that comes with these vehicles. Because covering 100 per cent of all cases is virtually impossible," it could take a generation of drivers growing up with autonomous cars to truly accept the risks.

So when will the autonomous car of your dreams be ready? It's complicated.

But while you wait, consume the hype with a healthy dose of skepticism.

This is the third article in a seven-part multimedia series on self-driving cars that examines the past, the current technology and what the future may hold.