Do self-driving cars drive in bad weather?

Source: Deep Learning on Medium

Do self-driving cars drive in bad weather?

This article is one of the daily emails I send to the mailing list.
If you’d like to join the mailing list, subscribe HERE.

Last wednesday’s email was about bad weather conditions in self-driving cars.

Photo by belkacem makhloufi on Unsplash

One Step Ahead — Do self-driving cars drive in rainy or snowy conditions?

A lot of people recently reached out to me wondering if self-driving cars can work in rainy or snowy conditions.
You might see a lot of videos of self-driving cars riding in California or in sunny environments
These environments usually also have very few pedestrians and cars.
The ideal condition for self-driving cars

Maybe you have wondered the same question
What if it rains or snow?
No one seems to talk about this
Companies will communicate on their upcoming improvements
But is it technically feasible?

The answers seem to converge to a future yes
But if it’s possible, why has no one succeeded yet?
Sure we see some videos on YouTube showing it can work
But again, we don’t know what the result will be like in real-life
Our opinions seem to lead to a no.

In general, self-driving cars are made in 4 main parts —

  • Perception — how we see the environment
  • Localization — how we localize the car in the world
  • Planning — how we navigate the car from point A to point B
  • Control — how we move the car

Instantly, what works well?
We can assume localization can work assuming the GPS reception is good.
In the case of a storm or heavy clouds, it might not be the case.
For a GPS with RTK (Real-Time Kinematic) correction, clouds often lead to a float number instead of a fixed number
What does it mean? That we can’t really trust it.

So localization is 50/50 but can be improved to work with redundancy for example

We can also assume planning/navigation works well
If localization is good, there should be no problem guiding the car to a specific route
AI algorithms are still the same
A* (A-star) algorithm is often used to plan a trajectory

And a finite-state machine is used to handle specific situations like overtaking a vehicle

Control can work, but we should be careful of slippery roads

Perception is another story
It is what causes problems to the functioning of self-driving cars in rainy or snowy conditions
What is perception about again?

What sensors are used?
Depending on the strategy or the company, it will use cameras, LiDARs, and RADARs.

Radars can work under rain or snow, but the sensor is not really trustworthy
It is generally noisy by essence, and even more in these conditions

LiDARS use laser and light to sense obstacles.

If it rains or snow, impact points will appear everywhere around the car
Because the laser might touch every drop of rain and snow it crosses

The only sensor we might use is the camera

Technically, a camera simply retransmits what we see with the eye
And humans are able to drive on the road in heavy conditions

I wanted to see if it would really work in a condition where no line is visible and no clear road is defined
I run a custom freespace algorithm I trained to detect drivable areas
Here is the result

Freespace on snowy conditions

It is quite impressive
It can detect my lane even if the camera is filled with snowflakes…
… and the wipes are constantly on

The next step was to see if obstacle detection would work

Obstacle detection and tracking on snowy conditions

And the result didn’t disappoint again

What does it mean?
Self-driving cars rarely rely solely on the camera
So it’s unlikely that we would assume self-driving cars can work under rain or snow…
…based on these two pictures

The camera has a lot of potential
Companies like Tesla are betting big on this sensor
It is the closest we have to human eyes
Some scenarios like Fog will still be a big issue
Making the driving impossible
But rain might be managed thanks to this sensor

💡 A couple of weeks ago, I shared a 3 question survey that got very little answers.
If you want to support the daily emails and make them better, please take the short survey.

If you have ever worked with autonomous robots on these bad conditions, don’t hesitate to reply to this email and talk about your work!

Hope to see you tomorrow here,

If you’d like to receive more emails like this one, subscribe here.

These emails will give you access to daily tips on AI & Self-Driving Cars, stories from the inside on autonomous tech companies, promotional offers on my courses and premium invites to online or offline events

It is like a continuous training on autonomous technologies! Every day, you learn a bit more. You probably missed today, don’t miss tomorrow’s!