Tesla's AI Approach for Reaching Full-Self Driving

#002

Mobile Phone - Set to Landscape (Horizontal) view for better visibility

GOLD NUGGETS

Sources - They are clickable if you need more information

Mr.Bean's Autonomous Car

Is there a better approach?

Automation Levels of Autonomous Cars

Tesla Aims to Be the Ultimate Level of Car Evolution

Full Self-Driving

What Does Tesla Do with AI?

Active Safety

Smart Summon

Autopilot

The car keeps its lane and keeps its distance from surrounding vehicles.  Autonomy on a highway.

With your phone, summon the car to you from the parking spot.

Automatic emergency braking when noticing a pedestrian.

What Does Tesla Do with AI?

However, to reach this level, it had to solve some problems...

Active Safety

Smart Summon

Autopilot

The car keeps its lane and keeps its distance from surrounding vehicles.  Autonomy on a highway.

With your phone, summon the car to you from the parking spot.

Automatic emergency braking when noticing a pedestrian.

Problem #1 - How to Give Eyes to a Car?

What Competitors Use? LiDARs

GOLD NUGGETS

LiDAR uses pulses of light to detect the objects, much like how radar works using radio waves. These pulses can determine the distance and range of an object, providing much needed data to self-driving cars. For example, to avoid a collision, the LiDAR can detect the distance to an object and apply the brakes to slow down the vehicle. [1]

LiDAR (Light Detection and Ranging)

LiDAR Example - Waymo

Waymo—formerly the Google self-driving car project—stands for a new way forward in mobility.

Telsa's Approach - Cameras

For Tesla, cameras are the keys to the future and its CEO sees a future when cameras will enable Tesla to see through the most adverse weather situations.

Lidar is a fool’s errand,” Elon Musk said. “Anyone relying on lidar is doomed. Doomed!

GOLD NUGGETS

Telsa's Approach - Cameras

AP - AutoPilot (software option in cars after 28-Feb-2019)

AP1 - Autopilot 1 (software and hardware prior to AP2)

AP2 - Autopilot 2 (software and hardware after AP1)

LiDAR - Light Detection and Ranging

FSD - Full-Self Driving

GOLD NUGGETS

Tesla Terminology

For Tesla, cameras are the keys to the future and its CEO sees a future when cameras will enable Tesla to see through the most adverse weather situations.

Lidar is a fool’s errand,” Elon Musk said. “Anyone relying on lidar is doomed. Doomed!

GOLD NUGGETS

 8 HD cameras delivering 360° view

How Does That Look in Practice?

Cameras vs Lidars

  • Better at safely navigating and avoiding hitting objects because they determine object distance
  • More expensive and requires more space to implement on cars, thus it tends to make self-driving cars look bulkier.
  • Don’t have the range detecting feature of LiDAR.
  • It's necessary to use other sensors besides the camera including radars to detect range and distance.
  • Problems seeing well enough to avoid danger in the difficult types of conditions.
  • Identify objects on the road and read road sign with AI
  • Work as a stand-alone system
  • The laser’s wavelength can be affected by temperature variations
  • Poor Signal-to-Noise Ratio affects the sensors in the LiDAR detector.
  • More reliable as a visioning system

How Can Tesla Detect These Different Scenarios with Cameras?

Problem #2 - How Hard Can it Be to Detect One Stop sign?

Temporarily

Located on a Wall

Funny Lights

Different Locations

Passive/Active Position

Heavily Ocluded

Variety of Modifiers 

Very Hard          Stop Sign Variations

How does Tesla tackle this problem? They use a HydraNet 

GOLD NUGGETS

The Lernaean Hydra or Hydra of Lerna, more often known simply as the Hydra is a serpentine water monster in Greek and Roman mythology. The Hydra possessed many heads, the exact number of which varies according to the source. [1]

Hydra of Lerna

Multiple Heads

Similar to ResNet50

Objects

Traffic Lights

Markings

Shared Backbone

HydraNet - Object Detection Network's Architecture

GOLD NUGGETS

Multiple heads are specialized neural networks for image segmentation. They do more than 1000 different tasks because for each detecting task, there can be multiple subtasks that come with it. Shared backbone is a pre-trained neural network that is an important piece when doing transfer learning.

Transfer Learning

Speak one language  

Learn how to speak another one

When people acquire knowledge about one task, they can use it to solve related tasks.

The idea of overcoming the isolated learning paradigm and utilizing knowledge acquired for one task to solve related ones.

Transfer Learning

AI KNOWLEDGE

How does Tesla use Transfer Learning?

Tesla uses similar pre-trained model as a backbone like ResNet50  

ResNet50 is a 50 layer Residual Neural Network.

How does Tesla use Transfer Learning?

Tesla uses similar pre-trained model as a backbone like ResNet50  

ResNet50 is a 50 layer Residual Neural Network.

Residual
Neural
Netwok?

A Quick Aside on Neural Networks

A neural network is a series of algorithms that tries to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature.

Neural Network

AI KNOWLEDGE

A Quick Aside on Neural Networks

A neural network is a series of algorithms that tries to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature.

Neural Network

Let's keep it simple. Residual neural networks learns by jumping over some layers.

Residual NN

AI KNOWLEDGE

AI KNOWLEDGE

A Quick Aside on Neural Networks

Explain like I'm 5 (ELI5)

Judo team did well on the tournament, so the coach asks them where are they going to eat to celebrate?

Each kid starts screaming their preference (burgers, pizza, etc.), and the coach goes with whatever was the loudest.

ELI5 - Neural Networks

Simplified, that's how neural networks work. The circle gets some input (stop sign image), detects certain properties (round edges), and screams when it sees it. The more intense the property, the louder they scream.

Judo team did well on the tournament, so the coach asks them where are they going to eat to celebrate?

Each kid starts screaming their preference (burgers, pizza, etc.), and the coach goes with whatever was the loudest.

  • Image Recognition - Ability of AI to detect the object, classify, and recognize it. A mix of Image Detection and Classification.
  •  
  • Object Detection - Identify the object category and locate it using a bounding box for every known object within an image.
  •  
  • Semantic Segmentation - Identify the object category of each pixel for every known object within an image. Labels are class-aware.
  •  
  • Instance Segmentation - Identify each object instance of each pixel for every known object within an image. Labels are instance-aware.

What Hydra Heads Do?

Tesla's FSD Autopilot Screen After Using All the Mentioned AI Techniques

Or is it...
2021
Embracing Tech Readers

Congratulations!

For those that really liked the case study...

How did you like the case study?

You want to share the case study with others?
You want to receive news about new case studies?
You want to talk with me about AI use-cases or content creation?