Tesla's AI Approach for Reaching Full-Self Driving
#002
Mobile Phone - Set to Landscape (Horizontal) view for better visibility
GOLD NUGGETS
Sources - They are clickable if you need more information
Mr.Bean's Autonomous Car
Is there a better approach?
Automation Levels of Autonomous Cars
Tesla Aims to Be the Ultimate Level of Car Evolution
Full Self-Driving
What Does Tesla Do with AI?
Active Safety
Smart Summon
Autopilot
The car keeps its lane and keeps its distance from surrounding vehicles. Autonomy on a highway.
With your phone, summon the car to you from the parking spot.
Automatic emergency braking when noticing a pedestrian.
What Does Tesla Do with AI?
However, to reach this level, it had to solve some problems...
Active Safety
Smart Summon
Autopilot
The car keeps its lane and keeps its distance from surrounding vehicles. Autonomy on a highway.
With your phone, summon the car to you from the parking spot.
Automatic emergency braking when noticing a pedestrian.
Problem #1 - How to Give Eyes to a Car?
What Competitors Use? LiDARs
GOLD NUGGETS
LiDAR uses pulses of light to detect the objects, much like how radar works using radio waves. These pulses can determine the distance and range of an object, providing much needed data to self-driving cars. For example, to avoid a collision, the LiDAR can detect the distance to an object and apply the brakes to slow down the vehicle. [1]
LiDAR (Light Detection and Ranging)
LiDAR Example - Waymo
Waymo—formerly the Google self-driving car project—stands for a new way forward in mobility.
Telsa's Approach - Cameras
For Tesla, cameras are the keys to the future and its CEO sees a future when cameras will enable Tesla to see through the most adverse weather situations.
Lidar is a fool’s errand,” Elon Musk said. “Anyone relying on lidar is doomed. Doomed!
GOLD NUGGETS
Telsa's Approach - Cameras
AP - AutoPilot (software option in cars after 28-Feb-2019)
AP1 - Autopilot 1 (software and hardware prior to AP2)
AP2 - Autopilot 2 (software and hardware after AP1)
LiDAR - Light Detection and Ranging
FSD - Full-Self Driving
GOLD NUGGETS
Tesla Terminology
For Tesla, cameras are the keys to the future and its CEO sees a future when cameras will enable Tesla to see through the most adverse weather situations.
Lidar is a fool’s errand,” Elon Musk said. “Anyone relying on lidar is doomed. Doomed!
GOLD NUGGETS
8 HD cameras delivering 360° view
How Does That Look in Practice?
Cameras vs Lidars
- Better at safely navigating and avoiding hitting objects because they determine object distance
- More expensive and requires more space to implement on cars, thus it tends to make self-driving cars look bulkier.
- Don’t have the range detecting feature of LiDAR.
- It's necessary to use other sensors besides the camera including radars to detect range and distance.
- Problems seeing well enough to avoid danger in the difficult types of conditions.
- Identify objects on the road and read road sign with AI
- Work as a stand-alone system
- The laser’s wavelength can be affected by temperature variations
- Poor Signal-to-Noise Ratio affects the sensors in the LiDAR detector.
- More reliable as a visioning system
How Can Tesla Detect These Different Scenarios with Cameras?
Problem #2 - How Hard Can it Be to Detect One Stop sign?
Temporarily
Located on a Wall
Funny Lights
Different Locations
Passive/Active Position
Heavily Ocluded
Variety of Modifiers
Very Hard Stop Sign Variations
How does Tesla tackle this problem? They use a HydraNet
GOLD NUGGETS
The Lernaean Hydra or Hydra of Lerna, more often known simply as the Hydra is a serpentine water monster in Greek and Roman mythology. The Hydra possessed many heads, the exact number of which varies according to the source. [1]
Hydra of Lerna
Multiple Heads
Similar to ResNet50
Objects
Traffic Lights
Markings
Shared Backbone
HydraNet - Object Detection Network's Architecture
GOLD NUGGETS
Multiple heads are specialized neural networks for image segmentation. They do more than 1000 different tasks because for each detecting task, there can be multiple subtasks that come with it. Shared backbone is a pre-trained neural network that is an important piece when doing transfer learning.
Transfer Learning
Speak one language
Learn how to speak another one
When people acquire knowledge about one task, they can use it to solve related tasks.
The idea of overcoming the isolated learning paradigm and utilizing knowledge acquired for one task to solve related ones.
Transfer Learning
AI KNOWLEDGE
How does Tesla use Transfer Learning?
Tesla uses similar pre-trained model as a backbone like ResNet50
ResNet50 is a 50 layer Residual Neural Network.
How does Tesla use Transfer Learning?
Tesla uses similar pre-trained model as a backbone like ResNet50
ResNet50 is a 50 layer Residual Neural Network.
Residual Neural Netwok?
A Quick Aside on Neural Networks
A neural network is a series of algorithms that tries to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature.
Neural Network
AI KNOWLEDGE
A Quick Aside on Neural Networks
A neural network is a series of algorithms that tries to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature.
Neural Network
Let's keep it simple. Residual neural networks learns by jumping over some layers.
Residual NN
AI KNOWLEDGE
AI KNOWLEDGE
A Quick Aside on Neural Networks
Explain like I'm 5 (ELI5)
Judo team did well on the tournament, so the coach asks them where are they going to eat to celebrate?
Each kid starts screaming their preference (burgers, pizza, etc.), and the coach goes with whatever was the loudest.
ELI5 - Neural Networks
Simplified, that's how neural networks work. The circle gets some input (stop sign image), detects certain properties (round edges), and screams when it sees it. The more intense the property, the louder they scream.
Judo team did well on the tournament, so the coach asks them where are they going to eat to celebrate?
Each kid starts screaming their preference (burgers, pizza, etc.), and the coach goes with whatever was the loudest.
- Image Recognition - Ability of AI to detect the object, classify, and recognize it. A mix of Image Detection and Classification.
- Object Detection - Identify the object category and locate it using a bounding box for every known object within an image.
- Semantic Segmentation - Identify the object category of each pixel for every known object within an image. Labels are class-aware.
- Instance Segmentation - Identify each object instance of each pixel for every known object within an image. Labels are instance-aware.
What Hydra Heads Do?
Tesla's FSD Autopilot Screen After Using All the Mentioned AI Techniques
Or is it...
2021
Embracing Tech Readers
Congratulations!
For those that really liked the case study...
How did you like the case study?
You want to share the case study with others?
You want to receive news about new case studies?
You want to talk with me about AI use-cases or content creation?
