Press "Enter" to skip to content

Eyes For Self-Driving Cars | Achuta Kadambi | TEDxBeaconStreet


the human eye is one of nature’s finest

the human eye is one of nature’s finest marvels with sensitivity to just three

marvels with sensitivity to just three

marvels with sensitivity to just three colors red green and blue the eye is

colors red green and blue the eye is

colors red green and blue the eye is able to make pictures as wondrous as

able to make pictures as wondrous as

able to make pictures as wondrous as this rainbow as spectacular as the

this rainbow as spectacular as the

this rainbow as spectacular as the Northern Lights and as picturesque as

Northern Lights and as picturesque as

Northern Lights and as picturesque as sunset in Boston now as beautiful as

sunset in Boston now as beautiful as

sunset in Boston now as beautiful as these images are they raise a question

these images are they raise a question

these images are they raise a question how can we get machines to perceive this

how can we get machines to perceive this

how can we get machines to perceive this visual world how can we teach machines

visual world how can we teach machines

visual world how can we teach machines how to see this is one of the core

how to see this is one of the core

how to see this is one of the core challenges in artificial intelligence AI

challenges in artificial intelligence AI

challenges in artificial intelligence AI and it’s a hard one if I were to show

and it’s a hard one if I were to show

and it’s a hard one if I were to show this picture to you to any of you in the

this picture to you to any of you in the

this picture to you to any of you in the audience you could probably tell me that

audience you could probably tell me that

audience you could probably tell me that there are two cars sitting in a parking

there are two cars sitting in a parking

there are two cars sitting in a parking lot maybe even near a Burger King but

lot maybe even near a Burger King but

lot maybe even near a Burger King but when a machine is asked to label this

when a machine is asked to label this

when a machine is asked to label this image to draw boxes around the object

image to draw boxes around the object

image to draw boxes around the object that sees it miss identifies the

that sees it miss identifies the

that sees it miss identifies the reflections in the car as belonging to

reflections in the car as belonging to

reflections in the car as belonging to cyclists and pedestrians to overcome

cyclists and pedestrians to overcome

cyclists and pedestrians to overcome this problem the self-driving car

this problem the self-driving car

this problem the self-driving car industry uses spinning rigs mounted on

industry uses spinning rigs mounted on

industry uses spinning rigs mounted on top of their cars known as lidar lidar

top of their cars known as lidar lidar

top of their cars known as lidar lidar is a new way to see the world it’s a way

is a new way to see the world it’s a way

is a new way to see the world it’s a way to see the world different from how

to see the world different from how

to see the world different from how humans do

humans do

humans do but despite seeing the world in

but despite seeing the world in

but despite seeing the world in different ways these cars are still not

different ways these cars are still not

different ways these cars are still not fully autonomous these light our cars

fully autonomous these light our cars

fully autonomous these light our cars need something else but this has not

need something else but this has not

need something else but this has not stopped artists designers scientists and

stopped artists designers scientists and

stopped artists designers scientists and engineers from imagining the

engineers from imagining the

engineers from imagining the possibilities in this simulation we can

possibilities in this simulation we can

possibilities in this simulation we can see a car as it smoothly avoids

see a car as it smoothly avoids

see a car as it smoothly avoids pedestrians by using the echoes of light

pedestrians by using the echoes of light

pedestrians by using the echoes of light from lidar to take concepts like this

from lidar to take concepts like this

from lidar to take concepts like this into reality our lab at MIT tries to

into reality our lab at MIT tries to

into reality our lab at MIT tries to build superhuman eyes

build superhuman eyes

build superhuman eyes to enable superhuman driving

to enable superhuman driving

to enable superhuman driving we’ve started along this path many years

we’ve started along this path many years

we’ve started along this path many years ago five years ago we had a paper from

ago five years ago we had a paper from

ago five years ago we had a paper from our group that tried to capture light in

our group that tried to capture light in

our group that tried to capture light in motion light as it moves through this

motion light as it moves through this

motion light as it moves through this coke bottle half a million dollars of

coke bottle half a million dollars of

coke bottle half a million dollars of equipment on an optical table and we

equipment on an optical table and we

equipment on an optical table and we were able to get images like this the

were able to get images like this the

were able to get images like this the following year I and my team we built a

following year I and my team we built a

following year I and my team we built a prototype a benchtop prototype that

prototype a benchtop prototype that

prototype a benchtop prototype that scaled this idea to a $500 device when

scaled this idea to a $500 device when

scaled this idea to a $500 device when industry labs and academia and even the

industry labs and academia and even the

industry labs and academia and even the popular press heard about our device

popular press heard about our device

popular press heard about our device they wanted to come by to check it out I

they wanted to come by to check it out I

they wanted to come by to check it out I guess we were in some ways successful

guess we were in some ways successful

guess we were in some ways successful when BBC came by for a live demo they

when BBC came by for a live demo they

when BBC came by for a live demo they liked it enough that they put it on

liked it enough that they put it on

liked it enough that they put it on their homepage but we knew that despite

their homepage but we knew that despite

their homepage but we knew that despite this warm reception there was a way to

this warm reception there was a way to

this warm reception there was a way to go there was a ways to go in order to

go there was a ways to go in order to

go there was a ways to go in order to take this to the next level

take this to the next level

take this to the next level so I’m back on the stage here in 2017 to

so I’m back on the stage here in 2017 to

so I’m back on the stage here in 2017 to present a new technique for lidar and

present a new technique for lidar and

present a new technique for lidar and our invention is to essentially tag

our invention is to essentially tag

our invention is to essentially tag light at every micrometer that’s

light at every micrometer that’s

light at every micrometer that’s one-tenth the width of a human hair to

one-tenth the width of a human hair to

one-tenth the width of a human hair to explain the significance of this

explain the significance of this

explain the significance of this invention let me start with the way

invention let me start with the way

invention let me start with the way things are done today lidar light

things are done today lidar light

things are done today lidar light detection and ranging a vehicle sends a

detection and ranging a vehicle sends a

detection and ranging a vehicle sends a packet of photons at a pedestrian that

packet of photons at a pedestrian that

packet of photons at a pedestrian that it wants to avoid and measures the

it wants to avoid and measures the

it wants to avoid and measures the echoes of light that reflects back we

echoes of light that reflects back we

echoes of light that reflects back we know from our basic high school classes

know from our basic high school classes

know from our basic high school classes distance equals velocity times time

distance equals velocity times time

distance equals velocity times time since we know the speed of light if we

since we know the speed of light if we

since we know the speed of light if we can measure the echo time we can

can measure the echo time we can

can measure the echo time we can estimate the distance the concept is

estimate the distance the concept is

estimate the distance the concept is simple but it’s taken decades and

simple but it’s taken decades and

simple but it’s taken decades and decades to realize this in practice

decades to realize this in practice

decades to realize this in practice because there’s a core culprit here and

because there’s a core culprit here and

because there’s a core culprit here and that’s that the speed of light is very

that’s that the speed of light is very

that’s that the speed of light is very very fastest the fastest thing in the

very fastest the fastest thing in the

very fastest the fastest thing in the universe in one nanosecond in one

universe in one nanosecond in one

universe in one nanosecond in one billionth of a second light travels one

billionth of a second light travels one

billionth of a second light travels one foot so even if you had a billion frames

foot so even if you had a billion frames

foot so even if you had a billion frames per second camera you would only be

per second camera you would only be

per second camera you would only be accurate to one foot the industry has

accurate to one foot the industry has

accurate to one foot the industry has done a wonderful job in making cameras

done a wonderful job in making cameras

done a wonderful job in making cameras that are even faster so that we can tag

that are even faster so that we can tag

that are even faster so that we can tag light not at 1 foot but at 5 millimeters

light not at 1 foot but at 5 millimeters

light not at 1 foot but at 5 millimeters a 5 millimeter light tag for example if

a 5 millimeter light tag for example if

a 5 millimeter light tag for example if I wanted to scan the

I wanted to scan the

I wanted to scan the object this einstein bus would give you

object this einstein bus would give you

object this einstein bus would give you a reconstruction of the object that

a reconstruction of the object that

a reconstruction of the object that looks something like this the

looks something like this the

looks something like this the reconstruction you can make out barely a

reconstruction you can make out barely a

reconstruction you can make out barely a face but the details are lost in

face but the details are lost in

face but the details are lost in situations for mission-critical

situations for mission-critical

situations for mission-critical applications like self-driving cars we

applications like self-driving cars we

applications like self-driving cars we want the best image quality possible and

want the best image quality possible and

want the best image quality possible and that’s our invention we call it

that’s our invention we call it

that’s our invention we call it heterodyne lidar which were trying to

heterodyne lidar which were trying to

heterodyne lidar which were trying to tag light not at five millimeters but at

tag light not at five millimeters but at

tag light not at five millimeters but at two micrometers one-tenth the width of a

two micrometers one-tenth the width of a

two micrometers one-tenth the width of a human hair

human hair

human hair the core idea the way we’re able to

the core idea the way we’re able to

the core idea the way we’re able to solve this is not by building faster and

solve this is not by building faster and

solve this is not by building faster and faster cameras the way we solve this is

faster cameras the way we solve this is

faster cameras the way we solve this is by exploiting the natural physics of

by exploiting the natural physics of

by exploiting the natural physics of light when I take this laser pointer and

light when I take this laser pointer and

light when I take this laser pointer and eischeid it on the floor so I don’t

eischeid it on the floor so I don’t

eischeid it on the floor so I don’t blind you when I shot it on the floor we

blind you when I shot it on the floor we

blind you when I shot it on the floor we often think of this abstraction as a

often think of this abstraction as a

often think of this abstraction as a laser beam a green beam of light just

laser beam a green beam of light just

laser beam a green beam of light just pointing towards the floor but light is

pointing towards the floor but light is

pointing towards the floor but light is far more complex and rich than that

far more complex and rich than that

far more complex and rich than that light is a wave it’s oscillating

light is a wave it’s oscillating

light is a wave it’s oscillating trillions and trillions of times a

trillions and trillions of times a

trillions and trillions of times a second up and down up and down and by by

second up and down up and down and by by

second up and down up and down and by by exploiting these natural oscillations of

exploiting these natural oscillations of

exploiting these natural oscillations of light we’re able to realize very high

light we’re able to realize very high

light we’re able to realize very high path length tagging very high resolution

path length tagging very high resolution

path length tagging very high resolution so the idea is to send a beam of light

so the idea is to send a beam of light

so the idea is to send a beam of light and this wave model at us target so we

and this wave model at us target so we

and this wave model at us target so we send photons and wave fronts at a target

send photons and wave fronts at a target

send photons and wave fronts at a target to enable two micrometer light tagging

to enable two micrometer light tagging

to enable two micrometer light tagging which makes that Einstein statue look

which makes that Einstein statue look

which makes that Einstein statue look much more photorealistic now you can see

much more photorealistic now you can see

much more photorealistic now you can see the wrinkles in the face the lines in

the wrinkles in the face the lines in

the wrinkles in the face the lines in the clothing and even the hair in the

the clothing and even the hair in the

the clothing and even the hair in the mustache of course the key idea is very

mustache of course the key idea is very

mustache of course the key idea is very specialized to my community of optics

specialized to my community of optics

specialized to my community of optics and so that detail is one left best left

and so that detail is one left best left

and so that detail is one left best left unsaid but the nice thing is that this

unsaid but the nice thing is that this

unsaid but the nice thing is that this implementation does not need to be as

implementation does not need to be as

implementation does not need to be as specialized as optical implementations

specialized as optical implementations

specialized as optical implementations it’s something that can be prototyped

it’s something that can be prototyped

it’s something that can be prototyped on a bench top on a regular work desk

on a bench top on a regular work desk

on a bench top on a regular work desk and so when we took this prototype and

and so when we took this prototype and

and so when we took this prototype and built it we want to test it against the

built it we want to test it against the

built it we want to test it against the industry metrics what the lidar industry

industry metrics what the lidar industry

industry metrics what the lidar industry uses today which is long range trying to

uses today which is long range trying to

uses today which is long range trying to achieve half a kilometer target

achieve half a kilometer target

achieve half a kilometer target avoidance evaluating low-power operation

avoidance evaluating low-power operation

avoidance evaluating low-power operation by

by

by Russians beat notes stability and all

Russians beat notes stability and all

Russians beat notes stability and all the metrics that the latter

the metrics that the latter

the metrics that the latter intra-industry is traditionally

intra-industry is traditionally

intra-industry is traditionally interested in and speaking of the light

interested in and speaking of the light

interested in and speaking of the light our industry it’s one of the hottest

our industry it’s one of the hottest

our industry it’s one of the hottest spaces in technology today Kwon ergy all

spaces in technology today Kwon ergy all

spaces in technology today Kwon ergy all these devices are of successful

these devices are of successful

these devices are of successful companies Kwon ergy the company making

companies Kwon ergy the company making

companies Kwon ergy the company making solid state light ours is a technology

solid state light ours is a technology

solid state light ours is a technology unicorn meaning it’s valued at more than

unicorn meaning it’s valued at more than

unicorn meaning it’s valued at more than a billion dollars strobe and way mo are

a billion dollars strobe and way mo are

a billion dollars strobe and way mo are now subsidiaries of Google and GM but

now subsidiaries of Google and GM but

now subsidiaries of Google and GM but all these successful implementations in

all these successful implementations in

all these successful implementations in our world they’re light tagging at

our world they’re light tagging at

our world they’re light tagging at millimetre precision and so the MIT

millimetre precision and so the MIT

millimetre precision and so the MIT proposal is to take this idea to

proposal is to take this idea to

proposal is to take this idea to micrometers 1/10 the width of a human

micrometers 1/10 the width of a human

micrometers 1/10 the width of a human hair

hair

hair in doing so doctors might be able to

in doing so doctors might be able to

in doing so doctors might be able to discriminate light paths within human

discriminate light paths within human

discriminate light paths within human body for example to see through the body

body for example to see through the body

body for example to see through the body without x-rays airplanes are equipped

without x-rays airplanes are equipped

without x-rays airplanes are equipped with light ARS but with micrometer path

with light ARS but with micrometer path

with light ARS but with micrometer path length separation an airplane flying

length separation an airplane flying

length separation an airplane flying over a fruit orchard might not just be

over a fruit orchard might not just be

over a fruit orchard might not just be able to map out a topology but tell you

able to map out a topology but tell you

able to map out a topology but tell you if the fruits are ripe and finally

if the fruits are ripe and finally

if the fruits are ripe and finally autonomous driving maybe we can have

autonomous driving maybe we can have

autonomous driving maybe we can have vehicles that can drive smoothly through

vehicles that can drive smoothly through

vehicles that can drive smoothly through fog taking a dangerous road condition

fog taking a dangerous road condition

fog taking a dangerous road condition like this and making it appear as if the

like this and making it appear as if the

like this and making it appear as if the road was clear these problems are being

road was clear these problems are being

road was clear these problems are being studied in our group and of course the

studied in our group and of course the

studied in our group and of course the technology details can be found on

technology details can be found on

technology details can be found on various web URLs but I’d like to wrap

various web URLs but I’d like to wrap

various web URLs but I’d like to wrap this idea back to the motivation and

this idea back to the motivation and

this idea back to the motivation and that’s the beauty of nature’s eyes the

that’s the beauty of nature’s eyes the

that’s the beauty of nature’s eyes the human eye with sensitivity to three

human eye with sensitivity to three

human eye with sensitivity to three colors is a magical device but it’s not

colors is a magical device but it’s not

colors is a magical device but it’s not as well known that there are animals in

as well known that there are animals in

as well known that there are animals in the kingdom with far superior eyes alien

the kingdom with far superior eyes alien

the kingdom with far superior eyes alien eyes this is a mantis shrimp perhaps

eyes this is a mantis shrimp perhaps

eyes this is a mantis shrimp perhaps some of you who are food connoisseurs

some of you who are food connoisseurs

some of you who are food connoisseurs might know it as a menu item in Japan or

might know it as a menu item in Japan or

might know it as a menu item in Japan or Korea but its visual system is able to

Korea but its visual system is able to

Korea but its visual system is able to pick up 16 bands of colors only three of

pick up 16 bands of colors only three of

pick up 16 bands of colors only three of which humans can see so one can only

which humans can see so one can only

which humans can see so one can only imagine one can literally only imagine

imagine one can literally only imagine

imagine one can literally only imagine how an animal like this would perceive

how an animal like this would perceive

how an animal like this would perceive and it’s these sort of motivations from

and it’s these sort of motivations from

and it’s these sort of motivations from biology bioinspiration

biology bioinspiration

biology bioinspiration that encouraged us to build the

that encouraged us to build the

that encouraged us to build the artificial eye for machines of the

artificial eye for machines of the

artificial eye for machines of the future

future

future thank you

thank you

thank you [Applause]

”

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *