great fantastic yeah it’s really a pleasure to be here and I have to say
pleasure to be here and I have to say
pleasure to be here and I have to say that we had the coolest Prime Minister
that we had the coolest Prime Minister
that we had the coolest Prime Minister ever and he’s is even cooler than you
ever and he’s is even cooler than you
ever and he’s is even cooler than you can’t think of it
can’t think of it
can’t think of it great so this is my pleasure to be here
great so this is my pleasure to be here
great so this is my pleasure to be here I’m gonna try to throw some light into
I’m gonna try to throw some light into
I’m gonna try to throw some light into the life of a self-driving car so first
the life of a self-driving car so first
the life of a self-driving car so first let me start by saying you know why we
let me start by saying you know why we
let me start by saying you know why we potentially my wanna have surviving cars
potentially my wanna have surviving cars
potentially my wanna have surviving cars in the first place
in the first place
in the first place unfortunately this more than 1 million
unfortunately this more than 1 million
unfortunately this more than 1 million people a year actually died due to
people a year actually died due to
people a year actually died due to traffic accidents so imagine the
traffic accidents so imagine the
traffic accidents so imagine the potential impact that we can have if you
potential impact that we can have if you
potential impact that we can have if you can reduce that number significantly the
can reduce that number significantly the
can reduce that number significantly the population get older and medicine gets
population get older and medicine gets
population get older and medicine gets better you know we are having more and
better you know we are having more and
better you know we are having more and more issues of mobility so surviving
more issues of mobility so surviving
more issues of mobility so surviving cars can definitely help us out a lot we
cars can definitely help us out a lot we
cars can definitely help us out a lot we shall decrease our pollution right our
shall decrease our pollution right our
shall decrease our pollution right our planet is now getting healthier and what
planet is now getting healthier and what
planet is now getting healthier and what I’m most excited also driving cars is
I’m most excited also driving cars is
I’m most excited also driving cars is really try to think about what is new
really try to think about what is new
really try to think about what is new way what are new ways to do public
way what are new ways to do public
way what are new ways to do public transportation that you know will allow
transportation that you know will allow
transportation that you know will allow us to commute faster and in a much
us to commute faster and in a much
us to commute faster and in a much better way and if you think about it you
better way and if you think about it you
better way and if you think about it you know having a car is really a waste
know having a car is really a waste
know having a car is really a waste right so we should be thinking of
right so we should be thinking of
right so we should be thinking of sharing resources both with materials as
sharing resources both with materials as
sharing resources both with materials as well as well as a space so for those of
well as well as a space so for those of
well as well as a space so for those of you that own a car so your car is bored
you that own a car so your car is bored
you that own a car so your car is bored to death it’s basically sitting around
to death it’s basically sitting around
to death it’s basically sitting around 95 percent of the time just maybe
95 percent of the time just maybe
95 percent of the time just maybe talking to the car next body right and
talking to the car next body right and
talking to the car next body right and so imagine you know the space that the
so imagine you know the space that the
so imagine you know the space that the car is taking right we could use a short
car is taking right we could use a short
car is taking right we could use a short parts you know that we could use that to
parts you know that we could use that to
parts you know that we could use that to make our cities much better so you
make our cities much better so you
make our cities much better so you probably all have seen some way you know
probably all have seen some way you know
probably all have seen some way you know one of these systems for example the
one of these systems for example the
one of these systems for example the Google car the uber cars etc somewhere
Google car the uber cars etc somewhere
Google car the uber cars etc somewhere in the media right and you know these
in the media right and you know these
in the media right and you know these systems are really good that even things
systems are really good that even things
systems are really good that even things like localization but planning avoiding
like localization but planning avoiding
like localization but planning avoiding obstacles
obstacles
obstacles however there is two problems that this
however there is two problems that this
however there is two problems that this systems have that prevent them from
systems have that prevent them from
systems have that prevent them from scaling up to you know a case where each
scaling up to you know a case where each
scaling up to you know a case where each one of us can actually use as a driving
one of us can actually use as a driving
one of us can actually use as a driving car mainly they use this thing on the
car mainly they use this thing on the
car mainly they use this thing on the top the typical is a spinning is a
top the typical is a spinning is a
top the typical is a spinning is a leader and the thing cost this day is
leader and the thing cost this day is
leader and the thing cost this day is our
our
our 100,000 Canadian dollars right so that’s
100,000 Canadian dollars right so that’s
100,000 Canadian dollars right so that’s really expensive and they use
really expensive and they use
really expensive and they use pre-recorded manually annotated maps and
pre-recorded manually annotated maps and
pre-recorded manually annotated maps and just for example to map the cold us it’s
just for example to map the cold us it’s
just for example to map the cold us it’s gonna take around one to ten billion
gonna take around one to ten billion
gonna take around one to ten billion dollars so in Mayan you know having maps
dollars so in Mayan you know having maps
dollars so in Mayan you know having maps up to date now so the goal of my
up to date now so the goal of my
up to date now so the goal of my research has been to try to create
research has been to try to create
research has been to try to create solutions that animals live in which
solutions that animals live in which
solutions that animals live in which rely on cheap sensors okay as well as as
rely on cheap sensors okay as well as as
rely on cheap sensors okay as well as as little prior knowledge as possible and
little prior knowledge as possible and
little prior knowledge as possible and this is actually possible revived
this is actually possible revived
this is actually possible revived because us as humans can actually solve
because us as humans can actually solve
because us as humans can actually solve this task and what do we use we use our
this task and what do we use we use our
this task and what do we use we use our eyes right so we can the same way we can
eyes right so we can the same way we can
eyes right so we can the same way we can actually use cameras in order to try to
actually use cameras in order to try to
actually use cameras in order to try to solve this at this problem now from the
solve this at this problem now from the
solve this at this problem now from the point of view of computer vision right
point of view of computer vision right
point of view of computer vision right which is trying to use computers to
which is trying to use computers to
which is trying to use computers to understand the world via cameras we have
understand the world via cameras we have
understand the world via cameras we have to solve many many different tasks to do
to solve many many different tasks to do
to solve many many different tasks to do so driving cars we need to recognize the
so driving cars we need to recognize the
so driving cars we need to recognize the environment we need to be able to
environment we need to be able to
environment we need to be able to estimate three-dimensional
estimate three-dimensional
estimate three-dimensional representation of the world we need to
representation of the world we need to
representation of the world we need to be able to know how our car is moving on
be able to know how our car is moving on
be able to know how our car is moving on the other cars we need to know what is
the other cars we need to know what is
the other cars we need to know what is happening in front of us right
happening in front of us right
happening in front of us right understand motion and intention of those
understand motion and intention of those
understand motion and intention of those cars and we need to do this in three
cars and we need to do this in three
cars and we need to do this in three dimensions and we need to again
dimensions and we need to again
dimensions and we need to again understand everything that is in the
understand everything that is in the
understand everything that is in the scene that we see now if we want to do
scene that we see now if we want to do
scene that we see now if we want to do this what do we need to do so first of
this what do we need to do so first of
this what do we need to do so first of all we need data and this might sound
all we need data and this might sound
all we need data and this might sound you know something that this is C but
you know something that this is C but
you know something that this is C but it’s not so easy because having access
it’s not so easy because having access
it’s not so easy because having access to one of these platforms costs already
to one of these platforms costs already
to one of these platforms costs already half a million dollars right so it’s not
half a million dollars right so it’s not
half a million dollars right so it’s not something that you can just buy in the
something that you can just buy in the
something that you can just buy in the supermarket also it’s very important to
supermarket also it’s very important to
supermarket also it’s very important to think of the role of machine learning we
think of the role of machine learning we
think of the role of machine learning we cannot fund craft brewers that will
cannot fund craft brewers that will
cannot fund craft brewers that will represent all possible scenarios for a
represent all possible scenarios for a
represent all possible scenarios for a car we need to be able to learn by
car we need to be able to learn by
car we need to be able to learn by driving by example we also need to think
driving by example we also need to think
driving by example we also need to think about models that are able to solve many
about models that are able to solve many
about models that are able to solve many many many tasks that we need to solve
many many tasks that we need to solve
many many tasks that we need to solve for doing self-driving cars and it has
for doing self-driving cars and it has
for doing self-driving cars and it has to be efficient right these systems have
to be efficient right these systems have
to be efficient right these systems have to react this scenario they have to be
to react this scenario they have to be
to react this scenario they have to be real-time and they seem implies working
real-time and they seem implies working
real-time and they seem implies working on you know so well I’m a computer
on you know so well I’m a computer
on you know so well I’m a computer scientist but also in hardware solutions
scientist but also in hardware solutions
scientist but also in hardware solutions for this so in the remaining of this
for this so in the remaining of this
for this so in the remaining of this talking on I a study
talking on I a study
talking on I a study what we have done in the spectrum of
what we have done in the spectrum of
what we have done in the spectrum of making self-driving cars affordable for
making self-driving cars affordable for
making self-driving cars affordable for each one of us
each one of us
each one of us so I mentioned data at the beginning
so I mentioned data at the beginning
so I mentioned data at the beginning right so seven years ago we started you
right so seven years ago we started you
right so seven years ago we started you know trying to think of okay when I work
know trying to think of okay when I work
know trying to think of okay when I work on Vista this Tasi of self-driving cars
on Vista this Tasi of self-driving cars
on Vista this Tasi of self-driving cars can we actually do that and it turns out
can we actually do that and it turns out
can we actually do that and it turns out that it was not possible because we
that it was not possible because we
that it was not possible because we couldn’t have access to any datasets
couldn’t have access to any datasets
couldn’t have access to any datasets right so this was you know we spend the
right so this was you know we spend the
right so this was you know we spend the first two and a half years actually
first two and a half years actually
first two and a half years actually building the benchmarks for people to
building the benchmarks for people to
building the benchmarks for people to compete in order to build the next
compete in order to build the next
compete in order to build the next generation of algorithms that would
generation of algorithms that would
generation of algorithms that would solve other Nomos driving so here you
solve other Nomos driving so here you
solve other Nomos driving so here you see the car that we use for this we
see the car that we use for this we
see the car that we use for this we basically mounted with four cameras this
basically mounted with four cameras this
basically mounted with four cameras this later is spinning on the top as well as
later is spinning on the top as well as
later is spinning on the top as well as GPS and IMU and we went around and drive
GPS and IMU and we went around and drive
GPS and IMU and we went around and drive and in this case the city of cars room
and in this case the city of cars room
and in this case the city of cars room okay so let me play a video so here is
okay so let me play a video so here is
okay so let me play a video so here is the car in the parking lot of ki T so
the car in the parking lot of ki T so
the car in the parking lot of ki T so this is in Assisi culture this is in
this is in Assisi culture this is in
this is in Assisi culture this is in collaboration with one of the labs over
collaboration with one of the labs over
collaboration with one of the labs over there and you know and as I said we went
there and you know and as I said we went
there and you know and as I said we went around I’m driving around you see
around I’m driving around you see
around I’m driving around you see scenarios not so easy scenarios you know
scenarios not so easy scenarios you know
scenarios not so easy scenarios you know where we have illegal maneuvers to you
where we have illegal maneuvers to you
where we have illegal maneuvers to you know really crowded the scenes and then
know really crowded the scenes and then
know really crowded the scenes and then use these fancy sensors to create ground
use these fancy sensors to create ground
use these fancy sensors to create ground truth in order to evaluate how well we
truth in order to evaluate how well we
truth in order to evaluate how well we can do with yes cameras okay so here you
can do with yes cameras okay so here you
can do with yes cameras okay so here you see examples of ground truth for Delta
see examples of ground truth for Delta
see examples of ground truth for Delta current with for how the vehicle moves
current with for how the vehicle moves
current with for how the vehicle moves as well as to the detection and tracking
as well as to the detection and tracking
as well as to the detection and tracking okay so this benchmark was actually a
okay so this benchmark was actually a
okay so this benchmark was actually a huge success and it is still the only
huge success and it is still the only
huge success and it is still the only benchmark out there for self-driving
benchmark out there for self-driving
benchmark out there for self-driving cars despite that it’s been online for
cars despite that it’s been online for
cars despite that it’s been online for now almost five years we have more than
now almost five years we have more than
now almost five years we have more than 20,000 folks actually working on this
20,000 folks actually working on this
20,000 folks actually working on this and more than 500 submission to the
and more than 500 submission to the
and more than 500 submission to the benchmarks were basically every car
benchmarks were basically every car
benchmarks were basically every car company and every you know group working
company and every you know group working
company and every you know group working on this is actually working with our
on this is actually working with our
on this is actually working with our data so let me tell you a little bit of
data so let me tell you a little bit of
data so let me tell you a little bit of what are the tasks that one needs to
what are the tasks that one needs to
what are the tasks that one needs to solve for self-driving cars so the first
solve for self-driving cars so the first
solve for self-driving cars so the first and canonical one is actually trying to
and canonical one is actually trying to
and canonical one is actually trying to yes reconstruct the environment without
yes reconstruct the environment without
yes reconstruct the environment without necessarily understanding what is in
necessarily understanding what is in
necessarily understanding what is in front of us right so this is a Tacitus
front of us right so this is a Tacitus
front of us right so this is a Tacitus as humans we solve all the time we have
as humans we solve all the time we have
as humans we solve all the time we have right so this you know imagine you have
right so this you know imagine you have
right so this you know imagine you have two cameras and we want to reconstruct
two cameras and we want to reconstruct
two cameras and we want to reconstruct right for every pixel that we see in our
right for every pixel that we see in our
right for every pixel that we see in our eyes right how far away it is from us
eyes right how far away it is from us
eyes right how far away it is from us right so this is basically you know our
right so this is basically you know our
right so this is basically you know our one of their latest algorithms that is
one of their latest algorithms that is
one of their latest algorithms that is able to after 40 meters we almost as
able to after 40 meters we almost as
able to after 40 meters we almost as accurate as this really fancy laser
accurate as this really fancy laser
accurate as this really fancy laser scanner while costing you know a camera
scanner while costing you know a camera
scanner while costing you know a camera these days cost $30 right okay so then
these days cost $30 right okay so then
these days cost $30 right okay so then once we reconcile the environment we
once we reconcile the environment we
once we reconcile the environment we need to perceive understand what it
need to perceive understand what it
need to perceive understand what it means what is in front of us right so
means what is in front of us right so
means what is in front of us right so one of the tasks that one is to solve is
one of the tasks that one is to solve is
one of the tasks that one is to solve is pixel labeling right so for every pixel
pixel labeling right so for every pixel
pixel labeling right so for every pixel in one of those images on the Left you
in one of those images on the Left you
in one of those images on the Left you want to be able to say okay is there a
want to be able to say okay is there a
want to be able to say okay is there a card there it’s the road there is there
card there it’s the road there is there
card there it’s the road there is there a person as well as the the thing that
a person as well as the the thing that
a person as well as the the thing that you also need to understand is being
you also need to understand is being
you also need to understand is being able to discriminate between you know
able to discriminate between you know
able to discriminate between you know every one of the individuals every one
every one of the individuals every one
every one of the individuals every one of the cars that is out there so systems
of the cars that is out there so systems
of the cars that is out there so systems right now using convolutional nets using
right now using convolutional nets using
right now using convolutional nets using dip lorina Sonya was talking about
dip lorina Sonya was talking about
dip lorina Sonya was talking about before are able to generalize really
before are able to generalize really
before are able to generalize really well to unseen in this case and since
well to unseen in this case and since
well to unseen in this case and since teens from different cities and create
teens from different cities and create
teens from different cities and create you know incredible results in terms of
you know incredible results in terms of
you know incredible results in terms of performance this is you know only using
performance this is you know only using
performance this is you know only using a single images import once we are able
a single images import once we are able
a single images import once we are able to understand pixels we need to
to understand pixels we need to
to understand pixels we need to understand three dimensions right we
understand three dimensions right we
understand three dimensions right we need to be able to predict trajectories
need to be able to predict trajectories
need to be able to predict trajectories of you know the traffic participants and
of you know the traffic participants and
of you know the traffic participants and this is what you see over here so this
this is what you see over here so this
this is what you see over here so this is our latest results that were actually
is our latest results that were actually
is our latest results that were actually done by a third year you of the
done by a third year you of the
done by a third year you of the undergrad Y which is basically showing a
undergrad Y which is basically showing a
undergrad Y which is basically showing a set of their results better than you
set of their results better than you
set of their results better than you know any of these car companies so when
know any of these car companies so when
know any of these car companies so when you see here is to see trajectories in
you see here is to see trajectories in
you see here is to see trajectories in 2d and 3d that the system is able to
2d and 3d that the system is able to
2d and 3d that the system is able to estimate again in places that the car
estimate again in places that the car
estimate again in places that the car has never driven before so it doesn’t
has never driven before so it doesn’t
has never driven before so it doesn’t record to use Maps or anything it’s just
record to use Maps or anything it’s just
record to use Maps or anything it’s just given the images it’s able to estimate
given the images it’s able to estimate
given the images it’s able to estimate this next we can you can close that so
this next we can you can close that so
this next we can you can close that so next we can actually estimate you know
next we can actually estimate you know
next we can actually estimate you know everything that is happening in the
everything that is happening in the
everything that is happening in the scene so what you see here is a single
scene so what you see here is a single
scene so what you see here is a single camera mounted on top of the car and
camera mounted on top of the car and
camera mounted on top of the car and base
base
base develop the algorithms that are able to
develop the algorithms that are able to
develop the algorithms that are able to translate that image that video into a
translate that image that video into a
translate that image that video into a representation of the scene where you
representation of the scene where you
representation of the scene where you can actually do control right so you see
can actually do control right so you see
can actually do control right so you see here in black the Eco car the car that
here in black the Eco car the car that
here in black the Eco car the car that is driving autonomously
is driving autonomously
is driving autonomously did you see the race or the field of
did you see the race or the field of
did you see the race or the field of view of what the car sees and you see
view of what the car sees and you see
view of what the car sees and you see that is able to compute maps on the fly
that is able to compute maps on the fly
that is able to compute maps on the fly and it’s able to estimate what every
and it’s able to estimate what every
and it’s able to estimate what every single car participant is in the same
single car participant is in the same
single car participant is in the same now it’s color-coded by the intention of
now it’s color-coded by the intention of
now it’s color-coded by the intention of those cars and on the bottom left you
those cars and on the bottom left you
those cars and on the bottom left you can see the traffic scenario without
can see the traffic scenario without
can see the traffic scenario without looking at traffic lights
looking at traffic lights
looking at traffic lights so basically saying what is the last set
so basically saying what is the last set
so basically saying what is the last set of maneuvers that our car can actually
of maneuvers that our car can actually
of maneuvers that our car can actually do safely another thurston we need to
do safely another thurston we need to
do safely another thurston we need to solve is actually the task of set
solve is actually the task of set
solve is actually the task of set localization but if you win this vehicle
localization but if you win this vehicle
localization but if you win this vehicle to be able to drive us around you need
to be able to drive us around you need
to be able to drive us around you need to know at every point in time where it
to know at every point in time where it
to know at every point in time where it is in the scene needed to click on the
is in the scene needed to click on the
is in the scene needed to click on the slide thank you and so the way that
slide thank you and so the way that
slide thank you and so the way that typically this is done in industry is
typically this is done in industry is
typically this is done in industry is that so a car manually drives around the
that so a car manually drives around the
that so a car manually drives around the world and everything that it sees it’s a
world and everything that it sees it’s a
world and everything that it sees it’s a story in a dataset okay and then one is
story in a dataset okay and then one is
story in a dataset okay and then one is driving autonomously you look in your
driving autonomously you look in your
driving autonomously you look in your data set will have a scene a place that
data set will have a scene a place that
data set will have a scene a place that looks exactly the same as what I see in
looks exactly the same as what I see in
looks exactly the same as what I see in front of me okay so so it’s a bit like
front of me okay so so it’s a bit like
front of me okay so so it’s a bit like web search right so this you know works
web search right so this you know works
web search right so this you know works fairly well but it’s not a scalable
fairly well but it’s not a scalable
fairly well but it’s not a scalable right imagine that you need to capture
right imagine that you need to capture
right imagine that you need to capture how the world looks like at every point
how the world looks like at every point
how the world looks like at every point in time in every season every time of
in time in every season every time of
in time in every season every time of the day you know there is construction
the day you know there is construction
the day you know there is construction all the time right the world changes so
all the time right the world changes so
all the time right the world changes so this is why doing self-driving cars it’s
this is why doing self-driving cars it’s
this is why doing self-driving cars it’s also very expensive right so what we
also very expensive right so what we
also very expensive right so what we look at is can we actually create
look at is can we actually create
look at is can we actually create solutions to localization where we don’t
solutions to localization where we don’t
solutions to localization where we don’t need to know the appearance of the world
need to know the appearance of the world
need to know the appearance of the world and as humans right we can do this we
and as humans right we can do this we
and as humans right we can do this we can navigate and then we never know how
can navigate and then we never know how
can navigate and then we never know how a particular Street like Laura Street
a particular Street like Laura Street
a particular Street like Laura Street looks like so it turns out that yes you
looks like so it turns out that yes you
looks like so it turns out that yes you can actually do this and it’s very
can actually do this and it’s very
can actually do this and it’s very simple you can download freely from the
simple you can download freely from the
simple you can download freely from the web opening street maps which is these
web opening street maps which is these
web opening street maps which is these maps that you see in blue in the screen
maps that you see in blue in the screen
maps that you see in blue in the screen and what you can basically do is
and what you can basically do is
and what you can basically do is estimated the trajectory of the car of
estimated the trajectory of the car of
estimated the trajectory of the car of the eco car biasing yes the cameras and
the eco car biasing yes the cameras and
the eco car biasing yes the cameras and the motion of the car is enough to tell
the motion of the car is enough to tell
the motion of the car is enough to tell you where you can potentially be in the
you where you can potentially be in the
you where you can potentially be in the same so at the beginning we’re going to
same so at the beginning we’re going to
same so at the beginning we’re going to have a region of interest where we know
have a region of interest where we know
have a region of interest where we know we are more or less there somewhere and
we are more or less there somewhere and
we are more or less there somewhere and it’s gonna be a uniform distribution
it’s gonna be a uniform distribution
it’s gonna be a uniform distribution right because we don’t know where we are
right because we don’t know where we are
right because we don’t know where we are and as we start driving around then
and as we start driving around then
and as we start driving around then we’re going to shrink it the probability
we’re going to shrink it the probability
we’re going to shrink it the probability distribution of possible locations where
distribution of possible locations where
distribution of possible locations where the car could potentially be okay and
the car could potentially be okay and
the car could potentially be okay and this is where you’re gonna see on the
this is where you’re gonna see on the
this is where you’re gonna see on the bottom right in a second yes for
bottom right in a second yes for
bottom right in a second yes for comparisons in black you’re gonna see
comparisons in black you’re gonna see
comparisons in black you’re gonna see what the ground truth is okay so the
what the ground truth is okay so the
what the ground truth is okay so the beginning is uniform you see a few modes
beginning is uniform you see a few modes
beginning is uniform you see a few modes but very quickly is able to say we’re
but very quickly is able to say we’re
but very quickly is able to say we’re exactly in this location okay so you see
exactly in this location okay so you see
exactly in this location okay so you see if here yes assuming of the same idea
if here yes assuming of the same idea
if here yes assuming of the same idea right a few modes and very quickly
right a few modes and very quickly
right a few modes and very quickly there’s only two months now and we turn
there’s only two months now and we turn
there’s only two months now and we turn alive therefore this you know this is
alive therefore this you know this is
alive therefore this you know this is the only location where we could
the only location where we could
the only location where we could potentially be and this you can narrow
potentially be and this you can narrow
potentially be and this you can narrow the maps from the web is free right you
the maps from the web is free right you
the maps from the web is free right you don’t need to pay anything
don’t need to pay anything
don’t need to pay anything yes mount one or two cameras in your car
yes mount one or two cameras in your car
yes mount one or two cameras in your car so here is a mama Hatton like a scenario
so here is a mama Hatton like a scenario
so here is a mama Hatton like a scenario so you see that in this case there is
so you see that in this case there is
so you see that in this case there is two moles of the distribution right it’s
two moles of the distribution right it’s
two moles of the distribution right it’s very symmetric but again at some point
very symmetric but again at some point
very symmetric but again at some point with one manoeuvre there is only one
with one manoeuvre there is only one
with one manoeuvre there is only one possibility let me just zoom into
possibility let me just zoom into
possibility let me just zoom into different regions sorry with that so
different regions sorry with that so
different regions sorry with that so what you’re gonna see now is the same
what you’re gonna see now is the same
what you’re gonna see now is the same thing but in a map that is the whole
thing but in a map that is the whole
thing but in a map that is the whole city of cultural there is 2,000
city of cultural there is 2,000
city of cultural there is 2,000 kilometers or potential roads okay which
kilometers or potential roads okay which
kilometers or potential roads okay which means that you know there is a lot of
means that you know there is a lot of
means that you know there is a lot of uncertainty of where you can potentially
uncertainty of where you can potentially
uncertainty of where you can potentially be in the scene and with only 30 seconds
be in the scene and with only 30 seconds
be in the scene and with only 30 seconds of driving it’s able to localize very
of driving it’s able to localize very
of driving it’s able to localize very accurately where you are in the scene
accurately where you are in the scene
accurately where you are in the scene okay so hopefully you can see that in
okay so hopefully you can see that in
okay so hopefully you can see that in the screen right again very quickly it
the screen right again very quickly it
the screen right again very quickly it says we are in this highway here this is
says we are in this highway here this is
says we are in this highway here this is where we are the mod is relatively big
where we are the mod is relatively big
where we are the mod is relatively big and at some point we take an exit and
and at some point we take an exit and
and at some point we take an exit and then it strings to you know very precise
then it strings to you know very precise
then it strings to you know very precise location of well this is where we are
Thanks so I mentioned before the mapping
Thanks so I mentioned before the mapping was very expensive right and the reason
was very expensive right and the reason
was very expensive right and the reason why is just the way that people are
why is just the way that people are
why is just the way that people are actually tackling this problem however
actually tackling this problem however
actually tackling this problem however if you could potentially have a further
if you could potentially have a further
if you could potentially have a further worse evolutions to map Ian then we
worse evolutions to map Ian then we
worse evolutions to map Ian then we should be able to use those maps that
should be able to use those maps that
should be able to use those maps that are up-to-date so why is this so
are up-to-date so why is this so
are up-to-date so why is this so expensive so this is the what companies
expensive so this is the what companies
expensive so this is the what companies are doing these days and you probably
are doing these days and you probably
are doing these days and you probably have seen one of these in you know in
have seen one of these in you know in
have seen one of these in you know in the city so the idea is that you have a
the city so the idea is that you have a
the city so the idea is that you have a car you mounted with a whole bunch of
car you mounted with a whole bunch of
car you mounted with a whole bunch of fancy sensors and then you drive around
fancy sensors and then you drive around
fancy sensors and then you drive around the world and then you capture how the
the world and then you capture how the
the world and then you capture how the world looks like again right so it has
world looks like again right so it has
world looks like again right so it has very small coverage and it’s really
very small coverage and it’s really
very small coverage and it’s really expensive this is what the car sees
expensive this is what the car sees
expensive this is what the car sees right it is a miniscule part of the
right it is a miniscule part of the
right it is a miniscule part of the world right however you know there is so
world right however you know there is so
world right however you know there is so many other ways to see the world that we
many other ways to see the world that we
many other ways to see the world that we could use in order to improve the
could use in order to improve the
could use in order to improve the coverage of our maps one example right
coverage of our maps one example right
coverage of our maps one example right is you know Big Brother right
is you know Big Brother right
is you know Big Brother right it sees us all the time of what we are
it sees us all the time of what we are
it sees us all the time of what we are doing right so we could use drones UAVs
doing right so we could use drones UAVs
doing right so we could use drones UAVs planes satellites go around the world
planes satellites go around the world
planes satellites go around the world twice a day every single day right so
twice a day every single day right so
twice a day every single day right so it’s very very up-to-date the imagery
it’s very very up-to-date the imagery
it’s very very up-to-date the imagery now it’s not so trivial how to estimate
now it’s not so trivial how to estimate
now it’s not so trivial how to estimate maps from yes aerial views okay so
maps from yes aerial views okay so
maps from yes aerial views okay so hopefully thanks so what you’re gonna
hopefully thanks so what you’re gonna
hopefully thanks so what you’re gonna see here is a system that is able to as
see here is a system that is able to as
see here is a system that is able to as it drives and using aerial views
it drives and using aerial views
it drives and using aerial views localize where the car is in terms of
localize where the car is in terms of
localize where the car is in terms of the location in geo coordinates and it’s
the location in geo coordinates and it’s
the location in geo coordinates and it’s able to also map clear HT maps of where
able to also map clear HT maps of where
able to also map clear HT maps of where the roads where are the lanes where the
the roads where are the lanes where the
the roads where are the lanes where the parking spots are in yellow here as well
parking spots are in yellow here as well
parking spots are in yellow here as well as in blue where the sidewalks okay so
as in blue where the sidewalks okay so
as in blue where the sidewalks okay so this is really efficient it takes three
this is really efficient it takes three
this is really efficient it takes three seconds per kilometer of road so you
seconds per kilometer of road so you
seconds per kilometer of road so you know in a simple of the computer cluster
know in a simple of the computer cluster
know in a simple of the computer cluster you can actually map the whole world and
you can actually map the whole world and
you can actually map the whole world and so again for mapping the problem is the
so again for mapping the problem is the
so again for mapping the problem is the same as it was for cell driving cars
same as it was for cell driving cars
same as it was for cell driving cars that again only companies like Nvidia
that again only companies like Nvidia
that again only companies like Nvidia for this have access to data so one of
for this have access to data so one of
for this have access to data so one of the last things that the most recent
the last things that the most recent
the last things that the most recent things that we have done in the in the
things that we have done in the in the
things that we have done in the in the lava saga is actually to create the next
lava saga is actually to create the next
lava saga is actually to create the next big benchmark for people to compete in
big benchmark for people to compete in
big benchmark for people to compete in to creating maps in an affordable manner
to creating maps in an affordable manner
to creating maps in an affordable manner and the cool thing is a lecture
and the cool thing is a lecture
and the cool thing is a lecture we selected you know Toronto area to be
we selected you know Toronto area to be
we selected you know Toronto area to be the main area right so the way that we
the main area right so the way that we
the main area right so the way that we created this but was by going to the
created this but was by going to the
created this but was by going to the government and going to the libraries
government and going to the libraries
government and going to the libraries and you seen the HD maps that have
and you seen the HD maps that have
and you seen the HD maps that have already been created and these are
already been created and these are
already been created and these are really really rich they have all sort of
really really rich they have all sort of
really really rich they have all sort of very interesting metadata so we have 3d
very interesting metadata so we have 3d
very interesting metadata so we have 3d buildings of the whole recreated Toronto
buildings of the whole recreated Toronto
buildings of the whole recreated Toronto as well as the location of every single
as well as the location of every single
as well as the location of every single three the location of every single pole
three the location of every single pole
three the location of every single pole the shape of every traffic light every
the shape of every traffic light every
the shape of every traffic light every every tree every pole in the city by an
every tree every pole in the city by an
every tree every pole in the city by an the idea is that now we want
the idea is that now we want
the idea is that now we want participants to be able to estimate all
participants to be able to estimate all
participants to be able to estimate all these different things automatically
and we also generated using again deep
and we also generated using again deep learning neural networks solutions that
learning neural networks solutions that
learning neural networks solutions that are able to estimate this map this Maps
are able to estimate this map this Maps
are able to estimate this map this Maps so here you see related results you’ve
so here you see related results you’ve
so here you see related results you’ve seen only I read news right so there are
seen only I read news right so there are
seen only I read news right so there are almost as good as the ground truth we
almost as good as the ground truth we
almost as good as the ground truth we can estimate you can estimate the
can estimate you can estimate the
can estimate you can estimate the topology of average streets and I
topology of average streets and I
topology of average streets and I mentioned before that you know for the
mentioned before that you know for the
mentioned before that you know for the localization we use these OpenStreetMaps
localization we use these OpenStreetMaps
localization we use these OpenStreetMaps these are generated by people clicking
these are generated by people clicking
these are generated by people clicking in aerial images the location of streets
in aerial images the location of streets
in aerial images the location of streets and we are only 1% below what people do
and we are only 1% below what people do
and we are only 1% below what people do for this for this task and it’s fully
for this for this task and it’s fully
for this for this task and it’s fully automatic so you can imagine and we can
automatic so you can imagine and we can
automatic so you can imagine and we can do this at a larger scale of the whole
do this at a larger scale of the whole
do this at a larger scale of the whole world so you can imagine the impact of
world so you can imagine the impact of
world so you can imagine the impact of doing things like this other things that
doing things like this other things that
doing things like this other things that we can do is create centimeter accurate
we can do is create centimeter accurate
we can do is create centimeter accurate maps for Nia’s Google Street View
maps for Nia’s Google Street View
maps for Nia’s Google Street View panoramas which are available out there
panoramas which are available out there
panoramas which are available out there for the whole world as well okay so just
for the whole world as well okay so just
for the whole world as well okay so just to conclude I show you a couple of
to conclude I show you a couple of
to conclude I show you a couple of things that one is to do in order to
things that one is to do in order to
things that one is to do in order to create self-driving cars for each one of
create self-driving cars for each one of
create self-driving cars for each one of us so things like sensing perception
us so things like sensing perception
us so things like sensing perception localization mapping I show you our
localization mapping I show you our
localization mapping I show you our latest coolest priority in to you know
latest coolest priority in to you know
latest coolest priority in to you know bring in Toronto to the community at
bring in Toronto to the community at
bring in Toronto to the community at large and there is you know many many
large and there is you know many many
large and there is you know many many many things to do in research in order
many things to do in research in order
many things to do in research in order to really solve all the different task
to really solve all the different task
to really solve all the different task forces driving cars and you know it’s
forces driving cars and you know it’s
forces driving cars and you know it’s one acknowledge the fantastic team that
one acknowledge the fantastic team that
one acknowledge the fantastic team that is behind what I show you today and I
is behind what I show you today and I
is behind what I show you today and I just want to finish also with a promise
just want to finish also with a promise
just want to finish also with a promise that we will have self-driving cars in
that we will have self-driving cars in
that we will have self-driving cars in Toronto really really sir thank you
Toronto really really sir thank you
Toronto really really sir thank you [Music]
[Music]
[Music] [Applause]
”
Be First to Comment