let’s start tonight with a challenge here we have a video of a public pool
here we have a video of a public pool
here we have a video of a public pool teeming with children we’re going to
teeming with children we’re going to
teeming with children we’re going to play about 15 seconds of the clip and
play about 15 seconds of the clip and
play about 15 seconds of the clip and see if you can identify the child in the
see if you can identify the child in the
see if you can identify the child in the video that is drowning
raise your hand if you were able to
raise your hand if you were able to identify the drowning child all right
identify the drowning child all right
identify the drowning child all right pretty good about ten percent of the
pretty good about ten percent of the
pretty good about ten percent of the audience so for those of you didn’t see
audience so for those of you didn’t see
audience so for those of you didn’t see here’s the kid highlighted in red and
here’s the kid highlighted in red and
here’s the kid highlighted in red and also the lifeguards safely swimming over
also the lifeguards safely swimming over
also the lifeguards safely swimming over to save the child now let’s do one other
to save the child now let’s do one other
to save the child now let’s do one other quick challenge we have a 12 hour long
quick challenge we have a 12 hour long
quick challenge we have a 12 hour long video from a traffic camera in Colorado
video from a traffic camera in Colorado
video from a traffic camera in Colorado we’re going to play the video at 1024 x
we’re going to play the video at 1024 x
we’re going to play the video at 1024 x speed and see if you can identify any
speed and see if you can identify any
speed and see if you can identify any patterns and how cars or pedestrians
patterns and how cars or pedestrians
patterns and how cars or pedestrians move across the intersection
okay now raise your hand if you notice
okay now raise your hand if you notice anything in particular in that great
anything in particular in that great
anything in particular in that great foot any patterns anybody okay it’s
foot any patterns anybody okay it’s
foot any patterns anybody okay it’s about 0% so the pattern we were actually
about 0% so the pattern we were actually
about 0% so the pattern we were actually looking for here was it shown in these
looking for here was it shown in these
looking for here was it shown in these two clips here so we pulled these two
two clips here so we pulled these two
two clips here so we pulled these two clips in the video we just showed you
clips in the video we just showed you
clips in the video we just showed you and in both of them if you look to the
and in both of them if you look to the
and in both of them if you look to the left side of the frame you’ll see the
left side of the frame you’ll see the
left side of the frame you’ll see the town sheriff pull it over to the side of
town sheriff pull it over to the side of
town sheriff pull it over to the side of the street at night getting out of his
the street at night getting out of his
the street at night getting out of his car and dancing in front of the traffic
car and dancing in front of the traffic
car and dancing in front of the traffic camera because he knows it’s there this
camera because he knows it’s there this
camera because he knows it’s there this clip actually appeared four separate
clip actually appeared four separate
clip actually appeared four separate times in the videos we just showed you
times in the videos we just showed you
times in the videos we just showed you the purpose of these examples was to
the purpose of these examples was to
the purpose of these examples was to highlight two abilities if you will the
highlight two abilities if you will the
highlight two abilities if you will the human brain and body mark great at first
human brain and body mark great at first
human brain and body mark great at first of the ability to focus on multiple
of the ability to focus on multiple
of the ability to focus on multiple things at once as illustrated by the
things at once as illustrated by the
things at once as illustrated by the swimming pool example will we really
swimming pool example will we really
swimming pool example will we really aren’t built to see and understand
aren’t built to see and understand
aren’t built to see and understand multiple things happening the scene for
multiple things happening the scene for
multiple things happening the scene for one time any of you have ever done the
one time any of you have ever done the
one time any of you have ever done the Harvard challenge where you watch a
Harvard challenge where you watch a
Harvard challenge where you watch a group of people practicing basketball
group of people practicing basketball
group of people practicing basketball you know how easy it can be to miss even
you know how easy it can be to miss even
you know how easy it can be to miss even painfully obvious things this is called
painfully obvious things this is called
painfully obvious things this is called selective attention and it’s the process
selective attention and it’s the process
selective attention and it’s the process our brains use to filter out the barrage
our brains use to filter out the barrage
our brains use to filter out the barrage of sensory information that they receive
of sensory information that they receive
of sensory information that they receive every second selective attention also
every second selective attention also
every second selective attention also comes up in witness testimony where you
comes up in witness testimony where you
comes up in witness testimony where you have two witnesses squaring on their
have two witnesses squaring on their
have two witnesses squaring on their life that they saw completely different
life that they saw completely different
life that they saw completely different things the second ability is the ability
things the second ability is the ability
things the second ability is the ability to perform a task indefinitely without
to perform a task indefinitely without
to perform a task indefinitely without any drop in performance we’re going to
any drop in performance we’re going to
any drop in performance we’re going to taste of this with the 24-hour video
taste of this with the 24-hour video
taste of this with the 24-hour video from Jacksonville the reality is we
from Jacksonville the reality is we
from Jacksonville the reality is we simply can’t do the same thing 24 hours
simply can’t do the same thing 24 hours
simply can’t do the same thing 24 hours a day 365 days a year which is some of
a day 365 days a year which is some of
a day 365 days a year which is some of us would like to be nonstop productivity
us would like to be nonstop productivity
us would like to be nonstop productivity machines we have to eat we have to
machines we have to eat we have to
machines we have to eat we have to hydrate we can really only operate for
hydrate we can really only operate for
hydrate we can really only operate for 16 hours a day before having to
16 hours a day before having to
16 hours a day before having to completely shut down for another eight
completely shut down for another eight
completely shut down for another eight if we really wanted to watch the whole
if we really wanted to watch the whole
if we really wanted to watch the whole video we’d have to watch it at
video we’d have to watch it at
video we’d have to watch it at ridiculously fast speed which means we
ridiculously fast speed which means we
ridiculously fast speed which means we would miss the important things we were
would miss the important things we were
would miss the important things we were trying to understand despite are not
trying to understand despite are not
trying to understand despite are not having these two abilities we think that
having these two abilities we think that
having these two abilities we think that these two things are the key to solving
these two things are the key to solving
these two things are the key to solving a set of safe
a set of safe
a set of safe and efficiency problems with the spaces
and efficiency problems with the spaces
and efficiency problems with the spaces that we use every single day for example
that we use every single day for example
that we use every single day for example we’ve managed to build absolutely
we’ve managed to build absolutely
we’ve managed to build absolutely incredible highway and city systems yet
incredible highway and city systems yet
incredible highway and city systems yet in the u.s. we still spend forty seven
in the u.s. we still spend forty seven
in the u.s. we still spend forty seven billion hours a year waiting in line in
billion hours a year waiting in line in
billion hours a year waiting in line in these cities and on these highways we
these cities and on these highways we
these cities and on these highways we have retail stores that stock every
have retail stores that stock every
have retail stores that stock every product we could ever possibly need yet
product we could ever possibly need yet
product we could ever possibly need yet when we go to these stores we still have
when we go to these stores we still have
when we go to these stores we still have to ask for an employee to get our size
to ask for an employee to get our size
to ask for an employee to get our size jeans from the back or spend 10 minutes
jeans from the back or spend 10 minutes
jeans from the back or spend 10 minutes wandering around the store trying to
wandering around the store trying to
wandering around the store trying to find the item we’re interested in and
find the item we’re interested in and
find the item we’re interested in and despite the proliferation of workplace
despite the proliferation of workplace
despite the proliferation of workplace safety protocols and employee safety
safety protocols and employee safety
safety protocols and employee safety training we still in the US experience
training we still in the US experience
training we still in the US experience 13 workplace related deaths every single
13 workplace related deaths every single
13 workplace related deaths every single day to solve these problems we need to
day to solve these problems we need to
day to solve these problems we need to measure and analyze our behavior in
measure and analyze our behavior in
measure and analyze our behavior in these spaces indefinitely and
these spaces indefinitely and
these spaces indefinitely and continuously by measuring how many
continuously by measuring how many
continuously by measuring how many millions of cars and pedestrians move
millions of cars and pedestrians move
millions of cars and pedestrians move throughout downtown Manhattan we can
throughout downtown Manhattan we can
throughout downtown Manhattan we can identify global patterns of behavior
identify global patterns of behavior
identify global patterns of behavior that help us to optimize street lights
that help us to optimize street lights
that help us to optimize street lights streets and sidewalks all to minimize
streets and sidewalks all to minimize
streets and sidewalks all to minimize congestion if we can analyze behavior on
congestion if we can analyze behavior on
congestion if we can analyze behavior on a factory floor we can provide real-time
a factory floor we can provide real-time
a factory floor we can provide real-time alerts to workers who miss a step and a
alerts to workers who miss a step and a
alerts to workers who miss a step and a safety protocol preventing them from
safety protocol preventing them from
safety protocol preventing them from entering the potentially dangerous
entering the potentially dangerous
entering the potentially dangerous scenario to accomplish all of this we
scenario to accomplish all of this we
scenario to accomplish all of this we can use cameras and computer vision to
can use cameras and computer vision to
can use cameras and computer vision to analyze our behavior in these spaces
analyze our behavior in these spaces
analyze our behavior in these spaces without calls and without any drop in
without calls and without any drop in
without calls and without any drop in performance over time these devices can
performance over time these devices can
performance over time these devices can operate indefinitely focus on as many
operate indefinitely focus on as many
operate indefinitely focus on as many items as we need them to and with recent
items as we need them to and with recent
items as we need them to and with recent advances in computer vision are getting
advances in computer vision are getting
advances in computer vision are getting closer and closer to recognizing not
closer and closer to recognizing not
closer and closer to recognizing not just objects but actions and behaviors
just objects but actions and behaviors
just objects but actions and behaviors so this brings up a really important
so this brings up a really important
so this brings up a really important question how exactly it is a computer
question how exactly it is a computer
question how exactly it is a computer see an image or a video and why is it
see an image or a video and why is it
see an image or a video and why is it such a difficult problem getting a
such a difficult problem getting a
such a difficult problem getting a computer to understand that image or
computer to understand that image or
computer to understand that image or that video to illustrate the complexity
that video to illustrate the complexity
that video to illustrate the complexity of the problem let’s look at an image
of the problem let’s look at an image
of the problem let’s look at an image from a computer’s perspective so can
from a computer’s perspective so can
from a computer’s perspective so can anyone tell what this is if we remove
anyone tell what this is if we remove
anyone tell what this is if we remove the background it gets slightly easier
the background it gets slightly easier
the background it gets slightly easier to see and if we look at the actual
to see and if we look at the actual
to see and if we look at the actual image we see that it’s a puppy
image we see that it’s a puppy
image we see that it’s a puppy so unfortunately computers are not
so unfortunately computers are not
so unfortunately computers are not blessed with our absolutely incredible
blessed with our absolutely incredible
blessed with our absolutely incredible vision processing system so where we see
vision processing system so where we see
vision processing system so where we see a puppy they just see a whole bunch of
a puppy they just see a whole bunch of
a puppy they just see a whole bunch of digits so the fundamental challenge of
digits so the fundamental challenge of
digits so the fundamental challenge of computer vision then is teaching the
computer vision then is teaching the
computer vision then is teaching the computer to recognize that this set of
computer to recognize that this set of
computer to recognize that this set of digits is a puppy and that set of digits
digits is a puppy and that set of digits
digits is a puppy and that set of digits as a person and so on and so on until
as a person and so on and so on until
as a person and so on and so on until they can understand people places and
they can understand people places and
they can understand people places and absent exactly as we are able to the
absent exactly as we are able to the
absent exactly as we are able to the last ten years deep learning has emerged
last ten years deep learning has emerged
last ten years deep learning has emerged as a promising new approach to computer
as a promising new approach to computer
as a promising new approach to computer vision keep learning is a branch of
vision keep learning is a branch of
vision keep learning is a branch of machine learning that uses artificial
machine learning that uses artificial
machine learning that uses artificial neural networks to analyze patterns and
neural networks to analyze patterns and
neural networks to analyze patterns and data at their core artificial neural
data at their core artificial neural
data at their core artificial neural networks are algorithms that bottle of
networks are algorithms that bottle of
networks are algorithms that bottle of our human brains our brains consists of
our human brains our brains consists of
our human brains our brains consists of hundreds of billions of neurons each of
hundreds of billions of neurons each of
hundreds of billions of neurons each of which is connected to many other neurons
which is connected to many other neurons
which is connected to many other neurons and a neurons basic function is to take
and a neurons basic function is to take
and a neurons basic function is to take in a set of inputs add them up and fire
in a set of inputs add them up and fire
in a set of inputs add them up and fire or activate at the input the sum of
or activate at the input the sum of
or activate at the input the sum of those inputs is greater than a certain
those inputs is greater than a certain
those inputs is greater than a certain value we attempt to recreate this
value we attempt to recreate this
value we attempt to recreate this phenomenon and artificial neural
phenomenon and artificial neural
phenomenon and artificial neural networks but on a much smaller scale
networks but on a much smaller scale
networks but on a much smaller scale using only millions of interconnected
using only millions of interconnected
using only millions of interconnected digital neural units these neural units
digital neural units these neural units
digital neural units these neural units also taken inputs and create a weighted
also taken inputs and create a weighted
also taken inputs and create a weighted sum and activate if that song is greater
sum and activate if that song is greater
sum and activate if that song is greater than some value but in order to be able
than some value but in order to be able
than some value but in order to be able to use these networks to do the task
to use these networks to do the task
to use these networks to do the task they were designed to do we have to
they were designed to do we have to
they were designed to do we have to train and before we train the networks
train and before we train the networks
train and before we train the networks they really resemble the brain of a
they really resemble the brain of a
they really resemble the brain of a human child moments after birth much
human child moments after birth much
human child moments after birth much like a human child these networks have
like a human child these networks have
like a human child these networks have to see and understand examples from the
to see and understand examples from the
to see and understand examples from the real world in order to be able to detect
real world in order to be able to detect
real world in order to be able to detect objects in order to be able to detect
objects in order to be able to detect
objects in order to be able to detect these objects we have to show the
these objects we have to show the
these objects we have to show the network millions of images for example
network millions of images for example
network millions of images for example we might have to show the network
we might have to show the network
we might have to show the network millions of images of dogs before it’s
millions of images of dogs before it’s
millions of images of dogs before it’s able to recognize dogs recently in
able to recognize dogs recently in
able to recognize dogs recently in computer vision the next focus for
computer vision the next focus for
computer vision the next focus for researchers has been detecting over a
researchers has been detecting over a
researchers has been detecting over a thousand different types of objects and
thousand different types of objects and
thousand different types of objects and images for example researchers from
images for example researchers from
images for example researchers from microsoft recently released a network
microsoft recently released a network
microsoft recently released a network like a draw and label over a thousand
like a draw and label over a thousand
like a draw and label over a thousand different types
different types
different types love us here we can see that the network
love us here we can see that the network
love us here we can see that the network was able to detect a person and it’s odd
was able to detect a person and it’s odd
was able to detect a person and it’s odd but also more peculiar things like an
but also more peculiar things like an
but also more peculiar things like an umbrella and a motorcycle but this image
umbrella and a motorcycle but this image
umbrella and a motorcycle but this image this single incident time is not enough
this single incident time is not enough
this single incident time is not enough to truly understand human behavior to do
to truly understand human behavior to do
to truly understand human behavior to do that we have to be able to look at the
that we have to be able to look at the
that we have to be able to look at the sequence of images and say that for
sequence of images and say that for
sequence of images and say that for example this sequence of frames is
example this sequence of frames is
example this sequence of frames is someone doing gymnastics researchers
someone doing gymnastics researchers
someone doing gymnastics researchers from Stanford have recently released the
from Stanford have recently released the
from Stanford have recently released the network that is actually able to watch
network that is actually able to watch
network that is actually able to watch video and understand what happens here
video and understand what happens here
video and understand what happens here at watch the YouTube videos and was able
at watch the YouTube videos and was able
at watch the YouTube videos and was able to say what sport it was for example
to say what sport it was for example
to say what sport it was for example here the sport was basketball in the
here the sport was basketball in the
here the sport was basketball in the retail industry marketers entire job
retail industry marketers entire job
retail industry marketers entire job revolves around understanding as much as
revolves around understanding as much as
revolves around understanding as much as they possibly can about their target
they possibly can about their target
they possibly can about their target customer their average age their gender
customer their average age their gender
customer their average age their gender what they eat were they shop all to
what they eat were they shop all to
what they eat were they shop all to ensure that their marketing campaigns
ensure that their marketing campaigns
ensure that their marketing campaigns resonate as much as possible with
resonate as much as possible with
resonate as much as possible with cameras that understood human behavior
cameras that understood human behavior
cameras that understood human behavior we could help marketers understand their
we could help marketers understand their
we could help marketers understand their customers at an even deeper level by
customers at an even deeper level by
customers at an even deeper level by actually analyzing their behavior in
actually analyzing their behavior in
actually analyzing their behavior in store this would have the consumer
store this would have the consumer
store this would have the consumer benefit of making stores that are both
benefit of making stores that are both
benefit of making stores that are both easier to shop and also better at
easier to shop and also better at
easier to shop and also better at stocking items that customers actually
stocking items that customers actually
stocking items that customers actually care about between 1990 and 2003 in the
care about between 1990 and 2003 in the
care about between 1990 and 2003 in the u.s. there were almost 6,000 cases of
u.s. there were almost 6,000 cases of
u.s. there were almost 6,000 cases of wrong site wrong procedure overall
wrong site wrong procedure overall
wrong site wrong procedure overall patient operations in the future we
patient operations in the future we
patient operations in the future we could place a camera above an operating
could place a camera above an operating
could place a camera above an operating table to actually provide real-time
table to actually provide real-time
table to actually provide real-time warnings to a surgeon when they begin
warnings to a surgeon when they begin
warnings to a surgeon when they begin operating on the wrong patient or the
operating on the wrong patient or the
operating on the wrong patient or the wrong area of the body and this system
wrong area of the body and this system
wrong area of the body and this system could be preloaded with standard details
could be preloaded with standard details
could be preloaded with standard details about the surgery site patient procedure
about the surgery site patient procedure
about the surgery site patient procedure surgical materials list allowing us to
surgical materials list allowing us to
surgical materials list allowing us to go even deeper alerting a surgeon when
go even deeper alerting a surgeon when
go even deeper alerting a surgeon when they leave before an object inside a
they leave before an object inside a
they leave before an object inside a body or warning an anesthesiologist when
body or warning an anesthesiologist when
body or warning an anesthesiologist when they provide an insufficient or
they provide an insufficient or
they provide an insufficient or incorrect dosage continuing in the
incorrect dosage continuing in the
incorrect dosage continuing in the healthcare space we could help reduce
healthcare space we could help reduce
healthcare space we could help reduce the spread of viral infection and
the spread of viral infection and
the spread of viral infection and hospitals the number one method or
hospitals the number one method or
hospitals the number one method or mechanism by which viral infection
mechanism by which viral infection
mechanism by which viral infection spreads in hospitals is health care
spreads in hospitals is health care
spreads in hospitals is health care professionals failing to wash their
professionals failing to wash their
professionals failing to wash their hands between patients with the camera
hands between patients with the camera
hands between patients with the camera system in hospitals we could provide a
system in hospitals we could provide a
system in hospitals we could provide a simple alert when a doctor or a nurse
simple alert when a doctor or a nurse
simple alert when a doctor or a nurse forgets to wash their patient or wash
forgets to wash their patient or wash
forgets to wash their patient or wash their hands after finishing with a
their hands after finishing with a
their hands after finishing with a patient potentially reducing the spread
patient potentially reducing the spread
patient potentially reducing the spread of infection to their next patient this
of infection to their next patient this
of infection to their next patient this could reduce the one in 25 people that
could reduce the one in 25 people that
could reduce the one in 25 people that get a viral infection after staying in
get a viral infection after staying in
get a viral infection after staying in hospital and this technology doesn’t
hospital and this technology doesn’t
hospital and this technology doesn’t only apply to life or death scenarios as
only apply to life or death scenarios as
only apply to life or death scenarios as anyone ever taken an exam in a large
anyone ever taken an exam in a large
anyone ever taken an exam in a large lecture hall and seeing other students
lecture hall and seeing other students
lecture hall and seeing other students cheating if so you know how frustrating
cheating if so you know how frustrating
cheating if so you know how frustrating that is after having studied a week for
that is after having studied a week for
that is after having studied a week for that exam when I was at Purdue most of
that exam when I was at Purdue most of
that exam when I was at Purdue most of my exams consisted of over a hundred
my exams consisted of over a hundred
my exams consisted of over a hundred students a few TAS and a professor
students a few TAS and a professor
students a few TAS and a professor making it effectively impossible for the
making it effectively impossible for the
making it effectively impossible for the professor in the TAS the kitchen
professor in the TAS the kitchen
professor in the TAS the kitchen instances of cheating but with just four
instances of cheating but with just four
instances of cheating but with just four cameras in this auditorium we could
cameras in this auditorium we could
cameras in this auditorium we could watch every scene that house and
watch every scene that house and
watch every scene that house and instantly report suspicious behavior
instantly report suspicious behavior
instantly report suspicious behavior down to the front of the room professor
down to the front of the room professor
down to the front of the room professor could then look at that video and make a
could then look at that video and make a
could then look at that video and make a determination about how to proceed in
determination about how to proceed in
determination about how to proceed in fact watching a large group of people is
fact watching a large group of people is
fact watching a large group of people is incredibly important like we saw in the
incredibly important like we saw in the
incredibly important like we saw in the beginning every day in the US 10 people
beginning every day in the US 10 people
beginning every day in the US 10 people die from incidence of unintentional
die from incidence of unintentional
die from incidence of unintentional dragon parents and lifeguards have to
dragon parents and lifeguards have to
dragon parents and lifeguards have to watch every person in the pool and look
watch every person in the pool and look
watch every person in the pool and look for that split-second moment when their
for that split-second moment when their
for that split-second moment when their head goes underwater with a deep
head goes underwater with a deep
head goes underwater with a deep learning system that understands human
learning system that understands human
learning system that understands human behavior we could assist parents and
behavior we could assist parents and
behavior we could assist parents and lifeguards and acts as a backup set of
lifeguards and acts as a backup set of
lifeguards and acts as a backup set of eyes for modern pools and beaches these
eyes for modern pools and beaches these
eyes for modern pools and beaches these are just a sample of the problems that
are just a sample of the problems that
are just a sample of the problems that we think can be solved with a computer
we think can be solved with a computer
we think can be solved with a computer vision system that understands human
vision system that understands human
vision system that understands human behavior this is where I realistically
behavior this is where I realistically
behavior this is where I realistically see the future of artificial
see the future of artificial
see the future of artificial intelligence as a tool to help people
intelligence as a tool to help people
intelligence as a tool to help people not replacement this is the future we
not replacement this is the future we
not replacement this is the future we are working to build
”
Be First to Comment