humans make sense of the world in a very sophisticated way we make use of our
sophisticated way we make use of our
sophisticated way we make use of our five senses of seeing hearing touching
five senses of seeing hearing touching
five senses of seeing hearing touching smelling and tasting and we combine all
smelling and tasting and we combine all
smelling and tasting and we combine all of that information in such a way that
of that information in such a way that
of that information in such a way that we can understand the world now
we can understand the world now
we can understand the world now computers don’t have access to these
computers don’t have access to these
computers don’t have access to these perceptual modalities computers do
perceptual modalities computers do
perceptual modalities computers do things differently the stuff that they
things differently the stuff that they
things differently the stuff that they do still amazes us if you think of what
do still amazes us if you think of what
do still amazes us if you think of what Google does and how effortlessly it
Google does and how effortlessly it
Google does and how effortlessly it retrieves the things that you’re looking
retrieves the things that you’re looking
retrieves the things that you’re looking for or how Facebook organizes your
for or how Facebook organizes your
for or how Facebook organizes your newsfeed or how you can order
newsfeed or how you can order
newsfeed or how you can order practically and anything on Amazon it’s
practically and anything on Amazon it’s
practically and anything on Amazon it’s amazing and we’re so used to interacting
amazing and we’re so used to interacting
amazing and we’re so used to interacting with computers and they’re so useful to
with computers and they’re so useful to
with computers and they’re so useful to us that we tend to kind of take them for
us that we tend to kind of take them for
us that we tend to kind of take them for granted and what happens then is that we
granted and what happens then is that we
granted and what happens then is that we overestimate their capabilities let me
overestimate their capabilities let me
overestimate their capabilities let me give you an example of what I mean so if
give you an example of what I mean so if
give you an example of what I mean so if we have these two sentences John and
we have these two sentences John and
we have these two sentences John and Mary’s pets like playing in the yard
Mary’s pets like playing in the yard
Mary’s pets like playing in the yard they constructed a fence to keep them
they constructed a fence to keep them
they constructed a fence to keep them safe
safe
safe if I give these two sentences to a human
if I give these two sentences to a human
if I give these two sentences to a human and then I say who does they refer to
and then I say who does they refer to
and then I say who does they refer to and who does them refer to we can do
and who does them refer to we can do
and who does them refer to we can do this without any trouble
this without any trouble
this without any trouble so we understand that John and Mary are
so we understand that John and Mary are
so we understand that John and Mary are people and that they have pets and that
people and that they have pets and that
people and that they have pets and that they like their pets and want to keep
they like their pets and want to keep
they like their pets and want to keep them safe and protected so if you have
them safe and protected so if you have
them safe and protected so if you have your pets playing around in your yard
your pets playing around in your yard
your pets playing around in your yard they might run out onto the street and
they might run out onto the street and
they might run out onto the street and maybe get hit by a car and this is
maybe get hit by a car and this is
maybe get hit by a car and this is something that they don’t want to happen
something that they don’t want to happen
something that they don’t want to happen so for that reason they construct a
so for that reason they construct a
so for that reason they construct a fence to keep their pet safe so they is
fence to keep their pet safe so they is
fence to keep their pet safe so they is John and Mary and then it’s their pets
John and Mary and then it’s their pets
John and Mary and then it’s their pets if I give this to a machine it’s much
if I give this to a machine it’s much
if I give this to a machine it’s much harder and the reason for that is that
harder and the reason for that is that
harder and the reason for that is that if I grammatically analyze this sentence
if I grammatically analyze this sentence
if I grammatically analyze this sentence there’s actually no way of knowing what
there’s actually no way of knowing what
there’s actually no way of knowing what they are what them refers to the way as
they are what them refers to the way as
they are what them refers to the way as humans do this is by looking at the
humans do this is by looking at the
humans do this is by looking at the meaning of these words so looking at the
meaning of these words so looking at the
meaning of these words so looking at the meaning of fence for example another way
meaning of fence for example another way
meaning of fence for example another way of saying this is that what humans are
of saying this is that what humans are
of saying this is that what humans are very good at is having common-sense
very good at is having common-sense
very good at is having common-sense reasoning and this is something that
reasoning and this is something that
reasoning and this is something that machines really struggle with so the
machines really struggle with so the
machines really struggle with so the core to getting to common-sense is
core to getting to common-sense is
core to getting to common-sense is having meaning in computer
having meaning in computer
having meaning in computer and the famous linguist ray jackendoff
and the famous linguist ray jackendoff
and the famous linguist ray jackendoff has said that meaning is the holy grail
has said that meaning is the holy grail
has said that meaning is the holy grail of a variety of disciplines not only
of a variety of disciplines not only
of a variety of disciplines not only linguistics but also philosophy
linguistics but also philosophy
linguistics but also philosophy psychology and the neurosciences and I
psychology and the neurosciences and I
psychology and the neurosciences and I would like to argue here that artificial
would like to argue here that artificial
would like to argue here that artificial intelligence the research discipline is
intelligence the research discipline is
intelligence the research discipline is very much a part of that list of
very much a part of that list of
very much a part of that list of disciplines where meaning is the Holy
disciplines where meaning is the Holy
disciplines where meaning is the Holy Grail so the end goal and the reason for
Grail so the end goal and the reason for
Grail so the end goal and the reason for that is that we absolutely need meaning
that is that we absolutely need meaning
that is that we absolutely need meaning if we want to add common-sense reasoning
if we want to add common-sense reasoning
if we want to add common-sense reasoning so my research has been concerned with
so my research has been concerned with
so my research has been concerned with trying to improve the way that computers
trying to improve the way that computers
trying to improve the way that computers understand meaning by making them learn
understand meaning by making them learn
understand meaning by making them learn to understand meaning in a more human
to understand meaning in a more human
to understand meaning in a more human way one of the problems that we
way one of the problems that we
way one of the problems that we absolutely need to solve if we want to
absolutely need to solve if we want to
absolutely need to solve if we want to have meaning is the so called symbol
have meaning is the so called symbol
have meaning is the so called symbol crowning problem and this is a problem
crowning problem and this is a problem
crowning problem and this is a problem that has a very long history in
that has a very long history in
that has a very long history in philosophy arguably going back all the
philosophy arguably going back all the
philosophy arguably going back all the way to Plato and Aristotle but it’s most
way to Plato and Aristotle but it’s most
way to Plato and Aristotle but it’s most famous recent incarnation with an
famous recent incarnation with an
famous recent incarnation with an application to AI is a thought
application to AI is a thought
application to AI is a thought experiment by the philosopher John
experiment by the philosopher John
experiment by the philosopher John Searle and Searle says suppose you have
Searle and Searle says suppose you have
Searle and Searle says suppose you have what he calls the Chinese room and
what he calls the Chinese room and
what he calls the Chinese room and there’s a person inside this room with a
there’s a person inside this room with a
there’s a person inside this room with a big book of rules that tell him what to
big book of rules that tell him what to
big book of rules that tell him what to do when Chinese characters get input
do when Chinese characters get input
do when Chinese characters get input into the room so we can have someone
into the room so we can have someone
into the room so we can have someone standing outside of the room scribbling
standing outside of the room scribbling
standing outside of the room scribbling Chinese characters on pieces of paper
Chinese characters on pieces of paper
Chinese characters on pieces of paper putting them in and if the person inside
putting them in and if the person inside
putting them in and if the person inside the room does a good job following the
the room does a good job following the
the room does a good job following the rules you might output sequences of
rules you might output sequences of
rules you might output sequences of Chinese characters that are completely
Chinese characters that are completely
Chinese characters that are completely fluent so it might look like he’s
fluent so it might look like he’s
fluent so it might look like he’s actually fluent in Chinese and then
actually fluent in Chinese and then
actually fluent in Chinese and then Searle asked you do you think that this
Searle asked you do you think that this
Searle asked you do you think that this person is fluent in Chinese and phrase
person is fluent in Chinese and phrase
person is fluent in Chinese and phrase this way of course we’re inclined to say
this way of course we’re inclined to say
this way of course we’re inclined to say no it doesn’t he’s just following the
no it doesn’t he’s just following the
no it doesn’t he’s just following the rules in a book that’s exactly the
rules in a book that’s exactly the
rules in a book that’s exactly the problem here what computers do is
problem here what computers do is
problem here what computers do is something that’s very similar to this so
something that’s very similar to this so
something that’s very similar to this so you might phrase this as saying can you
you might phrase this as saying can you
you might phrase this as saying can you learn Chinese from a Chinese dictionary
learn Chinese from a Chinese dictionary
learn Chinese from a Chinese dictionary that’s written completely in Chinese
that’s written completely in Chinese
that’s written completely in Chinese because that is what computers do so the
because that is what computers do so the
because that is what computers do so the way computers learn meaning these days
way computers learn meaning these days
way computers learn meaning these days is through something called the
is through something called the
is through something called the distributional hypothesis and the
distributional hypothesis and the
distributional hypothesis and the distributional hypothesis says you shall
distributional hypothesis says you shall
distributional hypothesis says you shall know a word by the company that it keeps
know a word by the company that it keeps
know a word by the company that it keeps so let me give you an example of what
so let me give you an example of what
so let me give you an example of what that
that
that means we take a large corpus of text
means we take a large corpus of text
means we take a large corpus of text let’s say Wikipedia and if we want to
let’s say Wikipedia and if we want to
let’s say Wikipedia and if we want to know the meaning of cat and dog we look
know the meaning of cat and dog we look
know the meaning of cat and dog we look at all the occurrences of these words in
at all the occurrences of these words in
at all the occurrences of these words in Wikipedia and then we look at the
Wikipedia and then we look at the
Wikipedia and then we look at the company that they keep so we can see
company that they keep so we can see
company that they keep so we can see that cat Co occurs very often with fur
that cat Co occurs very often with fur
that cat Co occurs very often with fur and with birth and dog Coker’s with Bart
and with birth and dog Coker’s with Bart
and with birth and dog Coker’s with Bart so this allows us to see something about
so this allows us to see something about
so this allows us to see something about what these things mean and if we add up
what these things mean and if we add up
what these things mean and if we add up all of these different counts then we
all of these different counts then we
all of these different counts then we can construct something that’s called a
can construct something that’s called a
can construct something that’s called a vector space model and a vector space
vector space model and a vector space
vector space model and a vector space model allows us to see how similar two
model allows us to see how similar two
model allows us to see how similar two things are to each other so every word
things are to each other so every word
things are to each other so every word is a vector and we can calculate how
is a vector and we can calculate how
is a vector and we can calculate how similar these two things are so that
similar these two things are so that
similar these two things are so that allows us to see that dog and cat are
allows us to see that dog and cat are
allows us to see that dog and cat are closer to each other in meaning than
closer to each other in meaning than
closer to each other in meaning than they are to moon because they occur in
they are to moon because they occur in
they are to moon because they occur in more similar contexts now nice thing
more similar contexts now nice thing
more similar contexts now nice thing about these vector spaces is that we can
about these vector spaces is that we can
about these vector spaces is that we can visualize them in a two-dimensional plot
visualize them in a two-dimensional plot
visualize them in a two-dimensional plot so every one of these points here is a
so every one of these points here is a
so every one of these points here is a word and its location is determined by
word and its location is determined by
word and its location is determined by its meaning so for example here we would
its meaning so for example here we would
its meaning so for example here we would have captain politician and soldier
have captain politician and soldier
have captain politician and soldier because they’re all humans and down
because they’re all humans and down
because they’re all humans and down there we have animals snakes and birds
there we have animals snakes and birds
there we have animals snakes and birds and hawks so looking at this space we
and hawks so looking at this space we
and hawks so looking at this space we can investigate how close to things are
can investigate how close to things are
can investigate how close to things are in meaning when we look at something
in meaning when we look at something
in meaning when we look at something from a particular angle and the way we
from a particular angle and the way we
from a particular angle and the way we learn where to put these points these
learn where to put these points these
learn where to put these points these days is by using neural networks you
days is by using neural networks you
days is by using neural networks you might have heard of neural networks or
might have heard of neural networks or
might have heard of neural networks or deep learning it’s very much the hype
deep learning it’s very much the hype
deep learning it’s very much the hype these days as a machine learning
these days as a machine learning
these days as a machine learning technique for learning representations
technique for learning representations
technique for learning representations and the way we do that is by trying to
and the way we do that is by trying to
and the way we do that is by trying to predict the context which means that we
predict the context which means that we
predict the context which means that we suffer from the grounding problem
suffer from the grounding problem
suffer from the grounding problem because we’re defining these meanings in
because we’re defining these meanings in
because we’re defining these meanings in terms of other words now you might say
terms of other words now you might say
terms of other words now you might say the grounding problem is not really a
the grounding problem is not really a
the grounding problem is not really a problem at all if I want to understand
problem at all if I want to understand
problem at all if I want to understand the meaning of democracy I can go to
the meaning of democracy I can go to
the meaning of democracy I can go to Wikipedia and I can read that it’s a
Wikipedia and I can read that it’s a
Wikipedia and I can read that it’s a political system etc etc and in this
political system etc etc and in this
political system etc etc and in this case for abstract words like democracy
case for abstract words like democracy
case for abstract words like democracy you’re probably right
you’re probably right
you’re probably right so the textual modality or the type of
so the textual modality or the type of
so the textual modality or the type of information text is enough for learning
information text is enough for learning
information text is enough for learning the meaning of democracy and indeed if I
the meaning of democracy and indeed if I
the meaning of democracy and indeed if I switch to a different modality pictures
switch to a different modality pictures
switch to a different modality pictures for example this is not really telling
for example this is not really telling
for example this is not really telling me much about what democracy actually
me much about what democracy actually
me much about what democracy actually means sure it’s an
means sure it’s an
means sure it’s an creation so if you type democracy in
creation so if you type democracy in
creation so if you type democracy in Google Images and you look at the
Google Images and you look at the
Google Images and you look at the pictures that you get you get American
pictures that you get you get American
pictures that you get you get American flags and things like that but that’s
flags and things like that but that’s
flags and things like that but that’s not really what democracy means so so
not really what democracy means so so
not really what democracy means so so far so good
far so good
far so good we don’t really suffer from the
we don’t really suffer from the
we don’t really suffer from the grounding problem however if I want to
grounding problem however if I want to
grounding problem however if I want to understand the meaning of cat and I
understand the meaning of cat and I
understand the meaning of cat and I again go to Wikipedia I can learn that
again go to Wikipedia I can learn that
again go to Wikipedia I can learn that it’s a small carnivorous furry mammal
it’s a small carnivorous furry mammal
it’s a small carnivorous furry mammal that people like to use as pets and sure
that people like to use as pets and sure
that people like to use as pets and sure that’s part of what cat is to us but to
that’s part of what cat is to us but to
that’s part of what cat is to us but to us humans the cat is much more than that
us humans the cat is much more than that
us humans the cat is much more than that because the cat is much closer to this
because the cat is much closer to this
because the cat is much closer to this and the reason for that is because of
and the reason for that is because of
and the reason for that is because of the way we interact with the world so we
the way we interact with the world so we
the way we interact with the world so we understand the meaning of cat through
understand the meaning of cat through
understand the meaning of cat through the perceptual experiences that we have
the perceptual experiences that we have
the perceptual experiences that we have so there’s a thing that we look at and
so there’s a thing that we look at and
so there’s a thing that we look at and that we can hear and that we can touch
that we can hear and that we can touch
that we can hear and that we can touch and that together makes the meaning of
and that together makes the meaning of
and that together makes the meaning of cat so my research is about trying to
cat so my research is about trying to
cat so my research is about trying to make computers do this so trying to make
make computers do this so trying to make
make computers do this so trying to make computers use perception in a similar
computers use perception in a similar
computers use perception in a similar way to how humans do that so how would
way to how humans do that so how would
way to how humans do that so how would we go about doing that we can take
we go about doing that we can take
we go about doing that we can take inspiration from the human brain so if I
inspiration from the human brain so if I
inspiration from the human brain so if I show the cat picture to the brain what
show the cat picture to the brain what
show the cat picture to the brain what happens is that the photons fall on your
happens is that the photons fall on your
happens is that the photons fall on your retina and you process this in your
retina and you process this in your
retina and you process this in your brain trying to make sense of it and as
brain trying to make sense of it and as
brain trying to make sense of it and as you go deeper inside your brain you
you go deeper inside your brain you
you go deeper inside your brain you become more and more abstract so you get
become more and more abstract so you get
become more and more abstract so you get a concept representation of cat at some
a concept representation of cat at some
a concept representation of cat at some point deep in your brain and you can
point deep in your brain and you can
point deep in your brain and you can reason with that concept representation
reason with that concept representation
reason with that concept representation and for example think about what would
and for example think about what would
and for example think about what would happen if your cat is not fenced in in
happen if your cat is not fenced in in
happen if your cat is not fenced in in the yard and gets hit by a car so we can
the yard and gets hit by a car so we can
the yard and gets hit by a car so we can do a similar thing again using neural
do a similar thing again using neural
do a similar thing again using neural networks which are inspired by the brain
networks which are inspired by the brain
networks which are inspired by the brain and we show pictures of cats – neural
and we show pictures of cats – neural
and we show pictures of cats – neural networks and we show lots of these
networks and we show lots of these
networks and we show lots of these pictures and we let the neural network
pictures and we let the neural network
pictures and we let the neural network learn what the meaning is of cat and
learn what the meaning is of cat and
learn what the meaning is of cat and then we look at the far end at the
then we look at the far end at the
then we look at the far end at the things that it’s learned and this allows
things that it’s learned and this allows
things that it’s learned and this allows us to see the visual meaning of a
us to see the visual meaning of a
us to see the visual meaning of a concept so instead of the textual
concept so instead of the textual
concept so instead of the textual meaning that we had earlier we can now
meaning that we had earlier we can now
meaning that we had earlier we can now have the visual meaning so what I’d like
have the visual meaning so what I’d like
have the visual meaning so what I’d like to show you is some examples of what
to show you is some examples of what
to show you is some examples of what happens when we make this transition
happens when we make this transition
happens when we make this transition from textual information to visual
from textual information to visual
from textual information to visual information so that you can really see
information so that you can really see
information so that you can really see the difference so here we have some
the difference so here we have some
the difference so here we have some concepts highlighted again in the
concepts highlighted again in the
concepts highlighted again in the spacer remember every point is a worth
spacer remember every point is a worth
spacer remember every point is a worth and his location is determined by its
and his location is determined by its
and his location is determined by its meaning give and take are so close
meaning give and take are so close
meaning give and take are so close linguistically that they actually
linguistically that they actually
linguistically that they actually overlap so you only see take here that’s
overlap so you only see take here that’s
overlap so you only see take here that’s how close they are but of course we know
how close they are but of course we know
how close they are but of course we know that generous should be much closer to
that generous should be much closer to
that generous should be much closer to give than it should be to take we also
give than it should be to take we also
give than it should be to take we also know visually that love and valentine
know visually that love and valentine
know visually that love and valentine should probably be closer to each other
should probably be closer to each other
should probably be closer to each other and the same goes for white in panorama
and the same goes for white in panorama
and the same goes for white in panorama what I’d like to show you is what
what I’d like to show you is what
what I’d like to show you is what happens when we move to visual space and
happens when we move to visual space and
happens when we move to visual space and then you can see the transition
then you can see the transition
then you can see the transition happening life so wide and panorama
happening life so wide and panorama
happening life so wide and panorama moves together love and Valentine give
moves together love and Valentine give
moves together love and Valentine give moves towards generous and away from
moves towards generous and away from
moves towards generous and away from take so what happens there is that you
take so what happens there is that you
take so what happens there is that you have a different perspective on the
have a different perspective on the
have a different perspective on the meaning of these concepts and in this
meaning of these concepts and in this
meaning of these concepts and in this case their perspective is probably
case their perspective is probably
case their perspective is probably closer to what you really want them what
closer to what you really want them what
closer to what you really want them what you got from the textual modality but we
you got from the textual modality but we
you got from the textual modality but we can also do this the other way around
can also do this the other way around
can also do this the other way around remember I said that for democracy you
remember I said that for democracy you
remember I said that for democracy you probably want to learn it from Wikipedia
probably want to learn it from Wikipedia
probably want to learn it from Wikipedia and you don’t want to learn it from
and you don’t want to learn it from
and you don’t want to learn it from Google Images so you want text rather
Google Images so you want text rather
Google Images so you want text rather than vision so if we start from vision
than vision so if we start from vision
than vision so if we start from vision we can see for these abstract concepts
we can see for these abstract concepts
we can see for these abstract concepts chaos Anarchy destruction they’re kind
chaos Anarchy destruction they’re kind
chaos Anarchy destruction they’re kind of evenly distributed in this space and
of evenly distributed in this space and
of evenly distributed in this space and this is not what we want we want these
this is not what we want we want these
this is not what we want we want these three things at the bottom to be close
three things at the bottom to be close
three things at the bottom to be close and far away from order because they are
and far away from order because they are
and far away from order because they are the opposites but because these concepts
the opposites but because these concepts
the opposites but because these concepts are abstract it’s very hard to learn
are abstract it’s very hard to learn
are abstract it’s very hard to learn that from pictures but if we make the
that from pictures but if we make the
that from pictures but if we make the transition we can see that these
transition we can see that these
transition we can see that these concepts move together as we turn
concepts move together as we turn
concepts move together as we turn towards the textual perspective so this
towards the textual perspective so this
towards the textual perspective so this shows that for different concepts there
shows that for different concepts there
shows that for different concepts there are different ways of looking at things
are different ways of looking at things
are different ways of looking at things and sometimes one way is better and
and sometimes one way is better and
and sometimes one way is better and sometimes the other and so far I’ve
sometimes the other and so far I’ve
sometimes the other and so far I’ve talked about vision and text but it’s
talked about vision and text but it’s
talked about vision and text but it’s not restricted to that we can do the
not restricted to that we can do the
not restricted to that we can do the same thing but then with audio so now
same thing but then with audio so now
same thing but then with audio so now we’re thinking about what things sound
we’re thinking about what things sound
we’re thinking about what things sound like so if I have a rocket and an
like so if I have a rocket and an
like so if I have a rocket and an explosion demolition and fireworks they
explosion demolition and fireworks they
explosion demolition and fireworks they should all be very close because they
should all be very close because they
should all be very close because they sound like explosions
sound like explosions
sound like explosions and the same goes for sounds produced by
and the same goes for sounds produced by
and the same goes for sounds produced by water so splash and river and waterfall
water so splash and river and waterfall
water so splash and river and waterfall and the same goes again for sounds
and the same goes again for sounds
and the same goes again for sounds produced by humans so protests or sing
produced by humans so protests or sing
produced by humans so protests or sing or the sound of children so we can make
or the sound of children so we can make
or the sound of children so we can make that transition from text
that transition from text
that transition from text to the audio space and we can see the
to the audio space and we can see the
to the audio space and we can see the water sounds moving to the top left the
water sounds moving to the top left the
water sounds moving to the top left the explosion sounds coming together and the
explosion sounds coming together and the
explosion sounds coming together and the people sounds coming together so if
people sounds coming together so if
people sounds coming together so if we’re thinking about the auditory
we’re thinking about the auditory
we’re thinking about the auditory modality and taking an auditory
modality and taking an auditory
modality and taking an auditory perspective on the meaning of these
perspective on the meaning of these
perspective on the meaning of these words
words
words then this is a much better way of
then this is a much better way of
then this is a much better way of thinking about the meaning of those
thinking about the meaning of those
thinking about the meaning of those concepts I’m not trying to say that
concepts I’m not trying to say that
concepts I’m not trying to say that there’s a best way to look at the
there’s a best way to look at the
there’s a best way to look at the meaning of these concepts what I’m
meaning of these concepts what I’m
meaning of these concepts what I’m trying to say is I pass with so many
trying to say is I pass with so many
trying to say is I pass with so many things the truth is probably somewhere
things the truth is probably somewhere
things the truth is probably somewhere in the middle so what you want ideally
in the middle so what you want ideally
in the middle so what you want ideally is to take information from all of these
is to take information from all of these
is to take information from all of these different perceptual modalities and
different perceptual modalities and
different perceptual modalities and combine it into one multimodal meeting
combine it into one multimodal meeting
combine it into one multimodal meeting representation that covers all of these
representation that covers all of these
representation that covers all of these different aspects of these concepts so
different aspects of these concepts so
different aspects of these concepts so we can have waves and water and ocean
we can have waves and water and ocean
we can have waves and water and ocean and surf and think about them in all of
and surf and think about them in all of
and surf and think about them in all of these different modalities and then as
these different modalities and then as
these different modalities and then as we make the transition towards
we make the transition towards
we make the transition towards multimodal space these things come
multimodal space these things come
multimodal space these things come together and we more accurately model
together and we more accurately model
together and we more accurately model their meaning in the sense of what they
their meaning in the sense of what they
their meaning in the sense of what they mean for humans experiments have been
mean for humans experiments have been
mean for humans experiments have been done with this and and it’s shown that
done with this and and it’s shown that
done with this and and it’s shown that if you go multimodal then you become
if you go multimodal then you become
if you go multimodal then you become much closer to how humans actually think
much closer to how humans actually think
much closer to how humans actually think about meaning so ultimately this is
about meaning so ultimately this is
about meaning so ultimately this is something worthwhile pursuing and my
something worthwhile pursuing and my
something worthwhile pursuing and my general point is that if we want
general point is that if we want
general point is that if we want computers to get common sense reasoning
computers to get common sense reasoning
computers to get common sense reasoning and if we want machines to develop human
and if we want machines to develop human
and if we want machines to develop human level artificial intelligence then we
level artificial intelligence then we
level artificial intelligence then we need to think about the world in a
need to think about the world in a
need to think about the world in a similar way to how humans perceive the
similar way to how humans perceive the
similar way to how humans perceive the world that is we need to make use of
world that is we need to make use of
world that is we need to make use of these perceptual modalities in order to
these perceptual modalities in order to
these perceptual modalities in order to learn the meaning of the things that
learn the meaning of the things that
learn the meaning of the things that happen in the world around us
happen in the world around us
happen in the world around us so I’ve talked about vision and an audio
so I’ve talked about vision and an audio
so I’ve talked about vision and an audio but you can also think of making
but you can also think of making
but you can also think of making machines smell things or taste things or
machines smell things or taste things or
machines smell things or taste things or touch things and ultimately this will
touch things and ultimately this will
touch things and ultimately this will get us much closer to human level
get us much closer to human level
get us much closer to human level artificial intelligence so let’s return
artificial intelligence so let’s return
artificial intelligence so let’s return to the example that we started with the
to the example that we started with the
to the example that we started with the core here is in understanding the
core here is in understanding the
core here is in understanding the meaning and one of the key words there
meaning and one of the key words there
meaning and one of the key words there that we need to understand is the word
that we need to understand is the word
that we need to understand is the word fence because the fence is something
fence because the fence is something
fence because the fence is something that you use to keep things in in such a
that you use to keep things in in such a
that you use to keep things in in such a way that they don’t run out onto the
way that they don’t run out onto the
way that they don’t run out onto the road and get hit by a car so how would
road and get hit by a car so how would
road and get hit by a car so how would we go about understanding the word fence
we go about understanding the word fence
we go about understanding the word fence well one way to do this is we
well one way to do this is we
well one way to do this is we can construct a physical robot and have
can construct a physical robot and have
can construct a physical robot and have him tried to climb the fence that we
him tried to climb the fence that we
him tried to climb the fence that we built maybe you can try to move around
built maybe you can try to move around
built maybe you can try to move around it every once in a while it might
it every once in a while it might
it every once in a while it might succeed in climbing over this fence and
succeed in climbing over this fence and
succeed in climbing over this fence and experience what it’s like to be hit by a
experience what it’s like to be hit by a
experience what it’s like to be hit by a car and maybe discover that that’s not
car and maybe discover that that’s not
car and maybe discover that that’s not something that you want to have happen
something that you want to have happen
something that you want to have happen to you but this would be very expensive
to you but this would be very expensive
to you but this would be very expensive not only because we keep destroying our
not only because we keep destroying our
not only because we keep destroying our robot but because it would take a very
robot but because it would take a very
robot but because it would take a very very long time for us to learn the
very long time for us to learn the
very long time for us to learn the things that humans take for granted if
things that humans take for granted if
things that humans take for granted if we have to do all of these different
we have to do all of these different
we have to do all of these different things individually it would take ages
things individually it would take ages
things individually it would take ages but the alternative is to do this not
but the alternative is to do this not
but the alternative is to do this not with the physical fence but with a
with the physical fence but with a
with the physical fence but with a virtual fence so now we have a virtual
virtual fence so now we have a virtual
virtual fence so now we have a virtual robot in a virtual world trying to climb
robot in a virtual world trying to climb
robot in a virtual world trying to climb the virtual fence and maybe every once
the virtual fence and maybe every once
the virtual fence and maybe every once in a while getting hit by a virtual car
in a while getting hit by a virtual car
in a while getting hit by a virtual car but that’s not much of a problem because
but that’s not much of a problem because
but that’s not much of a problem because we can do this an infinite amount of
we can do this an infinite amount of
we can do this an infinite amount of times and that means that we can learn
times and that means that we can learn
times and that means that we can learn things much more efficiently especially
things much more efficiently especially
things much more efficiently especially with neural networks that require a lot
with neural networks that require a lot
with neural networks that require a lot of data and for that reason there’s a
of data and for that reason there’s a
of data and for that reason there’s a lot of interest in the AI community now
lot of interest in the AI community now
lot of interest in the AI community now to be able to solve these complex
to be able to solve these complex
to be able to solve these complex problems that you see in video games
problems that you see in video games
problems that you see in video games because if AI is ever going to emerge
because if AI is ever going to emerge
because if AI is ever going to emerge it’s much more likely to do so in this
it’s much more likely to do so in this
it’s much more likely to do so in this virtual world than it is in our physical
virtual world than it is in our physical
virtual world than it is in our physical world and who knows maybe one day we’ll
world and who knows maybe one day we’ll
world and who knows maybe one day we’ll have machines that have common sense
have machines that have common sense
have machines that have common sense thank you
thank you
thank you [Applause]
”
Be First to Comment