
of artificial intelligence tonight and
I told you about the work we are doing
in the facebook lab where I
work is first I wanted
start with a small example imagine
that you are invited to visit friends and
an evening and you arrive but actually
you are visually impaired you approached
the table and there you go out your
phone and you arrive after a
photo you have after a photo you
could not get you to take
a photo and unfortunately the photo and there you
you ask a very simple question in
fact
is it a vegetarian dish
when it’s a question that can be
produce for full reason and so in
done obviously if you ever are
you asked this to a friend he’ll be
able to respond immediately
he will watch on the detected pizza
who are the little bits of who
looks like meat in my opinion
is not all is able to say that oak
is a vegetarian does not eat
meat
this meat pizza so this is
not a vegetarian dish it’s really
not very complicated and well imagine
that if you ask that an assistant
staff who would be in the phone
a supposedly intelligent machine and
well she will be very very annoyed
to answer such questions
and I will try to explain why
and explain how it’s done
for a machine after a while
since able to answer a
similar question so why is this
that it’s complicated for a machine
because in fact this task that are
very simple uses three of
main problems of intelligence
artificial
I know very well
three main problems 1 the
perception is what allows us in
does we detect that on the pizza
there is meat ok 2 this is the
knowledge it’s actually it’s all
knowledge that we have common sense
knowledge of the world it takes
still well right there at this time
there we are able to use the fact
we know because vegetarian does not eat
meat
ok but that’s not obvious either and
yet it’s all salt 3 this is the
reasoning ie it must
not that we set up the detection
meat and that we have also
square
find our ideas that we have knowledge
a vegetable handle the meat
combines the two said actually what
little reasoning that here is very
simple cards for a machine can
to be very complicated and so in fact
I unrolled you during this
presentation a little bit of work
research that is done in these three
directions to try to improve
since the machines are capable of
to do so let’s start with perception
there you see on the screen what the
most of the voice machines when
we show them a picture
So that’s still enough
abstract it’s not a cubic board
it’s supposed to be a photo
appears a scene of normal life
so in fact the maj he ‘s just going to
blocks that are actually around objects
that she detected and when it’s a
little bit more sophisticated but I
am also putting labels so
here I could have added that the machine
has made that the three blocks are in
makes people
this same if I give you this
information you will see a step in
before it hurts me to actually say
what’s going on in this scene so
perceptual research in this
was trying to improve this
representation that the machine can
to have images improve that wants
say maybe able to determine
outline
these agreements does instead of having a
box that is very rude we are going
try to be able to do what we
calls for segmentation that is to say
to be able to surround oneself
completely the objects then a
next step cdc same to put a
skeleton on people what we
call determine the break so there
for example could see a man getting
just like one of the crouching
external is already if you ever see
that’s what’s always to the voices
still no pixel of the image we have
still a lot easier to see
already what’s going on and with a little
little imagination can perhaps be
determined that in fact it was
of a baseball scene agree
facebook is an american company
I should have taken football
so that’s actually the idea it’s
to try to move forward and do that for
an image but to do that for
any image to do that for
all the images that arrive on
facebook and do that too on the
videos so it’s not just doing it
like that
an example like that we have to do it
very quickly and in real time
it takes work but then
when one improves the perception can
do interesting things and
funny we can say ok
will try to mix perception and
knowledge and for that we will create
we have a task that serves us
evaluate how machines
come to combine the two is a
very simple task we give pictures
an image and we ask the machine
describe the legend
ok so here’s where I show you
two photos that serve us as a test and
these are the legends that wrote the
machine ok it’s a machine all by itself
it’s not written by a human
etc. by seeking them reviewed his works
are two years old everyone has been
bluffed actually the legends are
extremely correct ie
right
the legend a man surfing on a
wave is perfect she sticks
perfectly in the picture we people
were there ok the machines are
really extremely strong and adroit and
we show another image that has nothing
to not even the same colors
not at all the same objects and the
machine capable of saying that it is
of a giraffe of a wolf near a tree
when we could say ah yes actually
machines still have the
sense strong enough but we have to do
attention in science so we can
give other images here are others
images that are less common and a
little more surprising and here are the
captions that were generated by
the machine
so once we’ve seen these
images we said okay ok there is still
of work is in fact what is
pass here
it’s actually happening that the machine is
capable of detecting objects that
I demonstrate early we are able to
to do pretty well roughly but we
is able to do it
so in fact there in the right picture
the machine will detect what the
sectors the plane and lvt a road
ok and after she’ll just try to
to make an association what is it
most likely when I see a plane
a road is well there the road is a
tarmac of an airport so it wants
simply say
that an airplane and on the internet reports
it’s all the other pictures with
planes it’s in the sky so
so it’s very simple like the other
photo I detect a motorcycle I detect
something that looks like
good sand dust ben it’s simple
it is someone who rolls on a beach
it’s obvious here and is actually so
you come back to remember both
pictures that I showed before khaki for
which it worked
in fact the reasoning there is
very very simple in the photo with the
surfer actually the machine was that
you are a man standing on two lots
okay that’s it from there
not fifty thousand possibilities
finally I do not know you can be
creative but the machine is not
creative she so she’ll say bomb
a man walking on oit surf
good here and after good we’re surfing on
what on a wave and not fifty
a thousand option so here is a man
surfing on one wave and the other is
the same machine detects a giraffe
Valais Jura are standing all the time
so it was easy so here it is
that gerard of taste so that was
easy and after near a tree
good ben actually it’s luck that is
a fiery image is still enough
high so it does not eat bread I
mean a giraffe standing near a
tree and that’s it’s actually have them so
in fact one had to be very careful
because in fact the machines are
able to detect objects and
make very simple associations in
surface so they will try to
look for the the most the phenomenon the
more common but as soon as we go
in more unexpected things
things that actually asked had a
better knowledge of the world and
phenomena at that moment it’s
always the disaster
another method another task
we use that to import
knowledge of the physical world to
machinery
these tasks too that is very simple
to define we will give the beginning of a
video and forward the machine of
predict the rest and so it’s giving way
very little video we are not at
make whole movies so puts in
does it serve us especially it’s the
little clips it’s to try to see
that the machine can have an idea
physical processes
when I see it’s an object or is it
the object when I move like that I
do not make me appear to
the back but rather to do
to appear before because in fact and the
aspects of displacement and so has it in
do behind me
the first images is when the
video that we gave and from the
when it’s surrounded in red it’s
what after ten stopovers predicts m
achille okay and so actually what
is very interesting for us it’s
that the machine was totally capable
to predict that the whole crowd
of the brass band was going to walk in the
good direction by following the right pace
it was really very very
impressive
however, there is something that is
very interesting for us is that
the trumpets have their explode I
not if you see
so that’s actually what is it
something that is still pretty
unexpected again and why
that’s actually the machines do not have
of common sense idea of what are
the objects they do not have an idea of
things that are very physics
simple ones that are that in fact of the
mass conservation et cetera so in
do they are just going to try to
plan a video that goes well with this
that she saw and actually the problem of
trumpets is that trumpeters
but still struggling to find
is exactly in the same place he can
the climb there goes down to turn it
and so actually the machine when she to
predict the position of the trumpet
she will try to find a species of
right middle good ben the middle right that
just made a big block species john
or it is possible to be the trumpet
so we would prefer it
choose a position even if it’s
not fashion and that’s something
which is very complicated but which also
we give ourselves research leads
extremely interesting so here
I talked about perception I talked about
knowledge and there now I
wanted to talk about the third pillar whose
I spoke at the beginning this is the
reasoning then I know it is a
little bit late maybe but I’m going
still give you some exercises
which are quite complicated so here
these two guys if what we actually have we have
design and in our laboratory for
try to test the capabilities of
reasoning machines here d
extremely hard exercise so I you
read these little stories and give them
the questions so I have in the
garden I pick up the ball
Paul goes to the kitchen where is the
okay balloon ok so here is
normally if no one sleeps in the
world should tell the garden apart
but I can ask you I put
uncomfortable person but
but nevertheless still is it that
this method there two years ago this
task there two years ago the machines
say smart did not have the
solve
okay so that’s why it takes
pay attention
why because in fact it’s very
simple but in fact you still have to
to be able to put in
sentence match so often
to make stories that are
maybe a little longer than the one
I have to find
the two sentences puts them in
correspondence the handsets since in
obviously does in the sentence we speak
of the balloon we give no garden so he
should understand that I
was picking up balloon so jean is holding the
balloon and that in fact the landreau shot is
find the ball but it’s my big
holes people for you it’s obvious
it’s funny bath for a machine
it was not obvious at all and in fact
thanks to his little stories one is
able to increase the complexity this
we’re going to ask the machines so we
will create more and more stories
complicated and so we will ask the
machines to do more reasonings
more complicated and it’s very
interesting for us because we
arrives to testify capacity of
reasoning without having to deal with
perception things I talked about
before will be treated the things of
knowledge of the physical world that are
problems that are complicated
also we arrive like that treated of
isolated way
we can make stories that
finally are also very happy but
which are already much more complicated
even for humans so cooking
and north of the hallway the bathroom
is west of the room the office
is east of the hallway and the kitchen and
south of the room how to get from
office in the room is here is needed
to follow the customer is not m here
it’s not much more words on
all very simple there are four sentences
there is a question is the asset the wii
already it’s more complicated is actually
why and it’s something that
actually joins what we do in the
machines too it’s a matter of
knowledge representation
we are not used to
represent a plan in the form of
phrases like that it’s still
it’s not absolutely natural if I
give you the plan like that it’s
extremely obvious you find everything
away as will the office
office colors agents actually
had to go through hallway but he
also had to go through the kitchens
but hey it’s a lot of voices there and
so in fact we can also test
the way the machines are
able to process reasonings
but also to see how much
how are we giving them
information what are they
represented that has an impact is this
that machines also need to
represent a plan to solve
this task or not the news it’s
they do not need they arrive
a lot easier than we
favor there is that we arrive much better
what
these are but obviously these tasks
synthetic these stories it serves us
tool to try to develop
methods
obviously that’s not what we want
to do in the end we have some goals
slightly higher and above
useful tasks in the end so here is
the idea is to try to extinguish
for example this job then would like
solve the problem from the beginning with the
pizza obviously but there are others
tasks ie ie for example
can tell me we’ll do the same thing
questions and answers except that there
question wants more complicated and then
it’s a step is a text but we’re going
to say that to seek to answer
in all but who pay ok so that’s
still longer it takes more than
time a link and we will give to the
machine just wikipedia articles
that finger links and found himself the
answer is so for example I
asked to our machine for
is developing right now
question that comes back tonight how
was called tower in antiquity and then
everyone knows that obviously I
I guess not I will not demand and
I knew it too obviously I
will say it anyway and that’s for
well pronounce myself there so and so in
do what we would like is that the
so here you can read you m
besides, it’s still difficult
to pronounce Thursday
it’s the road home prefect of
wait for it moment so in any case this
what we would like in this case is that
the machine is capable extending the
question of going to read the articles of
find the right article that is not
necessarily that of the city of towers
besides in this case then in
the article to find the sentence that if the
first is actually to read the sentence
of a phrase that says Caesar autonomous and
the name of the hot empire city that has
preceded lap besides so people
to debate the fact that actually
it was actually turn or not
always besides I ‘ll go there
in it but in any case me what
I’m interested in the machine
passes synthetic tasks that we have
made to an extension that appeals
almost hand process but at a
much larger scale and one can
try to see how the machines
are able to behave
Boy was to be honest the machine
answered Gregory
what was the name of the tower
antiquity we are hiding nothing and
is why but it’s interesting
why because in fact I have formula
with question is what was called and
was actually called is often associated
in feast called people and not
necessarily places where near the
machine ready to try rather
try to look for a person more
that a place so it’s still a mistake
you can say a little bit
important
but suddenly she will rather seek
an important personality connected to
turn in antiquity and in fact she
Gregory told me for the saint’s page
Gregory of towers that was not
correct
so here’s actually in this
presentation I wanted to show you a
a panorama already 2 where we are
intelligence official on
tasks that are really fundamental the
perception the acquisition of
knowledge the reasoning is
how can we try
to combine them to try to make
more and more complex tasks
also show you the limits of
current methods its limits that are
as important for us to try
determine where is the border and where
can we work for
try to improve and that’s how I
told you that he was working at
facebook at the beginning i ‘m still going
end up showing you an app
on which we work that actually
a direct application what I you
showed at the beginning so that actually
it’s the one involved that’s where it’s
facebook news feed interface
but such as actually and work it is
targeted for the time not visible
precisely it is accessible for
visually impaired and in fact there is a
it was way
it’s late and in any case there is a
interface that actually allows a voice
to read everything that happens on the wire
news and the problem that arises
is that obviously the flash was
filled with photos and that the photos
can not read them
so in fact what we developed
we continue with the developed
facebook teams
it’s actually actually able to
automatically generate
descriptions that can be him too
which gives a better idea
so there for now what is
deployed it’s in the United States it’s going
to say that in fact the image may contain
of the arts of the sky and going on outside
so we stop with that we are rather
at first what I demonstrated is
that we have a rather limited description and
in fact we simply with the
methods we do we would like
generate a description like this
there this picture contains a blue sky
beautiful surrounded by huge redwoods
was taken in low-angle is in
does it’s the real picture here and in
does it really align with what
I told you and right now it can
seem to give to people who are
not able to access the photos a
really a better representation
of what’s happening on the cooked tenderloin and
thank you very much
[Applause]
”
Be First to Comment