Press "Enter" to skip to content

Comprendre une image, pas si simple pour une machine | Antoine Bordes | TEDxTours


good evening everyone i’ll talk to you

of artificial intelligence tonight and

I told you about the work we are doing

in the facebook lab where I

work is first I wanted

start with a small example imagine

that you are invited to visit friends and

an evening and you arrive but actually

you are visually impaired you approached

the table and there you go out your

phone and you arrive after a

photo you have after a photo you

could not get you to take

a photo and unfortunately the photo and there you

you ask a very simple question in

fact

is it a vegetarian dish

when it’s a question that can be

produce for full reason and so in

done obviously if you ever are

you asked this to a friend he’ll be

able to respond immediately

he will watch on the detected pizza

who are the little bits of who

looks like meat in my opinion

is not all is able to say that oak

is a vegetarian does not eat

meat

this meat pizza so this is

not a vegetarian dish it’s really

not very complicated and well imagine

that if you ask that an assistant

staff who would be in the phone

a supposedly intelligent machine and

well she will be very very annoyed

to answer such questions

and I will try to explain why

and explain how it’s done

for a machine after a while

since able to answer a

similar question so why is this

that it’s complicated for a machine

because in fact this task that are

very simple uses three of

main problems of intelligence

artificial

I know very well

three main problems 1 the

perception is what allows us in

does we detect that on the pizza

there is meat ok 2 this is the

knowledge it’s actually it’s all

knowledge that we have common sense

knowledge of the world it takes

still well right there at this time

there we are able to use the fact

we know because vegetarian does not eat

meat

ok but that’s not obvious either and

yet it’s all salt 3 this is the

reasoning ie it must

not that we set up the detection

meat and that we have also

square

find our ideas that we have knowledge

a vegetable handle the meat

combines the two said actually what

little reasoning that here is very

simple cards for a machine can

to be very complicated and so in fact

I unrolled you during this

presentation a little bit of work

research that is done in these three

directions to try to improve

since the machines are capable of

to do so let’s start with perception

there you see on the screen what the

most of the voice machines when

we show them a picture

So that’s still enough

abstract it’s not a cubic board

it’s supposed to be a photo

appears a scene of normal life

so in fact the maj he ‘s just going to

blocks that are actually around objects

that she detected and when it’s a

little bit more sophisticated but I

am also putting labels so

here I could have added that the machine

has made that the three blocks are in

makes people

this same if I give you this

information you will see a step in

before it hurts me to actually say

what’s going on in this scene so

perceptual research in this

was trying to improve this

representation that the machine can

to have images improve that wants

say maybe able to determine

outline

these agreements does instead of having a

box that is very rude we are going

try to be able to do what we

calls for segmentation that is to say

to be able to surround oneself

completely the objects then a

next step cdc same to put a

skeleton on people what we

call determine the break so there

for example could see a man getting

just like one of the crouching

external is already if you ever see

that’s what’s always to the voices

still no pixel of the image we have

still a lot easier to see

already what’s going on and with a little

little imagination can perhaps be

determined that in fact it was

of a baseball scene agree

facebook is an american company

I should have taken football

so that’s actually the idea it’s

to try to move forward and do that for

an image but to do that for

any image to do that for

all the images that arrive on

facebook and do that too on the

videos so it’s not just doing it

like that

an example like that we have to do it

very quickly and in real time

it takes work but then

when one improves the perception can

do interesting things and

funny we can say ok

will try to mix perception and

knowledge and for that we will create

we have a task that serves us

evaluate how machines

come to combine the two is a

very simple task we give pictures

an image and we ask the machine

describe the legend

ok so here’s where I show you

two photos that serve us as a test and

these are the legends that wrote the

machine ok it’s a machine all by itself

it’s not written by a human

etc. by seeking them reviewed his works

are two years old everyone has been

bluffed actually the legends are

extremely correct ie

right

the legend a man surfing on a

wave is perfect she sticks

perfectly in the picture we people

were there ok the machines are

really extremely strong and adroit and

we show another image that has nothing

to not even the same colors

not at all the same objects and the

machine capable of saying that it is

of a giraffe of a wolf near a tree

when we could say ah yes actually

machines still have the

sense strong enough but we have to do

attention in science so we can

give other images here are others

images that are less common and a

little more surprising and here are the

captions that were generated by

the machine

so once we’ve seen these

images we said okay ok there is still

of work is in fact what is

pass here

it’s actually happening that the machine is

capable of detecting objects that

I demonstrate early we are able to

to do pretty well roughly but we

is able to do it

so in fact there in the right picture

the machine will detect what the

sectors the plane and lvt a road

ok and after she’ll just try to

to make an association what is it

most likely when I see a plane

a road is well there the road is a

tarmac of an airport so it wants

simply say

that an airplane and on the internet reports

it’s all the other pictures with

planes it’s in the sky so

so it’s very simple like the other

photo I detect a motorcycle I detect

something that looks like

good sand dust ben it’s simple

it is someone who rolls on a beach

it’s obvious here and is actually so

you come back to remember both

pictures that I showed before khaki for

which it worked

in fact the reasoning there is

very very simple in the photo with the

surfer actually the machine was that

you are a man standing on two lots

okay that’s it from there

not fifty thousand possibilities

finally I do not know you can be

creative but the machine is not

creative she so she’ll say bomb

a man walking on oit surf

good here and after good we’re surfing on

what on a wave and not fifty

a thousand option so here is a man

surfing on one wave and the other is

the same machine detects a giraffe

Valais Jura are standing all the time

so it was easy so here it is

that gerard of taste so that was

easy and after near a tree

good ben actually it’s luck that is

a fiery image is still enough

high so it does not eat bread I

mean a giraffe standing near a

tree and that’s it’s actually have them so

in fact one had to be very careful

because in fact the machines are

able to detect objects and

make very simple associations in

surface so they will try to

look for the the most the phenomenon the

more common but as soon as we go

in more unexpected things

things that actually asked had a

better knowledge of the world and

phenomena at that moment it’s

always the disaster

another method another task

we use that to import

knowledge of the physical world to

machinery

these tasks too that is very simple

to define we will give the beginning of a

video and forward the machine of

predict the rest and so it’s giving way

very little video we are not at

make whole movies so puts in

does it serve us especially it’s the

little clips it’s to try to see

that the machine can have an idea

physical processes

when I see it’s an object or is it

the object when I move like that I

do not make me appear to

the back but rather to do

to appear before because in fact and the

aspects of displacement and so has it in

do behind me

the first images is when the

video that we gave and from the

when it’s surrounded in red it’s

what after ten stopovers predicts m

achille okay and so actually what

is very interesting for us it’s

that the machine was totally capable

to predict that the whole crowd

of the brass band was going to walk in the

good direction by following the right pace

it was really very very

impressive

however, there is something that is

very interesting for us is that

the trumpets have their explode I

not if you see

so that’s actually what is it

something that is still pretty

unexpected again and why

that’s actually the machines do not have

of common sense idea of ​​what are

the objects they do not have an idea of

things that are very physics

simple ones that are that in fact of the

mass conservation et cetera so in

do they are just going to try to

plan a video that goes well with this

that she saw and actually the problem of

trumpets is that trumpeters

but still struggling to find

is exactly in the same place he can

the climb there goes down to turn it

and so actually the machine when she to

predict the position of the trumpet

she will try to find a species of

right middle good ben the middle right that

just made a big block species john

or it is possible to be the trumpet

so we would prefer it

choose a position even if it’s

not fashion and that’s something

which is very complicated but which also

we give ourselves research leads

extremely interesting so here

I talked about perception I talked about

knowledge and there now I

wanted to talk about the third pillar whose

I spoke at the beginning this is the

reasoning then I know it is a

little bit late maybe but I’m going

still give you some exercises

which are quite complicated so here

these two guys if what we actually have we have

design and in our laboratory for

try to test the capabilities of

reasoning machines here d

extremely hard exercise so I you

read these little stories and give them

the questions so I have in the

garden I pick up the ball

Paul goes to the kitchen where is the

okay balloon ok so here is

normally if no one sleeps in the

world should tell the garden apart

but I can ask you I put

uncomfortable person but

but nevertheless still is it that

this method there two years ago this

task there two years ago the machines

say smart did not have the

solve

okay so that’s why it takes

pay attention

why because in fact it’s very

simple but in fact you still have to

to be able to put in

sentence match so often

to make stories that are

maybe a little longer than the one

I have to find

the two sentences puts them in

correspondence the handsets since in

obviously does in the sentence we speak

of the balloon we give no garden so he

should understand that I

was picking up balloon so jean is holding the

balloon and that in fact the landreau shot is

find the ball but it’s my big

holes people for you it’s obvious

it’s funny bath for a machine

it was not obvious at all and in fact

thanks to his little stories one is

able to increase the complexity this

we’re going to ask the machines so we

will create more and more stories

complicated and so we will ask the

machines to do more reasonings

more complicated and it’s very

interesting for us because we

arrives to testify capacity of

reasoning without having to deal with

perception things I talked about

before will be treated the things of

knowledge of the physical world that are

problems that are complicated

also we arrive like that treated of

isolated way

we can make stories that

finally are also very happy but

which are already much more complicated

even for humans so cooking

and north of the hallway the bathroom

is west of the room the office

is east of the hallway and the kitchen and

south of the room how to get from

office in the room is here is needed

to follow the customer is not m here

it’s not much more words on

all very simple there are four sentences

there is a question is the asset the wii

already it’s more complicated is actually

why and it’s something that

actually joins what we do in the

machines too it’s a matter of

knowledge representation

we are not used to

represent a plan in the form of

phrases like that it’s still

it’s not absolutely natural if I

give you the plan like that it’s

extremely obvious you find everything

away as will the office

office colors agents actually

had to go through hallway but he

also had to go through the kitchens

but hey it’s a lot of voices there and

so in fact we can also test

the way the machines are

able to process reasonings

but also to see how much

how are we giving them

information what are they

represented that has an impact is this

that machines also need to

represent a plan to solve

this task or not the news it’s

they do not need they arrive

a lot easier than we

favor there is that we arrive much better

what

these are but obviously these tasks

synthetic these stories it serves us

tool to try to develop

methods

obviously that’s not what we want

to do in the end we have some goals

slightly higher and above

useful tasks in the end so here is

the idea is to try to extinguish

for example this job then would like

solve the problem from the beginning with the

pizza obviously but there are others

tasks ie ie for example

can tell me we’ll do the same thing

questions and answers except that there

question wants more complicated and then

it’s a step is a text but we’re going

to say that to seek to answer

in all but who pay ok so that’s

still longer it takes more than

time a link and we will give to the

machine just wikipedia articles

that finger links and found himself the

answer is so for example I

asked to our machine for

is developing right now

question that comes back tonight how

was called tower in antiquity and then

everyone knows that obviously I

I guess not I will not demand and

I knew it too obviously I

will say it anyway and that’s for

well pronounce myself there so and so in

do what we would like is that the

so here you can read you m

besides, it’s still difficult

to pronounce Thursday

it’s the road home prefect of

wait for it moment so in any case this

what we would like in this case is that

the machine is capable extending the

question of going to read the articles of

find the right article that is not

necessarily that of the city of towers

besides in this case then in

the article to find the sentence that if the

first is actually to read the sentence

of a phrase that says Caesar autonomous and

the name of the hot empire city that has

preceded lap besides so people

to debate the fact that actually

it was actually turn or not

always besides I ‘ll go there

in it but in any case me what

I’m interested in the machine

passes synthetic tasks that we have

made to an extension that appeals

almost hand process but at a

much larger scale and one can

try to see how the machines

are able to behave

Boy was to be honest the machine

answered Gregory

what was the name of the tower

antiquity we are hiding nothing and

is why but it’s interesting

why because in fact I have formula

with question is what was called and

was actually called is often associated

in feast called people and not

necessarily places where near the

machine ready to try rather

try to look for a person more

that a place so it’s still a mistake

you can say a little bit

important

but suddenly she will rather seek

an important personality connected to

turn in antiquity and in fact she

Gregory told me for the saint’s page

Gregory of towers that was not

correct

so here’s actually in this

presentation I wanted to show you a

a panorama already 2 where we are

intelligence official on

tasks that are really fundamental the

perception the acquisition of

knowledge the reasoning is

how can we try

to combine them to try to make

more and more complex tasks

also show you the limits of

current methods its limits that are

as important for us to try

determine where is the border and where

can we work for

try to improve and that’s how I

told you that he was working at

facebook at the beginning i ‘m still going

end up showing you an app

on which we work that actually

a direct application what I you

showed at the beginning so that actually

it’s the one involved that’s where it’s

facebook news feed interface

but such as actually and work it is

targeted for the time not visible

precisely it is accessible for

visually impaired and in fact there is a

it was way

it’s late and in any case there is a

interface that actually allows a voice

to read everything that happens on the wire

news and the problem that arises

is that obviously the flash was

filled with photos and that the photos

can not read them

so in fact what we developed

we continue with the developed

facebook teams

it’s actually actually able to

automatically generate

descriptions that can be him too

which gives a better idea

so there for now what is

deployed it’s in the United States it’s going

to say that in fact the image may contain

of the arts of the sky and going on outside

so we stop with that we are rather

at first what I demonstrated is

that we have a rather limited description and

in fact we simply with the

methods we do we would like

generate a description like this

there this picture contains a blue sky

beautiful surrounded by huge redwoods

was taken in low-angle is in

does it’s the real picture here and in

does it really align with what

I told you and right now it can

seem to give to people who are

not able to access the photos a

really a better representation

of what’s happening on the cooked tenderloin and

thank you very much

[Applause]

”

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *