
this is my daughter she was so adorable when she was born when she was two she
when she was born when she was two she
when she was born when she was two she was all about pianos so everywhere we
was all about pianos so everywhere we
was all about pianos so everywhere we went she wanted to play one when she saw
went she wanted to play one when she saw
went she wanted to play one when she saw one when she got five she forgot about
one when she got five she forgot about
one when she got five she forgot about pianos she was all about makeup and
pianos she was all about makeup and
pianos she was all about makeup and lipsticks to tell arias then later she
lipsticks to tell arias then later she
lipsticks to tell arias then later she was all about horses and now I have two
was all about horses and now I have two
was all about horses and now I have two daughters and the stories continued and
daughters and the stories continued and
daughters and the stories continued and said this is how I like to think about
said this is how I like to think about
said this is how I like to think about my photographs I captured my experiences
my photographs I captured my experiences
my photographs I captured my experiences my memories my photos tell a story yes
my memories my photos tell a story yes
my memories my photos tell a story yes all 35,000 of them except they don’t
all 35,000 of them except they don’t
all 35,000 of them except they don’t really what they actually are 35,000
really what they actually are 35,000
really what they actually are 35,000 thumbnails or my harddrive in my camera
thumbnails or my harddrive in my camera
thumbnails or my harddrive in my camera roll and you guys probably have the same
roll and you guys probably have the same
roll and you guys probably have the same story right what we all have it’s just a
story right what we all have it’s just a
story right what we all have it’s just a digital version of the shoebox our
digital version of the shoebox our
digital version of the shoebox our parents had but in our case it’s a
parents had but in our case it’s a
parents had but in our case it’s a ginormous box so I’m a computer vision
ginormous box so I’m a computer vision
ginormous box so I’m a computer vision researcher and I wanted to see if the
researcher and I wanted to see if the
researcher and I wanted to see if the technology that I’m developing the model
technology that I’m developing the model
technology that I’m developing the model people from big foreign video
people from big foreign video
people from big foreign video collections can be useful to create
collections can be useful to create
collections can be useful to create stories from my own photographs and so I
stories from my own photographs and so I
stories from my own photographs and so I had the following idea I said okay so I
had the following idea I said okay so I
had the following idea I said okay so I have this huge photo collection I’m
have this huge photo collection I’m
have this huge photo collection I’m gonna detect the faces and all my photos
gonna detect the faces and all my photos
gonna detect the faces and all my photos now automatically of course I’m gonna
now automatically of course I’m gonna
now automatically of course I’m gonna recognize all the faces then I’m gonna
recognize all the faces then I’m gonna
recognize all the faces then I’m gonna focus on a single person let’s say my
focus on a single person let’s say my
focus on a single person let’s say my daughter and then what I’ll do I’m gonna
daughter and then what I’ll do I’m gonna
daughter and then what I’ll do I’m gonna create a huge graph out of all these
create a huge graph out of all these
create a huge graph out of all these photographs where each photo is
photographs where each photo is
photographs where each photo is represented by a node in this graph and
represented by a node in this graph and
represented by a node in this graph and the edges represent similarities between
the edges represent similarities between
the edges represent similarities between each pair of photographs similarities in
each pair of photographs similarities in
each pair of photographs similarities in terms of the facial expression and if
terms of the facial expression and if
terms of the facial expression and if she smiles or not in terms of the
she smiles or not in terms of the
she smiles or not in terms of the hairstyle they had posed the time when
hairstyle they had posed the time when
hairstyle they had posed the time when the photo was taken and that once I have
the photo was taken and that once I have
the photo was taken and that once I have this graph is tablished I can teach a
this graph is tablished I can teach a
this graph is tablished I can teach a computer
computer
computer to walk on this graph to create a path
to walk on this graph to create a path
to walk on this graph to create a path or if you will a story so this was
or if you will a story so this was
or if you will a story so this was awesome and I thought okay so I’ve got
awesome and I thought okay so I’ve got
awesome and I thought okay so I’ve got the path now I can do it all right
the path now I can do it all right
the path now I can do it all right automatically the only thing that is
automatically the only thing that is
automatically the only thing that is missing is how to represent this set of
missing is how to represent this set of
missing is how to represent this set of photographs and so I realized that
photographs and so I realized that
photographs and so I realized that before the story part that if I’ll just
before the story part that if I’ll just
before the story part that if I’ll just take the detected faces and I put them
take the detected faces and I put them
take the detected faces and I put them one on top of the other and play them
one on top of the other and play them
one on top of the other and play them chronologically it’s it’s kind of
chronologically it’s it’s kind of
chronologically it’s it’s kind of magical so see I can see how she growled
magical so see I can see how she growled
magical so see I can see how she growled in front of my eyes and what happens now
in front of my eyes and what happens now
in front of my eyes and what happens now is that there’s lots of things while
is that there’s lots of things while
is that there’s lots of things while it’s a cool experience there’s lots of
it’s a cool experience there’s lots of
it’s a cool experience there’s lots of things going on right and so it’s very
things going on right and so it’s very
things going on right and so it’s very long and so the combination of the story
long and so the combination of the story
long and so the combination of the story the big graph is a path plus the video
the big graph is a path plus the video
the big graph is a path plus the video representation creates magic we can even
representation creates magic we can even
representation creates magic we can even summarize twenty years of photographs in
summarize twenty years of photographs in
summarize twenty years of photographs in 30 seconds so let’s watch together the
30 seconds so let’s watch together the
30 seconds so let’s watch together the final result what you will notice is
final result what you will notice is
final result what you will notice is that the person is in the focus the
that the person is in the focus the
that the person is in the focus the transitions between the different
transitions between the different
transitions between the different photographs in the final video are
photographs in the final video are
photographs in the final video are really nice you can see that it’s really
really nice you can see that it’s really
really nice you can see that it’s really transition smoothly between their facial
transition smoothly between their facial
transition smoothly between their facial expressions and the time and so on and
expressions and the time and so on and
expressions and the time and so on and it tells the story
thank you so this is fun I was able to
thank you so this is fun I was able to teach a computer to tell his story from
teach a computer to tell his story from
teach a computer to tell his story from my existing photographs now the next
my existing photographs now the next
my existing photographs now the next question that I asked I thought wow this
question that I asked I thought wow this
question that I asked I thought wow this is kind of cool so can I actually teach
is kind of cool so can I actually teach
is kind of cool so can I actually teach a computer to simulate what my daughter
a computer to simulate what my daughter
a computer to simulate what my daughter will look like she’s now nine what will
will look like she’s now nine what will
will look like she’s now nine what will she look like when she’s 25 this seems
she look like when she’s 25 this seems
she look like when she’s 25 this seems really hard though how would I how would
really hard though how would I how would
really hard though how would I how would I go about it so I thought one idea
I go about it so I thought one idea
I go about it so I thought one idea right so if I do have a pair of
right so if I do have a pair of
right so if I do have a pair of photographs of the same person these are
photographs of the same person these are
photographs of the same person these are two real photos in different ages that
two real photos in different ages that
two real photos in different ages that if I have this transition already I can
if I have this transition already I can
if I have this transition already I can teach a computer to how to learn these
teach a computer to how to learn these
teach a computer to how to learn these transitions but I want to do it for any
transitions but I want to do it for any
transitions but I want to do it for any identity for any person out there in the
identity for any person out there in the
identity for any person out there in the world so what I really need is to be
world so what I really need is to be
world so what I really need is to be able to somehow estimate the average
able to somehow estimate the average
able to somehow estimate the average transformation across all people in the
transformation across all people in the
transformation across all people in the world average aging transformation to be
world average aging transformation to be
world average aging transformation to be able to simulate later so but to get the
able to simulate later so but to get the
able to simulate later so but to get the every transformation and it’s and it’s a
every transformation and it’s and it’s a
every transformation and it’s and it’s a representation of all the people in the
representation of all the people in the
representation of all the people in the world so I had the following idea if I
world so I had the following idea if I
world so I had the following idea if I go to the search engine and just look
go to the search engine and just look
go to the search engine and just look for babies I get lots of photos of
for babies I get lots of photos of
for babies I get lots of photos of babies right here’s an example and then
babies right here’s an example and then
babies right here’s an example and then if I use the same technique as in the
if I use the same technique as in the
if I use the same technique as in the video representation I put all the I
video representation I put all the I
video representation I put all the I detect all the faces and I put them one
detect all the faces and I put them one
detect all the faces and I put them one on top of the other but instead of
on top of the other but instead of
on top of the other but instead of showing them as a video I will Everage
showing them as a video I will Everage
showing them as a video I will Everage them per pixel I will get the following
them per pixel I will get the following
them per pixel I will get the following result so I get an average baby and look
result so I get an average baby and look
result so I get an average baby and look this is not a real photo of a baby this
this is not a real photo of a baby this
this is not a real photo of a baby this is an average baby from the search
is an average baby from the search
is an average baby from the search results I can do it for any age so I can
results I can do it for any age so I can
results I can do it for any age so I can search for 25 year old male and I get a
search for 25 year old male and I get a
search for 25 year old male and I get a ton of photos and then average them
ton of photos and then average them
ton of photos and then average them again
again
again and I get a 25 year old every male so
and I get a 25 year old every male so
and I get a 25 year old every male so far okay cool this is my solution and so
far okay cool this is my solution and so
far okay cool this is my solution and so now I have two photographs if you will
now I have two photographs if you will
now I have two photographs if you will not real but you know and I can teach a
not real but you know and I can teach a
not real but you know and I can teach a computer to estimate the average agent
computer to estimate the average agent
computer to estimate the average agent transformation now the final step is
transformation now the final step is
transformation now the final step is that we want to apply the
that we want to apply the
that we want to apply the every transformation on any photo out
every transformation on any photo out
every transformation on any photo out there even for a three-year-old boy here
there even for a three-year-old boy here
there even for a three-year-old boy here and so the another component that we
and so the another component that we
and so the another component that we developed is that we can apply the
developed is that we can apply the
developed is that we can apply the average transformation on this photo but
average transformation on this photo but
average transformation on this photo but we need to make sure that we keep the
we need to make sure that we keep the
we need to make sure that we keep the identity of the person and so we did
identity of the person and so we did
identity of the person and so we did that and we were able to produce from an
that and we were able to produce from an
that and we were able to produce from an three year old photo of this boy
three year old photo of this boy
three year old photo of this boy let’s say 25 year old version of him
we can even produce an eight-year-old
we can even produce an eight-year-old version of him
version of him
version of him so you’ll notice that somehow the phase
so you’ll notice that somehow the phase
so you’ll notice that somehow the phase changes you see the skin changes the
changes you see the skin changes the
changes you see the skin changes the nose shape changed and so on and all
nose shape changed and so on and all
nose shape changed and so on and all those are real aging changes you will
those are real aging changes you will
those are real aging changes you will notice one thing the milk master state
notice one thing the milk master state
notice one thing the milk master state at eight year old this is because the
at eight year old this is because the
at eight year old this is because the algorithm thinks that it’s part of his
algorithm thinks that it’s part of his
algorithm thinks that it’s part of his identity we can do it for any type of
identity we can do it for any type of
identity we can do it for any type of photographs and equality of photographs
photographs and equality of photographs
photographs and equality of photographs here’s one example and he colored
here’s one example and he colored
here’s one example and he colored females and males of course and we also
females and males of course and we also
females and males of course and we also tested it with ground truth is it is it
tested it with ground truth is it is it
tested it with ground truth is it is it close to reality or not so we had a
close to reality or not so we had a
close to reality or not so we had a photograph of one person where we had
photograph of one person where we had
photograph of one person where we had him at age three and we also had him in
him at age three and we also had him in
him at age three and we also had him in like real photographs in different
like real photographs in different
like real photographs in different different other ages 16 and 16 and then
different other ages 16 and 16 and then
different other ages 16 and 16 and then we applied our algorithm on the
we applied our algorithm on the
we applied our algorithm on the three-year-old for of him just on the
three-year-old for of him just on the
three-year-old for of him just on the face area so we created his synthesized
face area so we created his synthesized
face area so we created his synthesized face and blended it into the head of the
face and blended it into the head of the
face and blended it into the head of the real for us and we got this we did a
real for us and we got this we did a
real for us and we got this we did a human study and people had a really hard
human study and people had a really hard
human study and people had a really hard time to decide which one is real which
time to decide which one is real which
time to decide which one is real which one is not so that was promising so I
one is not so that was promising so I
one is not so that was promising so I talked about Xavier look like in the
talked about Xavier look like in the
talked about Xavier look like in the future but you know some of you may ask
future but you know some of you may ask
future but you know some of you may ask other questions that are related to our
other questions that are related to our
other questions that are related to our appearance and photographs and how we
appearance and photographs and how we
appearance and photographs and how we look and have you may look so I don’t
look and have you may look so I don’t
look and have you may look so I don’t know some of you may ask what if I
know some of you may ask what if I
know some of you may ask what if I exercise more will I look differently do
exercise more will I look differently do
exercise more will I look differently do I look good in hats yeah
I look good in hats yeah
I look good in hats yeah what if I become vegetarian right no I
what if I become vegetarian right no I
what if I become vegetarian right no I look differently maybe not so I wanted
look differently maybe not so I wanted
look differently maybe not so I wanted to ask will black hair feed me okay and
to ask will black hair feed me okay and
to ask will black hair feed me okay and so we all know it is well known that
so we all know it is well known that
so we all know it is well known that when people change their hairstyles or
when people change their hairstyles or
when people change their hairstyles or the hair color they can even become
the hair color they can even become
the hair color they can even become unrecognizable it’s pretty dramatic for
unrecognizable it’s pretty dramatic for
unrecognizable it’s pretty dramatic for our appearance and sometimes research
our appearance and sometimes research
our appearance and sometimes research even showed that sometimes the face is
even showed that sometimes the face is
even showed that sometimes the face is less important than the hair actually
less important than the hair actually
less important than the hair actually because we have sometimes unique
because we have sometimes unique
because we have sometimes unique hairstyles and this is how people
hairstyles and this is how people
hairstyles and this is how people recognize us and so it’s scary to make a
recognize us and so it’s scary to make a
recognize us and so it’s scary to make a big change without knowing if it is
big change without knowing if it is
big change without knowing if it is gonna look nice or not and
gonna look nice or not and
gonna look nice or not and I took a photo of me and I wanted to
I took a photo of me and I wanted to
I took a photo of me and I wanted to answer this question and so of course I
answer this question and so of course I
answer this question and so of course I taught a computer to do it and so I
taught a computer to do it and so I
taught a computer to do it and so I render it myself with black hair started
render it myself with black hair started
render it myself with black hair started to write code to create that
to write code to create that
to write code to create that automatically and that looked fun so I
automatically and that looked fun so I
automatically and that looked fun so I said okay so let’s try to render myself
said okay so let’s try to render myself
said okay so let’s try to render myself with different hairstyles in black hair
with different hairstyles in black hair
with different hairstyles in black hair so I got this and that I said and then I
so I got this and that I said and then I
so I got this and that I said and then I started turning to people and people
started turning to people and people
started turning to people and people were excited like oh yeah that’s so cool
were excited like oh yeah that’s so cool
were excited like oh yeah that’s so cool and so I run myself is curly hair and
and so I run myself is curly hair and
and so I run myself is curly hair and different colors and in fact I started
different colors and in fact I started
different colors and in fact I started to develop it and I built it as a search
to develop it and I built it as a search
to develop it and I built it as a search engine where I could type there anything
engine where I could type there anything
engine where I could type there anything kind of imagine an image search engine
kind of imagine an image search engine
kind of imagine an image search engine so I could type there India and imagine
so I could type there India and imagine
so I could type there India and imagine myself in India so I could see how I
myself in India so I could see how I
myself in India so I could see how I look like an Indian clothing and hair
look like an Indian clothing and hair
look like an Indian clothing and hair and Sun or even go back in time and I
and Sun or even go back in time and I
and Sun or even go back in time and I could write 1930 and see how I would
could write 1930 and see how I would
could write 1930 and see how I would look like in 1930s I tried it on 80
look like in 1930s I tried it on 80
look like in 1930s I tried it on 80 different queries so I could see myself
different queries so I could see myself
different queries so I could see myself with dreadlocks and with different hair
with dreadlocks and with different hair
with dreadlocks and with different hair colors and styles and with shaved hair
colors and styles and with shaved hair
colors and styles and with shaved hair and as a bride and in different
and as a bride and in different
and as a bride and in different traditional clothing it was a lot of fun
traditional clothing it was a lot of fun
traditional clothing it was a lot of fun there are many important applications to
there are many important applications to
there are many important applications to the technology that we’re developing in
the technology that we’re developing in
the technology that we’re developing in modeling people from photos and videos I
modeling people from photos and videos I
modeling people from photos and videos I will talk only about one which is really
will talk only about one which is really
will talk only about one which is really close to my heart there are half a
close to my heart there are half a
close to my heart there are half a million missing children out there
million missing children out there
million missing children out there Justin 2016 Justin you say there are
Justin 2016 Justin you say there are
Justin 2016 Justin you say there are similar numbers in other countries these
similar numbers in other countries these
similar numbers in other countries these numbers are insane it turns out that a
numbers are insane it turns out that a
numbers are insane it turns out that a critical component in the search for
critical component in the search for
critical component in the search for missing children are portraits of how
missing children are portraits of how
missing children are portraits of how they may look like in the future
they may look like in the future
they may look like in the future sometimes people go missing for many
sometimes people go missing for many
sometimes people go missing for many many years for example in this example
many years for example in this example
many years for example in this example this woman is missing since 1963 and
this woman is missing since 1963 and
this woman is missing since 1963 and what you see on the right is a sketch
what you see on the right is a sketch
what you see on the right is a sketch that an artist is making a mate to try
that an artist is making a mate to try
that an artist is making a mate to try to predict how she looks like to be able
to predict how she looks like to be able
to predict how she looks like to be able to find her or here’s another
to find her or here’s another
to find her or here’s another sometimes the artists do photoshop’s and
sometimes the artists do photoshop’s and
sometimes the artists do photoshop’s and try to predict an age progression how so
try to predict an age progression how so
try to predict an age progression how so for example in this case but who knows
for example in this case but who knows
for example in this case but who knows if it is still blonde or if she actually
if it is still blonde or if she actually
if it is still blonde or if she actually looks like this this is based on
looks like this this is based on
looks like this this is based on intuitions of artists it’s also very
intuitions of artists it’s also very
intuitions of artists it’s also very hard to do it for half a million for us
hard to do it for half a million for us
hard to do it for half a million for us because it just manual work in Photoshop
because it just manual work in Photoshop
because it just manual work in Photoshop and so we wanted to combine the two
and so we wanted to combine the two
and so we wanted to combine the two projects one were you imagine how a face
projects one were you imagine how a face
projects one were you imagine how a face of a person looks like in the future and
of a person looks like in the future and
of a person looks like in the future and the other one is how to imagine it
the other one is how to imagine it
the other one is how to imagine it combined with a like a full portrait of
combined with a like a full portrait of
combined with a like a full portrait of the person and so here’s an example
the person and so here’s an example
the person and so here’s an example where we take a baby photo and in the
where we take a baby photo and in the
where we take a baby photo and in the search engine we search for a five year
search engine we search for a five year
search engine we search for a five year old male and we combine the h progressed
old male and we combine the h progressed
old male and we combine the h progressed the face and then create also different
the face and then create also different
the face and then create also different hairstyles to try to imagine what if
hairstyles to try to imagine what if
hairstyles to try to imagine what if somebody am changed the color of the
somebody am changed the color of the
somebody am changed the color of the hair of this person or how he will look
hair of this person or how he will look
hair of this person or how he will look like is different clothing we can
like is different clothing we can
like is different clothing we can imagine how she looks like and as a 20
imagine how she looks like and as a 20
imagine how she looks like and as a 20 year old male as a 30 year old male
year old male as a 30 year old male
year old male as a 30 year old male several examples or even as a 65 year
several examples or even as a 65 year
several examples or even as a 65 year old male I would like to finish with
old male I would like to finish with
old male I would like to finish with saying that computer vision allows us to
saying that computer vision allows us to
saying that computer vision allows us to see ourselves in new lights there are
see ourselves in new lights there are
see ourselves in new lights there are many fun and important applications and
many fun and important applications and
many fun and important applications and I’m so excited about the impact that
I’m so excited about the impact that
I’m so excited about the impact that it’s having and will have in the future
it’s having and will have in the future
it’s having and will have in the future thank you
”
Be First to Comment