[Music] [Applause]
[Applause]
[Applause] all right so I’m Sonia I’m part of the
all right so I’m Sonia I’m part of the
all right so I’m Sonia I’m part of the artificial intelligent group and
artificial intelligent group and
artificial intelligent group and actually AI is really blooming these
actually AI is really blooming these
actually AI is really blooming these days right so everywhere I can basically
days right so everywhere I can basically
days right so everywhere I can basically answer any questions you’ve tried it and
answer any questions you’ve tried it and
answer any questions you’ve tried it and then there is robot I don’t ask any
then there is robot I don’t ask any
then there is robot I don’t ask any question they just do basically whatever
question they just do basically whatever
question they just do basically whatever you tell them and I know you know there
you tell them and I know you know there
you tell them and I know you know there is algorithms that did world champions
is algorithms that did world champions
is algorithms that did world champions in really complicated games but perhaps
in really complicated games but perhaps
in really complicated games but perhaps the most life-changing domain is sign
the most life-changing domain is sign
the most life-changing domain is sign yourself for autonomous driving class
yourself for autonomous driving class
yourself for autonomous driving class right just imagine a car that can you
right just imagine a car that can you
right just imagine a car that can you know drive you to school or work on its
know drive you to school or work on its
know drive you to school or work on its own you know how cool would that be
own you know how cool would that be
own you know how cool would that be right and maybe not so far out actually
right and maybe not so far out actually
right and maybe not so far out actually the movies from the eighties my actually
the movies from the eighties my actually
the movies from the eighties my actually true pretty soon all right so all this
true pretty soon all right so all this
true pretty soon all right so all this is amazed how much of this is actually
is amazed how much of this is actually
is amazed how much of this is actually powered by this really powerful concept
powered by this really powerful concept
powered by this really powerful concept neural networks which is a machine
neural networks which is a machine
neural networks which is a machine learning water and basically you give it
learning water and basically you give it
learning water and basically you give it some data and then it goes through
some data and then it goes through
some data and then it goes through layers and layers of computation until
layers and layers of computation until
layers and layers of computation until it basically does whatever you want it
it basically does whatever you want it
it basically does whatever you want it to do
to do
to do all right and this concept has been
all right and this concept has been
all right and this concept has been around for a decade but it has been 2012
around for a decade but it has been 2012
around for a decade but it has been 2012 that actually a group at U of T really
that actually a group at U of T really
that actually a group at U of T really made it a significant breakthrough this
made it a significant breakthrough this
made it a significant breakthrough this was a group of Geoff Hinton here and
was a group of Geoff Hinton here and
was a group of Geoff Hinton here and this really started to work it is much
this really started to work it is much
this really started to work it is much better than anything that has been
better than anything that has been
better than anything that has been around earth before actually why are
around earth before actually why are
around earth before actually why are just recently posted this top hundred
just recently posted this top hundred
just recently posted this top hundred people in the world have the most
people in the world have the most
people in the world have the most influence on the full digital digital
influence on the full digital digital
influence on the full digital digital world and actually Jeff made it on this
world and actually Jeff made it on this
world and actually Jeff made it on this list
list
list probably one of the few professors that
probably one of the few professors that
probably one of the few professors that made it in this list and what’s more
made it in this list and what’s more
made it in this list and what’s more impressive is actually only a few spots
impressive is actually only a few spots
impressive is actually only a few spots below Taylors
below Taylors
below Taylors I’d call that a success all right so
I’d call that a success all right so
I’d call that a success all right so let’s start with the beginning of the
let’s start with the beginning of the
let’s start with the beginning of the story actually my story so the kid on
story actually my story so the kid on
story actually my story so the kid on the left the cute kid on the left is me
the left the cute kid on the left is me
the left the cute kid on the left is me I’m on the right is my first cognitive
I’m on the right is my first cognitive
I’m on the right is my first cognitive agent so my sister actually
agent so my sister actually
agent so my sister actually so you’re basically looking at the
so you’re basically looking at the
so you’re basically looking at the moment I kind of decided on my life
moment I kind of decided on my life
moment I kind of decided on my life right it was AI so that’s what I wanted
right it was AI so that’s what I wanted
right it was AI so that’s what I wanted to do and you kind of see the
to do and you kind of see the
to do and you kind of see the expectation right so there I wanted me
expectation right so there I wanted me
expectation right so there I wanted me to perceive the world needs to talk to
to perceive the world needs to talk to
to perceive the world needs to talk to the person definitely do stuff for the
the person definitely do stuff for the
the person definitely do stuff for the person right and entertain the owner so
person right and entertain the owner so
person right and entertain the owner so you know there was the first day of my
you know there was the first day of my
you know there was the first day of my page dear you know how how to actually
page dear you know how how to actually
page dear you know how how to actually get there all right so now we have a
get there all right so now we have a
get there all right so now we have a robot in this scene or an apartment and
robot in this scene or an apartment and
robot in this scene or an apartment and I wanted to actually you know bring your
I wanted to actually you know bring your
I wanted to actually you know bring your coffee or something like that so at
coffee or something like that so at
coffee or something like that so at least it just understands to go to the
least it just understands to go to the
least it just understands to go to the kitchen it’s actually you know recognize
kitchen it’s actually you know recognize
kitchen it’s actually you know recognize a coffee pot right not to mention sugar
a coffee pot right not to mention sugar
a coffee pot right not to mention sugar a meal right so what I started with was
a meal right so what I started with was
a meal right so what I started with was the problem of recognition right which
the problem of recognition right which
the problem of recognition right which basically means that you have a photo
basically means that you have a photo
basically means that you have a photo and you want to recognize the object in
and you want to recognize the object in
and you want to recognize the object in from the photo but what makes this
from the photo but what makes this
from the photo but what makes this problem rich challenging is there’s so
problem rich challenging is there’s so
problem rich challenging is there’s so many different classes so many different
many different classes so many different
many different classes so many different types of objects in the world and each
types of objects in the world and each
types of objects in the world and each object can be basically in any location
object can be basically in any location
object can be basically in any location and any size and viewpoint and so many
and any size and viewpoint and so many
and any size and viewpoint and so many different poses or number or even
different poses or number or even
different poses or number or even abstraction so how can you make a model
abstraction so how can you make a model
abstraction so how can you make a model that can actually deal with this kind of
that can actually deal with this kind of
that can actually deal with this kind of you know complexity so but people
you know complexity so but people
you know complexity so but people believed in that those days including me
believed in that those days including me
believed in that those days including me is that there’s some compositionality to
is that there’s some compositionality to
is that there’s some compositionality to objects right subjects are composed of
objects right subjects are composed of
objects right subjects are composed of parts like a car is composed of wheels
parts like a car is composed of wheels
parts like a car is composed of wheels and wheels may be composed of other
and wheels may be composed of other
and wheels may be composed of other elements right so how can we translate
elements right so how can we translate
elements right so how can we translate that idea to vision okay so let’s take a
that idea to vision okay so let’s take a
that idea to vision okay so let’s take a very simple example right you were a kid
very simple example right you were a kid
very simple example right you were a kid you were probably playing with Lego
you were probably playing with Lego
you were probably playing with Lego right see it took really simple pieces
right see it took really simple pieces
right see it took really simple pieces and you compose them a little bit more
and you compose them a little bit more
and you compose them a little bit more complicated
complicated
complicated or shapes doing that until in the end
or shapes doing that until in the end
or shapes doing that until in the end you had maybe a full sculpture a full
you had maybe a full sculpture a full
you had maybe a full sculpture a full object right so how can we translate
object right so how can we translate
object right so how can we translate that to vision okay so does something
that to vision okay so does something
that to vision okay so does something like this it can start a really small
like this it can start a really small
like this it can start a really small tiny patches like little pieces of
tiny patches like little pieces of
tiny patches like little pieces of curves and that keep combining them
curves and that keep combining them
curves and that keep combining them until in the end the top of this
until in the end the top of this
until in the end the top of this hierarchy you have something that looks
hierarchy you have something that looks
hierarchy you have something that looks like an object right and this model
like an object right and this model
like an object right and this model which kind of preceded this big success
which kind of preceded this big success
which kind of preceded this big success of neural networks actually had a
of neural networks actually had a
of neural networks actually had a structure a special structure of a
structure a special structure of a
structure a special structure of a neural net all right but by the time it
neural net all right but by the time it
neural net all right but by the time it was actually one of the deepest model
was actually one of the deepest model
was actually one of the deepest model existed it worked pretty well so I was
existed it worked pretty well so I was
existed it worked pretty well so I was super proud of myself but I’m Jeff came
super proud of myself but I’m Jeff came
super proud of myself but I’m Jeff came in kind of ruined the dream basically in
in kind of ruined the dream basically in
in kind of ruined the dream basically in the bottom you see the original image
the bottom you see the original image
the bottom you see the original image and then you know this color show you
and then you know this color show you
and then you know this color show you what the model which kind of sits in the
what the model which kind of sits in the
what the model which kind of sits in the top there is looking at when it’s
top there is looking at when it’s
top there is looking at when it’s recognizing objects so it was doing
recognizing objects so it was doing
recognizing objects so it was doing pretty well I’m just going to show you a
pretty well I’m just going to show you a
pretty well I’m just going to show you a little video can a robot write a
little video can a robot write a
little video can a robot write a symphony can a robot turn a canvas into
symphony can a robot turn a canvas into
symphony can a robot turn a canvas into a beautiful masterpiece can you all
a beautiful masterpiece can you all
a beautiful masterpiece can you all right so this basically brings us to the
right so this basically brings us to the
right so this basically brings us to the other issue right so now we have a robot
other issue right so now we have a robot
other issue right so now we have a robot in this thing that cannot recognize the
in this thing that cannot recognize the
in this thing that cannot recognize the world but the scene is actually not
world but the scene is actually not
world but the scene is actually not empty right there’s people in this world
empty right there’s people in this world
empty right there’s people in this world and this people are gonna want to talk
and this people are gonna want to talk
and this people are gonna want to talk to the robot right so in this case is
to the robot right so in this case is
to the robot right so in this case is grandma saying something and the robot
grandma saying something and the robot
grandma saying something and the robot needs to understand language needs to
needs to understand language needs to
needs to understand language needs to understand what the grandma is actually
understand what the grandma is actually
understand what the grandma is actually referring to this this little bottle in
referring to this this little bottle in
referring to this this little bottle in this case right I know it’s very sad the
this case right I know it’s very sad the
this case right I know it’s very sad the robot needs to also be able to talk talk
robot needs to also be able to talk talk
robot needs to also be able to talk talk back right it’s so engaging the
back right it’s so engaging the
back right it’s so engaging the conversation suddens to be able to
conversation suddens to be able to
conversation suddens to be able to describe the world and I’m gonna tell
describe the world and I’m gonna tell
describe the world and I’m gonna tell you a little bit about what we did on
you a little bit about what we did on
you a little bit about what we did on this description party so this is
this description party so this is
this description party so this is basically what the current systems do so
basically what the current systems do so
basically what the current systems do so they take a picture and then they’re
they take a picture and then they’re
they take a picture and then they’re able to write a little sentence that
able to write a little sentence that
able to write a little sentence that describes this picture my robo take the
describes this picture my robo take the
describes this picture my robo take the picture and basically generates word by
picture and basically generates word by
picture and basically generates word by word describing what’s on the picture
word describing what’s on the picture
word describing what’s on the picture right so this would be one example of
right so this would be one example of
right so this would be one example of I’m holding a metal ball in the table
I’m holding a metal ball in the table
I’m holding a metal ball in the table all right but if you look at this
all right but if you look at this
all right but if you look at this picture it’s so much richer than while
picture it’s so much richer than while
picture it’s so much richer than while it caption is actually saying right
it caption is actually saying right
it caption is actually saying right there’s a party maybe the gondola
there’s a party maybe the gondola
there’s a party maybe the gondola graduated the gondola ride is worried
graduated the gondola ride is worried
graduated the gondola ride is worried that there is no more food writers why
that there is no more food writers why
that there is no more food writers why stop so much more alive and this reason
stop so much more alive and this reason
stop so much more alive and this reason caption over here right so maybe I can
caption over here right so maybe I can
caption over here right so maybe I can we actually want a robot you know to do
we actually want a robot you know to do
we actually want a robot you know to do more maybe they want to understand what
more maybe they want to understand what
more maybe they want to understand what these people are actually feeling and
these people are actually feeling and
these people are actually feeling and you know why they’re feeling you
you know why they’re feeling you
you know why they’re feeling you particularly why they’re gonna be
particularly why they’re gonna be
particularly why they’re gonna be thinking and basically what they will do
thinking and basically what they will do
thinking and basically what they will do next or want to do next so maybe we want
next or want to do next so maybe we want
next or want to do next so maybe we want our robots to actually write a much more
our robots to actually write a much more
our robots to actually write a much more descriptive text about this picture
descriptive text about this picture
descriptive text about this picture right maybe write a little story and
right maybe write a little story and
right maybe write a little story and it’s gonna describe it right so if we
it’s gonna describe it right so if we
it’s gonna describe it right so if we actually had a date I forgot the
actually had a date I forgot the
actually had a date I forgot the pictures in this story then we would be
pictures in this story then we would be
pictures in this story then we would be able to train on your lines gonna be
able to train on your lines gonna be
able to train on your lines gonna be able to produce this kind of data and
able to produce this kind of data and
able to produce this kind of data and this kind of stories right so the
this kind of stories right so the
this kind of stories right so the question is we have pictures but how can
question is we have pictures but how can
question is we have pictures but how can we get such really rich descriptions
we get such really rich descriptions
we get such really rich descriptions okay books all right so books are really
okay books all right so books are really
okay books all right so books are really rich in kind of bringing the characters
rich in kind of bringing the characters
rich in kind of bringing the characters to life for you right you really imagine
to life for you right you really imagine
to life for you right you really imagine how they look like what they’re thinking
how they look like what they’re thinking
how they look like what they’re thinking and feeling basically any like
and feeling basically any like
and feeling basically any like throughout the story right I’m gonna
throughout the story right I’m gonna
throughout the story right I’m gonna show you as I walk towards the bar
show you as I walk towards the bar
show you as I walk towards the bar across the concrete parking lot I looked
across the concrete parking lot I looked
across the concrete parking lot I looked straight down the road and saw the river
straight down the road and saw the river
straight down the road and saw the river moving a pace with the river was a long
moving a pace with the river was a long
moving a pace with the river was a long single line of men eyes aimed at their
single line of men eyes aimed at their
single line of men eyes aimed at their feet walking steadfastly nowhere I felt
feet walking steadfastly nowhere I felt
feet walking steadfastly nowhere I felt an immediate need to get inside so
an immediate need to get inside so
an immediate need to get inside so basically there’s this three sentences
basically there’s this three sentences
basically there’s this three sentences from from this book are kind of really
from from this book are kind of really
from from this book are kind of really telling you a lot right so basically the
telling you a lot right so basically the
telling you a lot right so basically the author is in the parking lot and it’s
author is in the parking lot and it’s
author is in the parking lot and it’s this man walking across the street it
this man walking across the street it
this man walking across the street it makes me feel very uneasy okay but and
makes me feel very uneasy okay but and
makes me feel very uneasy okay but and this book on burner maybe some of your
this book on burner maybe some of your
this book on burner maybe some of your read ready
read ready
read ready actually it was converted into a movie
actually it was converted into a movie
actually it was converted into a movie right and then you can see the same
right and then you can see the same
right and then you can see the same scene basically seen from the eyes of a
scene basically seen from the eyes of a
scene basically seen from the eyes of a director so our idea was can we take
director so our idea was can we take
director so our idea was can we take move in a book basically a movie that
move in a book basically a movie that
move in a book basically a movie that was reported after a book and try to
was reported after a book and try to
was reported after a book and try to align them so basically take a clip so
align them so basically take a clip so
align them so basically take a clip so each clip from the book from the movie
each clip from the book from the movie
each clip from the book from the movie and find where it actually happens in
and find where it actually happens in
and find where it actually happens in the book okay and we can find that text
the book okay and we can find that text
the book okay and we can find that text you can just copy it over to our clip
you can just copy it over to our clip
you can just copy it over to our clip and then we have the data that we were
and then we have the data that we were
and then we have the data that we were looking for all right
looking for all right
looking for all right but this far is actually super
but this far is actually super
but this far is actually super challenging alright so movie contains
challenging alright so movie contains
challenging alright so movie contains two hundred thousand pictures okay and
two hundred thousand pictures okay and
two hundred thousand pictures okay and the book has almost a say one hundred
the book has almost a say one hundred
the book has almost a say one hundred and fifty thousand words it’s really
and fifty thousand words it’s really
and fifty thousand words it’s really challenging and furthermore there’s so
challenging and furthermore there’s so
challenging and furthermore there’s so much diversity in books right so here is
much diversity in books right so here is
much diversity in books right so here is the longest sentence this is from
the longest sentence this is from
the longest sentence this is from American Psycho one sentence four
American Psycho one sentence four
American Psycho one sentence four hundred words all right and this is the
hundred words all right and this is the
hundred words all right and this is the shortest one there’s so much variety
shortest one there’s so much variety
shortest one there’s so much variety right and actually the student or the
right and actually the student or the
right and actually the student or the student told me it’s not possible so
student told me it’s not possible so
student told me it’s not possible so spend Friday night annotating one movie
spend Friday night annotating one movie
spend Friday night annotating one movie in a book and this is what you can come
in a book and this is what you can come
in a book and this is what you can come up with so I just see on the bottom is
up with so I just see on the bottom is
up with so I just see on the bottom is basically a movie in minutes from left
basically a movie in minutes from left
basically a movie in minutes from left to right laughs is the beginning the
to right laughs is the beginning the
to right laughs is the beginning the right is then and on the top there are
right is then and on the top there are
right is then and on the top there are red bubbles are sentences in the book
red bubbles are sentences in the book
red bubbles are sentences in the book again from left in the beginning right
again from left in the beginning right
again from left in the beginning right is the end and the edges that you see is
is the end and the edges that you see is
is the end and the edges that you see is basically correspondences that means
basically correspondences that means
basically correspondences that means that the particular chunk in the movie
that the particular chunk in the movie
that the particular chunk in the movie is corresponding to the sentence in the
is corresponding to the sentence in the
is corresponding to the sentence in the book so you can actually do it it took
book so you can actually do it it took
book so you can actually do it it took me ten hours for a two-hour movie that’s
me ten hours for a two-hour movie that’s
me ten hours for a two-hour movie that’s not too bad and then we actually hired
not too bad and then we actually hired
not too bad and then we actually hired annotator basically we’re just watching
annotator basically we’re just watching
annotator basically we’re just watching movies and reading books for one month
movies and reading books for one month
movies and reading books for one month and she she even got paid for it so I
and she she even got paid for it so I
and she she even got paid for it so I think wasn’t too bad
think wasn’t too bad
think wasn’t too bad alright so now you can take this data
alright so now you can take this data
alright so now you can take this data and what you can do is take the movie
and what you can do is take the movie
and what you can do is take the movie clip it in
clip it in
clip it in short clips and then I wanna do some
short clips and then I wanna do some
short clips and then I wanna do some matches I’m gonna compute a similarity
matches I’m gonna compute a similarity
matches I’m gonna compute a similarity between each clip and a particular
between each clip and a particular
between each clip and a particular sentence in the book and you can use a
sentence in the book and you can use a
sentence in the book and you can use a neural net to do this for you okay so
neural net to do this for you okay so
neural net to do this for you okay so hopefully if sound works you’re gonna
hopefully if sound works you’re gonna
hopefully if sound works you’re gonna see a clip from Harry Potter and what
see a clip from Harry Potter and what
see a clip from Harry Potter and what you’re gonna hear is not the movie
you’re gonna hear is not the movie
you’re gonna hear is not the movie you’re gonna hear the narration of the
you’re gonna hear the narration of the
you’re gonna hear the narration of the part of the book this clip was matched
part of the book this clip was matched
part of the book this clip was matched to Harry please relax or Madame Pomfrey
to Harry please relax or Madame Pomfrey
to Harry please relax or Madame Pomfrey will have me thrown out
will have me thrown out
will have me thrown out Harry swallowed and looked around him he
Harry swallowed and looked around him he
Harry swallowed and looked around him he realized he must be in the hospital wing
realized he must be in the hospital wing
realized he must be in the hospital wing he was lying in a bed with white linen
he was lying in a bed with white linen
he was lying in a bed with white linen sheets and next to him was a table piled
sheets and next to him was a table piled
sheets and next to him was a table piled high with what looked like half the
high with what looked like half the
high with what looked like half the candy shop then you can take a movie
candy shop then you can take a movie
candy shop then you can take a movie that doesn’t have a book and just match
that doesn’t have a book and just match
that doesn’t have a book and just match it to you know like all sorts of books
it to you know like all sorts of books
it to you know like all sorts of books and try to find some Paraguayans when I
and try to find some Paraguayans when I
and try to find some Paraguayans when I kind of best match to this clip right so
kind of best match to this clip right so
kind of best match to this clip right so with this for Batman if you got matched
with this for Batman if you got matched
with this for Batman if you got matched to some facts and fiction cube I believe
to some facts and fiction cube I believe
to some facts and fiction cube I believe you will enjoy your time here I am NOT a
you will enjoy your time here I am NOT a
you will enjoy your time here I am NOT a harsh master but I am strict when we are
harsh master but I am strict when we are
harsh master but I am strict when we are with others I expect you to present
with others I expect you to present
with others I expect you to present yourself properly what we do here in
yourself properly what we do here in
yourself properly what we do here in your room and in the dungeon is between
your room and in the dungeon is between
your room and in the dungeon is between you and I it is a testament to the trust
you and I it is a testament to the trust
you and I it is a testament to the trust and respect we have for each other and
and respect we have for each other and
and respect we have for each other and no one else needs to know about our
no one else needs to know about our
no one else needs to know about our arrangement
arrangement
arrangement I’m sure the past few days have been
I’m sure the past few days have been
I’m sure the past few days have been overwhelming thus far but I have tried
overwhelming thus far but I have tried
overwhelming thus far but I have tried to give you as much information as
to give you as much information as
to give you as much information as possible do you have any questions I
possible do you have any questions I
possible do you have any questions I need an umbrella and I shake Spock this
need an umbrella and I shake Spock this
need an umbrella and I shake Spock this robot actually be able to answer these
robot actually be able to answer these
robot actually be able to answer these questions right so we also this is
questions right so we also this is
questions right so we also this is something like this so basically this is
something like this so basically this is
something like this so basically this is where the current methods are doing so
where the current methods are doing so
where the current methods are doing so they are taking a picture and they’re
they are taking a picture and they’re
they are taking a picture and they’re putting a little quiz to the robot so
putting a little quiz to the robot so
putting a little quiz to the robot so the robot needs to look at the picture
the robot needs to look at the picture
the robot needs to look at the picture and the question and then be able to
and the question and then be able to
and the question and then be able to answer questions to it
answer questions to it
answer questions to it if you are able to ask to answer any
if you are able to ask to answer any
if you are able to ask to answer any questions that someone can ask you know
questions that someone can ask you know
questions that someone can ask you know about the picture this really means that
about the picture this really means that
about the picture this really means that the robust understanding was there all
the robust understanding was there all
the robust understanding was there all right but there is just so many things
right but there is just so many things
right but there is just so many things you can ask about a single photo right
you can ask about a single photo right
you can ask about a single photo right what is there maybe with the color so
what is there maybe with the color so
what is there maybe with the color so it’s kind of boring right but the
it’s kind of boring right but the
it’s kind of boring right but the picture right there’s like the world is
picture right there’s like the world is
picture right there’s like the world is not started there’s actually a story
not started there’s actually a story
not started there’s actually a story with her in every everything
all right so this is like 20 second flip
all right so this is like 20 second flip actually you know there’s so many more
actually you know there’s so many more
actually you know there’s so many more rich questions that you can ask right so
rich questions that you can ask right so
rich questions that you can ask right so what’s the relationship between
what’s the relationship between
what’s the relationship between character is what it is character lie to
character is what it is character lie to
character is what it is character lie to the other what’s gonna happen is what’s
the other what’s gonna happen is what’s
the other what’s gonna happen is what’s there so there’s so much more you can
there so there’s so much more you can
there so there’s so much more you can ask so idea was not just take a single
ask so idea was not just take a single
ask so idea was not just take a single photo in a quiz take two hundred
photo in a quiz take two hundred
photo in a quiz take two hundred thousand photos so full movie and give
thousand photos so full movie and give
thousand photos so full movie and give them the robotic quiz and now the robot
them the robotic quiz and now the robot
them the robotic quiz and now the robot needs to read this quiz and learn
needs to read this quiz and learn
needs to read this quiz and learn basically what’s going on so understand
basically what’s going on so understand
basically what’s going on so understand the story
the story
the story alright so this is basically our robots
alright so this is basically our robots
alright so this is basically our robots so they’re just chilling out and
so they’re just chilling out and
so they’re just chilling out and watching TV watching videos and
watching TV watching videos and
watching TV watching videos and basically learning how the world and
basically learning how the world and
basically learning how the world and people work right basically what how and
people work right basically what how and
people work right basically what how and why things are happening right don’t
why things are happening right don’t
why things are happening right don’t worry it’s actually not working very
worry it’s actually not working very
worry it’s actually not working very well yet it’s a really challenging
well yet it’s a really challenging
well yet it’s a really challenging problem twenty percent is random the
problem twenty percent is random the
problem twenty percent is random the best we can do is forty percent but I
best we can do is forty percent but I
best we can do is forty percent but I think is gonna work you know soon enough
think is gonna work you know soon enough
think is gonna work you know soon enough all right so what that is every need
all right so what that is every need
all right so what that is every need right so now they can talk to each other
right so now they can talk to each other
right so now they can talk to each other I can ask questions but this brings us
I can ask questions but this brings us
I can ask questions but this brings us to the basic thing why did the grandma
to the basic thing why did the grandma
to the basic thing why did the grandma actually got the robot right to
actually got the robot right to
actually got the robot right to basically do stuff for her right maybe
basically do stuff for her right maybe
basically do stuff for her right maybe she wants to have the robot make tea or
she wants to have the robot make tea or
she wants to have the robot make tea or coffee and so on right so basically the
coffee and so on right so basically the
coffee and so on right so basically the robot needs to take action not just
robot needs to take action not just
robot needs to take action not just pursue the world right now imagine the
pursue the world right now imagine the
pursue the world right now imagine the scenario so how does grandma and the
scenario so how does grandma and the
scenario so how does grandma and the answers make me coffee right now there
answers make me coffee right now there
answers make me coffee right now there is all this robot or any other that have
is all this robot or any other that have
is all this robot or any other that have been pre-programmed to know how to do
been pre-programmed to know how to do
been pre-programmed to know how to do coffee right all the stuff that are
coffee right all the stuff that are
coffee right all the stuff that are there right but maybe say the grandma
there right but maybe say the grandma
there right but maybe say the grandma actually bought the robot things have
actually bought the robot things have
actually bought the robot things have changed right the world is changing
changed right the world is changing
changed right the world is changing maybe she wants to do other stuff right
maybe she wants to do other stuff right
maybe she wants to do other stuff right more interesting stuff
more interesting stuff
more interesting stuff and they’re like crazy things happen
and they’re like crazy things happen
and they’re like crazy things happen every day
so how can we actually have this robot
so how can we actually have this robot or that and actually be able to follow
or that and actually be able to follow
or that and actually be able to follow anything that we were gonna be asking it
anything that we were gonna be asking it
anything that we were gonna be asking it alright so here was our idea so let’s
alright so here was our idea so let’s
alright so here was our idea so let’s say we actually as the robot to do
say we actually as the robot to do
say we actually as the robot to do something that it has never heard how to
something that it has never heard how to
something that it has never heard how to do before right so how would you do it
do before right so how would you do it
do before right so how would you do it as a human you would describe the steps
as a human you would describe the steps
as a human you would describe the steps how to do it you will describe I know
how to do it you will describe I know
how to do it you will describe I know the recipe you know do this and this and
the recipe you know do this and this and
the recipe you know do this and this and this in natural language so the idea was
this in natural language so the idea was
this in natural language so the idea was in the robot read this instruction just
in the robot read this instruction just
in the robot read this instruction just a natural language and self program
a natural language and self program
a natural language and self program itself basically write code that is
itself basically write code that is
itself basically write code that is going to be able to execute and actually
going to be able to execute and actually
going to be able to execute and actually learn how to do this task right so what
learn how to do this task right so what
learn how to do this task right so what you see on the left is just kind of a
you see on the left is just kind of a
you see on the left is just kind of a set of really simple instructions a
set of really simple instructions a
set of really simple instructions a sequence of instructions that the robot
sequence of instructions that the robot
sequence of instructions that the robot knows how to execute but actually if
knows how to execute but actually if
knows how to execute but actually if it’s executing it then it basically does
it’s executing it then it basically does
it’s executing it then it basically does what the grandma wants right so then in
what the grandma wants right so then in
what the grandma wants right so then in the end you can have an environment and
the end you can have an environment and
the end you can have an environment and then it knows how to do that stuff right
then it knows how to do that stuff right
then it knows how to do that stuff right luckily no sound so here is a
luckily no sound so here is a
luckily no sound so here is a description the robot read this
description the robot read this
description the robot read this description basically does its own code
description basically does its own code
description basically does its own code in a material gonna see is this agent
in a material gonna see is this agent
in a material gonna see is this agent performing this action this is actually
performing this action this is actually
performing this action this is actually done in a game engine called unity and
done in a game engine called unity and
done in a game engine called unity and it was done in collaboration but the
it was done in collaboration but the
it was done in collaboration but the main driving force was actually a view
main driving force was actually a view
main driving force was actually a view of the undergrad so she came to my
of the undergrad so she came to my
of the undergrad so she came to my office he said I know computer vision
office he said I know computer vision
office he said I know computer vision when I have been programming games and
when I have been programming games and
when I have been programming games and was like all right you’re hired
was like all right you’re hired
was like all right you’re hired we’re gonna be doing this cool project
it’s a simulation is really the thing to
it’s a simulation is really the thing to do these days because if you have a
do these days because if you have a
do these days because if you have a robot do stuff in a real environment you
robot do stuff in a real environment you
robot do stuff in a real environment you can just you know go really wrong so
can just you know go really wrong so
can just you know go really wrong so first needs to the stuff in the
first needs to the stuff in the
first needs to the stuff in the simulated environment alright so now we
simulated environment alright so now we
simulated environment alright so now we can kind of close you know bit knows how
can kind of close you know bit knows how
can kind of close you know bit knows how to perceive understand language even do
to perceive understand language even do
to perceive understand language even do stuff right but what else can we do
stuff right but what else can we do
stuff right but what else can we do right and since this grammar is actually
right and since this grammar is actually
right and since this grammar is actually kind of tech savvy right maybe we can
kind of tech savvy right maybe we can
kind of tech savvy right maybe we can throw it I
throw it I
throw it I a little phone and do all these nice
a little phone and do all these nice
a little phone and do all these nice things for her as well right so maybe
things for her as well right so maybe
things for her as well right so maybe she’s interested in fashion or other
she’s interested in fashion or other
she’s interested in fashion or other possible applications so our idea was
possible applications so our idea was
possible applications so our idea was can you take a photo and maybe the user
can you take a photo and maybe the user
can you take a photo and maybe the user is going to tell you the settings right
is going to tell you the settings right
is going to tell you the settings right go into the beach or something like that
go into the beach or something like that
go into the beach or something like that and you want to predict how fashionable
and you want to predict how fashionable
and you want to predict how fashionable this this user actually looks for the
this this user actually looks for the
this this user actually looks for the occasion okay so what we did was take
occasion okay so what we did was take
occasion okay so what we did was take this website for chief topia we’re
this website for chief topia we’re
this website for chief topia we’re basically a lot of lifestyle enthusiast
basically a lot of lifestyle enthusiast
basically a lot of lifestyle enthusiast suppose their latest outfit and then
suppose their latest outfit and then
suppose their latest outfit and then other people can vote on them so
other people can vote on them so
other people can vote on them so basically tell you how well you actually
basically tell you how well you actually
basically tell you how well you actually look and that was a proxy so basically
look and that was a proxy so basically
look and that was a proxy so basically we took the number of votes and this was
we took the number of votes and this was
we took the number of votes and this was our proxy as fashionability this is the
our proxy as fashionability this is the
our proxy as fashionability this is the stuff that you want to predict when
stuff that you want to predict when
stuff that you want to predict when someone shows you the photo so this is
someone shows you the photo so this is
someone shows you the photo so this is basically our model thinking that these
basically our model thinking that these
basically our model thinking that these people are fashionable so this is a
people are fashionable so this is a
people are fashionable so this is a prediction and the nice thing is that it
prediction and the nice thing is that it
prediction and the nice thing is that it as a side effect also discover styles
as a side effect also discover styles
as a side effect also discover styles right so imagine you give it a photo and
right so imagine you give it a photo and
right so imagine you give it a photo and then basically it’s able to show you the
then basically it’s able to show you the
then basically it’s able to show you the pictures of all the other people who
pictures of all the other people who
pictures of all the other people who have the same style maybe that gives you
have the same style maybe that gives you
have the same style maybe that gives you some idea of what you want to wear next
some idea of what you want to wear next
some idea of what you want to wear next right it can also do recommendations so
right it can also do recommendations so
right it can also do recommendations so let’s say that you know this photo
let’s say that you know this photo
let’s say that you know this photo doesn’t maybe look so good and it’s
doesn’t maybe look so good and it’s
doesn’t maybe look so good and it’s gonna tell you you know maybe one of our
gonna tell you you know maybe one of our
gonna tell you you know maybe one of our something else to increase how well you
something else to increase how well you
something else to increase how well you look for that occasion maybe that gives
look for that occasion maybe that gives
look for that occasion maybe that gives you some idea and maybe you just want to
you some idea and maybe you just want to
you some idea and maybe you just want to describe in a little bit more detail of
describe in a little bit more detail of
describe in a little bit more detail of what you want to kind of see on yourself
what you want to kind of see on yourself
what you want to kind of see on yourself and then you can actually generate that
and then you can actually generate that
and then you can actually generate that kind of clothing on the user maybe the
kind of clothing on the user maybe the
kind of clothing on the user maybe the user wants to do say something else a
user wants to do say something else a
user wants to do say something else a blue shirt and then you can generate
blue shirt and then you can generate
blue shirt and then you can generate that kind of clothing right so in a
that kind of clothing right so in a
that kind of clothing right so in a couple of minutes if I’m actually see
couple of minutes if I’m actually see
couple of minutes if I’m actually see like so many outfits on yourself I kind
like so many outfits on yourself I kind
like so many outfits on yourself I kind of see what goes well on you and maybe
of see what goes well on you and maybe
of see what goes well on you and maybe just maybe that’s the only way to get a
just maybe that’s the only way to get a
just maybe that’s the only way to get a nerd like me in cosmopolitan
nerd like me in cosmopolitan
nerd like me in cosmopolitan it was the first moment and my mom was
it was the first moment and my mom was
it was the first moment and my mom was actually proud all right so this project
actually proud all right so this project
actually proud all right so this project was actually thanks to Edgar should I
was actually thanks to Edgar should I
was actually thanks to Edgar should I say Edgar before the project
all right so I hope in this 15 maybe a
all right so I hope in this 15 maybe a little bit more minutes I have convinced
little bit more minutes I have convinced
little bit more minutes I have convinced you that AI is the thing to do and AI is
you that AI is the thing to do and AI is
you that AI is the thing to do and AI is a new cool so you should all be working
a new cool so you should all be working
a new cool so you should all be working on it
on it
on it [Applause]
[Applause]
[Applause] [Music]
”
Be First to Comment