[Music] when it was a boy I wanted to maximize
when it was a boy I wanted to maximize
when it was a boy I wanted to maximize my impact on the world and I was smart
my impact on the world and I was smart
my impact on the world and I was smart enough to realize that I am NOT very
enough to realize that I am NOT very
enough to realize that I am NOT very smart and that I have to build a machine
smart and that I have to build a machine
smart and that I have to build a machine that learns to become much smarter than
that learns to become much smarter than
that learns to become much smarter than myself such that it can solve all the
myself such that it can solve all the
myself such that it can solve all the problems that I cannot solve myself and
problems that I cannot solve myself and
problems that I cannot solve myself and I can retire and my first publication on
I can retire and my first publication on
I can retire and my first publication on that dates back 30 years 1987 my diploma
that dates back 30 years 1987 my diploma
that dates back 30 years 1987 my diploma thesis where I already try to solve the
thesis where I already try to solve the
thesis where I already try to solve the grand problem of AI not only build a
grand problem of AI not only build a
grand problem of AI not only build a machine that learns a little bit here
machine that learns a little bit here
machine that learns a little bit here and learns a little bit there but also
and learns a little bit there but also
and learns a little bit there but also learns to improve the learning algorithm
learns to improve the learning algorithm
learns to improve the learning algorithm itself and the way it learns the way it
itself and the way it learns the way it
itself and the way it learns the way it learns and so on recursively without any
learns and so on recursively without any
learns and so on recursively without any limits except the limits of logics and
limits except the limits of logics and
limits except the limits of logics and physics and I’m still working on the
physics and I’m still working on the
physics and I’m still working on the same old thing and I’m still pretty much
same old thing and I’m still pretty much
same old thing and I’m still pretty much saying the same thing except that now
saying the same thing except that now
saying the same thing except that now more people are listening because the
more people are listening because the
more people are listening because the learning algorithms that we have
learning algorithms that we have
learning algorithms that we have developed on the way to this goal they
developed on the way to this goal they
developed on the way to this goal they are now on three thousand million
are now on three thousand million
are now on three thousand million smartphones and all of you have them in
smartphones and all of you have them in
smartphones and all of you have them in your pockets what you see here are the
your pockets what you see here are the
your pockets what you see here are the five most valuable companies of the
five most valuable companies of the
five most valuable companies of the Western world Apple Google Facebook
Western world Apple Google Facebook
Western world Apple Google Facebook Microsoft and Amazon and all of them are
Microsoft and Amazon and all of them are
Microsoft and Amazon and all of them are emphasizing that AI artificial
emphasizing that AI artificial
emphasizing that AI artificial intelligence is central to what they are
intelligence is central to what they are
intelligence is central to what they are doing and all of them are using heavily
doing and all of them are using heavily
doing and all of them are using heavily the deep learning methods that my team
the deep learning methods that my team
the deep learning methods that my team has developed since the early nineties
has developed since the early nineties
has developed since the early nineties and yone
and yone
and yone in Munich and in Switzerland especially
in Munich and in Switzerland especially
in Munich and in Switzerland especially something which is called so long short
something which is called so long short
something which is called so long short term memory has anybody in this room
term memory has anybody in this room
term memory has anybody in this room ever heard of the long short-term memory
ever heard of the long short-term memory
ever heard of the long short-term memory or the LST M hands up anybody ever heard
or the LST M hands up anybody ever heard
or the LST M hands up anybody ever heard of that okay
of that okay
of that okay has anybody never heard of the STM I see
has anybody never heard of the STM I see
has anybody never heard of the STM I see we have a third group in this room
the lsdm is a little bit like your brain
the lsdm is a little bit like your brain it’s an artificial neural network which
it’s an artificial neural network which
it’s an artificial neural network which also has neurons and in your brain
also has neurons and in your brain
also has neurons and in your brain you’ve got about 100 billion neurons and
you’ve got about 100 billion neurons and
you’ve got about 100 billion neurons and each of them is connected to roughly
each of them is connected to roughly
each of them is connected to roughly 10,000 other neurons on average which
10,000 other neurons on average which
10,000 other neurons on average which means that you have got a million
means that you have got a million
means that you have got a million billion connections and each of these
billion connections and each of these
billion connections and each of these connections has a strength which says
connections has a strength which says
connections has a strength which says how much does this neuron over here
how much does this neuron over here
how much does this neuron over here influence that neuron over there at the
influence that neuron over there at the
influence that neuron over there at the next time step and in the beginning all
next time step and in the beginning all
next time step and in the beginning all these connections are random and the
these connections are random and the
these connections are random and the system knows nothing within through a
system knows nothing within through a
system knows nothing within through a smart learning algorithm it learns from
smart learning algorithm it learns from
smart learning algorithm it learns from lots of examples to translate the
lots of examples to translate the
lots of examples to translate the incoming data such as video through the
incoming data such as video through the
incoming data such as video through the cameras or audio through the microphones
cameras or audio through the microphones
cameras or audio through the microphones or pain signals through the pain sensors
or pain signals through the pain sensors
or pain signals through the pain sensors it learns to translate that into output
it learns to translate that into output
it learns to translate that into output actions because some of these neurons
actions because some of these neurons
actions because some of these neurons are output neurons that control speech
are output neurons that control speech
are output neurons that control speech muscles and finger muscles and only
muscles and finger muscles and only
muscles and finger muscles and only through experience it can learn to solve
through experience it can learn to solve
through experience it can learn to solve all kinds of interesting problems such
all kinds of interesting problems such
all kinds of interesting problems such as driving a car or do the speech
as driving a car or do the speech
as driving a car or do the speech recognition on your smartphone because
recognition on your smartphone because
recognition on your smartphone because whenever you take out your smartphone an
whenever you take out your smartphone an
whenever you take out your smartphone an Android phone for example and you speak
Android phone for example and you speak
Android phone for example and you speak to it and you say ok Google show me is
to it and you say ok Google show me is
to it and you say ok Google show me is the shortest way to Milano then it
the shortest way to Milano then it
the shortest way to Milano then it understands your speech because there is
understands your speech because there is
understands your speech because there is an lsdm in there which has learned to
an lsdm in there which has learned to
an lsdm in there which has learned to understand speech every 10 milliseconds
understand speech every 10 milliseconds
understand speech every 10 milliseconds 100 times a second new inputs are coming
100 times a second new inputs are coming
100 times a second new inputs are coming from the microphone and then translates
from the microphone and then translates
from the microphone and then translates it after thinking into letters which is
it after thinking into letters which is
it after thinking into letters which is then question to the search engine and
then question to the search engine and
then question to the search engine and it has long to do that by listening to
it has long to do that by listening to
it has long to do that by listening to lots of speech from women from me
lots of speech from women from me
lots of speech from women from me all kinds of people and that’s how since
all kinds of people and that’s how since
all kinds of people and that’s how since 2015 Google speech recognition is now
2015 Google speech recognition is now
2015 Google speech recognition is now much better than it used to be the basic
much better than it used to be the basic
much better than it used to be the basic lsdm cell looks like that I don’t have
lsdm cell looks like that I don’t have
lsdm cell looks like that I don’t have the time to explain that but at least I
the time to explain that but at least I
the time to explain that but at least I can list the names of the brilliant
can list the names of the brilliant
can list the names of the brilliant students in my lab who made that
students in my lab who made that
students in my lab who made that possible and what are the big companies
possible and what are the big companies
possible and what are the big companies doing with that
doing with that
doing with that well speech recognition is only one
well speech recognition is only one
well speech recognition is only one example if you are on Facebook is
example if you are on Facebook is
example if you are on Facebook is anybody on Facebook okay I use sometimes
anybody on Facebook okay I use sometimes
anybody on Facebook okay I use sometimes clicking at the translate button because
clicking at the translate button because
clicking at the translate button because somebody sent you something in a foreign
somebody sent you something in a foreign
somebody sent you something in a foreign language and then you can translate it
language and then you can translate it
language and then you can translate it is anybody doing that yeah whatever you
is anybody doing that yeah whatever you
is anybody doing that yeah whatever you do that you are waking up again a long
do that you are waking up again a long
do that you are waking up again a long short term memory and lsdm which has
short term memory and lsdm which has
short term memory and lsdm which has learned to translate text in one
learned to translate text in one
learned to translate text in one language into translated text and
language into translated text and
language into translated text and Facebook is doing that four billion
Facebook is doing that four billion
Facebook is doing that four billion times a day so every 50 every second
times a day so every 50 every second
times a day so every 50 every second 50,000 sentences are being translated by
50,000 sentences are being translated by
50,000 sentences are being translated by an LST am working for Facebook and
an LST am working for Facebook and
an LST am working for Facebook and another 50,000 in the second and another
another 50,000 in the second and another
another 50,000 in the second and another 50,000 and to see how much this thing is
50,000 and to see how much this thing is
50,000 and to see how much this thing is now permitting the modern world just
now permitting the modern world just
now permitting the modern world just note that almost 30 percent of the
note that almost 30 percent of the
note that almost 30 percent of the awesome computational power for
awesome computational power for
awesome computational power for interference and all these Google Data
interference and all these Google Data
interference and all these Google Data Centers all these data centers of Google
Centers all these data centers of Google
Centers all these data centers of Google are all over the world is used for LST
are all over the world is used for LST
are all over the world is used for LST on almost 30 percent if you have an
on almost 30 percent if you have an
on almost 30 percent if you have an Amazon echo you can ask a questions and
Amazon echo you can ask a questions and
Amazon echo you can ask a questions and it answers you and the voice that you
it answers you and the voice that you
it answers you and the voice that you hear it’s not a recording it’s an LS TM
hear it’s not a recording it’s an LS TM
hear it’s not a recording it’s an LS TM network which has learned from training
network which has learned from training
network which has learned from training examples to sound like a female voice if
examples to sound like a female voice if
examples to sound like a female voice if you have an iPhone and you’re using the
you have an iPhone and you’re using the
you have an iPhone and you’re using the quick type it’s trying to predict what
quick type it’s trying to predict what
quick type it’s trying to predict what you want to do next given all the
you want to do next given all the
you want to do next given all the previous context of what you did so far
previous context of what you did so far
previous context of what you did so far again that’s an LS DM which has
again that’s an LS DM which has
again that’s an LS DM which has to do that so it’s on a billion iPhones
to do that so it’s on a billion iPhones
to do that so it’s on a billion iPhones you are a large audience by my standards
you are a large audience by my standards
you are a large audience by my standards but when we started this work decades
but when we started this work decades
but when we started this work decades ago in the early 90s only few people who
ago in the early 90s only few people who
ago in the early 90s only few people who were interested in that
were interested in that
were interested in that because computers were so slow and you
because computers were so slow and you
because computers were so slow and you couldn’t do so much with it and I
couldn’t do so much with it and I
couldn’t do so much with it and I remember I gave a talk at a conference
remember I gave a talk at a conference
remember I gave a talk at a conference and there was just one single person in
and there was just one single person in
and there was just one single person in the audience a young lady I said young
the audience a young lady I said young
the audience a young lady I said young lady it’s very embarrassing but
lady it’s very embarrassing but
lady it’s very embarrassing but apparently today I’m going to give this
apparently today I’m going to give this
apparently today I’m going to give this talk just to you and she said okay but
talk just to you and she said okay but
talk just to you and she said okay but please hurry I am the next speaker
since then we have greatly profited from
since then we have greatly profited from the fact that every five years computers
the fact that every five years computers
the fact that every five years computers again in ten times cheaper which is an
again in ten times cheaper which is an
again in ten times cheaper which is an old trend that has held since 1941 at
old trend that has held since 1941 at
old trend that has held since 1941 at least since this man Conrad Susan built
least since this man Conrad Susan built
least since this man Conrad Susan built the first working program control
the first working program control
the first working program control computer in Berlin and he could could do
computer in Berlin and he could could do
computer in Berlin and he could could do roughly one operation per second one and
roughly one operation per second one and
roughly one operation per second one and then ten years later for the same prize
then ten years later for the same prize
then ten years later for the same prize one could do 100 operations 30 years
one could do 100 operations 30 years
one could do 100 operations 30 years later 1 million operations were the same
later 1 million operations were the same
later 1 million operations were the same price and today after 75 years we can do
price and today after 75 years we can do
price and today after 75 years we can do a million billion times as much for the
a million billion times as much for the
a million billion times as much for the same price and the trend is not about to
same price and the trend is not about to
same price and the trend is not about to stop because the physical limits are
stop because the physical limits are
stop because the physical limits are much further out there rather soon and
much further out there rather soon and
much further out there rather soon and not so many years or decades we will for
not so many years or decades we will for
not so many years or decades we will for the first time have little computational
the first time have little computational
the first time have little computational devices that can compute as much as a
devices that can compute as much as a
devices that can compute as much as a human brain and this a trend doesn’t
human brain and this a trend doesn’t
human brain and this a trend doesn’t break 50 years later there will be a
break 50 years later there will be a
break 50 years later there will be a little computational device for the same
little computational device for the same
little computational device for the same price that can compute as much as all 10
price that can compute as much as all 10
price that can compute as much as all 10 billion human brains taken together and
billion human brains taken together and
billion human brains taken together and there will not only be one of those
there will not only be one of those
there will not only be one of those devices but many many many everything is
devices but many many many everything is
devices but many many many everything is going to change already in 2011
going to change already in 2011
going to change already in 2011 computers were fast enough such that our
computers were fast enough such that our
computers were fast enough such that our deep learning methods for the first time
deep learning methods for the first time
deep learning methods for the first time could achieve a superhuman
could achieve a superhuman
could achieve a superhuman pattern-recognition result and was the
pattern-recognition result and was the
pattern-recognition result and was the first superhuman result and the history
first superhuman result and the history
first superhuman result and the history of computer vision and back then
of computer vision and back then
of computer vision and back then computers were 20 times more expensive
computers were 20 times more expensive
computers were 20 times more expensive than today so today for the same price
than today so today for the same price
than today so today for the same price we can do 20 times as much and just a
we can do 20 times as much and just a
we can do 20 times as much and just a few five years ago five years ago when
few five years ago five years ago when
few five years ago five years ago when computers were 10 times more expensive
computers were 10 times more expensive
computers were 10 times more expensive than today
than today
than today we already could win for the first time
we already could win for the first time
we already could win for the first time medical imaging competitions what you
medical imaging competitions what you
medical imaging competitions what you see behind me is a slice through the
see behind me is a slice through the
see behind me is a slice through the female breast and the tissue that you
female breast and the tissue that you
female breast and the tissue that you see there has all kinds of cells and
see there has all kinds of cells and
see there has all kinds of cells and normally you need a trained doctor a
normally you need a trained doctor a
normally you need a trained doctor a trained the solid
trained the solid
trained the solid who is able to detect the dangerous
who is able to detect the dangerous
who is able to detect the dangerous cancer cells or pre-cancer cells now our
cancer cells or pre-cancer cells now our
cancer cells or pre-cancer cells now our stupid network knows nothing about
stupid network knows nothing about
stupid network knows nothing about cancer knows nothing about vision it
cancer knows nothing about vision it
cancer knows nothing about vision it knows nothing in the beginning but we
knows nothing in the beginning but we
knows nothing in the beginning but we can train it to imitate the human
can train it to imitate the human
can train it to imitate the human teacher the doctor and it became as good
teacher the doctor and it became as good
teacher the doctor and it became as good or better than the best competitors and
or better than the best competitors and
or better than the best competitors and very soon all of medical diagnosis is
very soon all of medical diagnosis is
very soon all of medical diagnosis is going to be superhuman and it’s going to
going to be superhuman and it’s going to
going to be superhuman and it’s going to be mandatory because it’s going to be so
be mandatory because it’s going to be so
be mandatory because it’s going to be so much better than the doctors after this
much better than the doctors after this
much better than the doctors after this all kinds of medical imaging startups
all kinds of medical imaging startups
all kinds of medical imaging startups were founded focusing just on this
were founded focusing just on this
were founded focusing just on this because it’s so important we can also
because it’s so important we can also
because it’s so important we can also use lsdm to train robots one important
use lsdm to train robots one important
use lsdm to train robots one important thing I want to say is that we not only
thing I want to say is that we not only
thing I want to say is that we not only have systems that slavishly
have systems that slavishly
have systems that slavishly imitate what humans show them no we also
imitate what humans show them no we also
imitate what humans show them no we also have a eyes that set themselves their
have a eyes that set themselves their
have a eyes that set themselves their own goals and like little babies invent
own goals and like little babies invent
own goals and like little babies invent their own experiment to explore the
their own experiment to explore the
their own experiment to explore the world and to figure out what you can do
world and to figure out what you can do
world and to figure out what you can do in the world without a teacher and
in the world without a teacher and
in the world without a teacher and becoming more and more general problem
becoming more and more general problem
becoming more and more general problem solvers in the process by learning new
solvers in the process by learning new
solvers in the process by learning new skills on top of old skills and this is
skills on top of old skills and this is
skills on top of old skills and this is going to scale we call that artificial
going to scale we call that artificial
going to scale we call that artificial curiosity or a recent password is power
curiosity or a recent password is power
curiosity or a recent password is power plain learning to become a more and more
plain learning to become a more and more
plain learning to become a more and more general problems over by learning to
general problems over by learning to
general problems over by learning to invent like a scientist one new
invent like a scientist one new
invent like a scientist one new interesting goal after Nathan and and
interesting goal after Nathan and and
interesting goal after Nathan and and it’s going to scale and I think in not
it’s going to scale and I think in not
it’s going to scale and I think in not so many years from now for the first
so many years from now for the first
so many years from now for the first time we are going to have an animal like
time we are going to have an animal like
time we are going to have an animal like AI you don’t have that yet on the level
AI you don’t have that yet on the level
AI you don’t have that yet on the level of a little crowd which already can
of a little crowd which already can
of a little crowd which already can learn to use two worlds for example
learn to use two worlds for example
learn to use two worlds for example little monkey and once we have that it
little monkey and once we have that it
little monkey and once we have that it may take just a few decades to do the
may take just a few decades to do the
may take just a few decades to do the final step towards human level
final step towards human level
final step towards human level intelligence because technological
intelligence because technological
intelligence because technological evolution is about a million times
evolution is about a million times
evolution is about a million times a million times faster than biological
a million times faster than biological
a million times faster than biological evolution and biological evolution
evolution and biological evolution
evolution and biological evolution needed 3.5 billion years to evolve a
needed 3.5 billion years to evolve a
needed 3.5 billion years to evolve a monkey a monkey from scratch but then
monkey a monkey from scratch but then
monkey a monkey from scratch but then just a few tens of millions of years
just a few tens of millions of years
just a few tens of millions of years afterwards to evolve human level
afterwards to evolve human level
afterwards to evolve human level intelligence we have a company which is
intelligence we have a company which is
intelligence we have a company which is called Mason’s like birth in English
called Mason’s like birth in English
called Mason’s like birth in English Mason’s but spelled in a different way
Mason’s but spelled in a different way
Mason’s but spelled in a different way which is trying to make this a reality
which is trying to make this a reality
which is trying to make this a reality and build the first true general and
and build the first true general and
and build the first true general and purpose AI at the moment almost all
purpose AI at the moment almost all
purpose AI at the moment almost all research in AI is very human centric and
research in AI is very human centric and
research in AI is very human centric and it’s all about making human lives longer
it’s all about making human lives longer
it’s all about making human lives longer and healthier and easier and making
and healthier and easier and making
and healthier and easier and making humans more addicted to their
humans more addicted to their
humans more addicted to their smartphones but in the long run a eyes
smartphones but in the long run a eyes
smartphones but in the long run a eyes are going to especially the smart ones
are going to especially the smart ones
are going to especially the smart ones are going to set themselves their own
are going to set themselves their own
are going to set themselves their own goals and I have no doubt in my mind
goals and I have no doubt in my mind
goals and I have no doubt in my mind that they are going to become much
that they are going to become much
that they are going to become much smarter than we are
smarter than we are
smarter than we are and what are they going to do of course
and what are they going to do of course
and what are they going to do of course they are going to realize what we have
they are going to realize what we have
they are going to realize what we have realized a long time ago namely that
realized a long time ago namely that
realized a long time ago namely that most of the resources in the solar
most of the resources in the solar
most of the resources in the solar system or in general are not in our
system or in general are not in our
system or in general are not in our little biosphere they are out there in
little biosphere they are out there in
little biosphere they are out there in space and so of course they are going to
space and so of course they are going to
space and so of course they are going to emigrate and of course they are going to
use trillions of self-replicating robot
use trillions of self-replicating robot factories to expand in form of growing
factories to expand in form of growing
factories to expand in form of growing AI bubble which within a few hundred
AI bubble which within a few hundred
AI bubble which within a few hundred thousand years is going to cover the
thousand years is going to cover the
thousand years is going to cover the entire galaxy by senders and receivers
entire galaxy by senders and receivers
entire galaxy by senders and receivers such that a eyes can travel the way they
such that a eyes can travel the way they
such that a eyes can travel the way they are already traveling in my lab by radio
are already traveling in my lab by radio
are already traveling in my lab by radio from sender to receiver Wireless so what
from sender to receiver Wireless so what
from sender to receiver Wireless so what we are witnessing now is much more than
we are witnessing now is much more than
we are witnessing now is much more than just another Industrial Revolution this
just another Industrial Revolution this
just another Industrial Revolution this is something that transcends humankind
is something that transcends humankind
is something that transcends humankind and even life itself the last time
and even life itself the last time
and even life itself the last time something so important has happened was
something so important has happened was
something so important has happened was maybe 3.5 billion years ago when life
maybe 3.5 billion years ago when life
maybe 3.5 billion years ago when life was invented a new type of life is going
was invented a new type of life is going
was invented a new type of life is going to emerge from our little planet and
to emerge from our little planet and
to emerge from our little planet and it’s going to colonize and transform the
it’s going to colonize and transform the
it’s going to colonize and transform the entire universe the universe is still
entire universe the universe is still
entire universe the universe is still young it’s only 13.8 billion years old
young it’s only 13.8 billion years old
young it’s only 13.8 billion years old it’s going to become much older than
it’s going to become much older than
it’s going to become much older than that many times more many times older
that many times more many times older
that many times more many times older than that so there’s plenty of time to
than that so there’s plenty of time to
than that so there’s plenty of time to reach all of it or all of the visible
reach all of it or all of the visible
reach all of it or all of the visible parts totally within the limits of light
parts totally within the limits of light
parts totally within the limits of light speed and physics a new type of life is
speed and physics a new type of life is
speed and physics a new type of life is going to make the universe intelligent
going to make the universe intelligent
going to make the universe intelligent now of course we are not going to remain
now of course we are not going to remain
now of course we are not going to remain the crown of creation of course not but
the crown of creation of course not but
the crown of creation of course not but there is still beauty in seeing yourself
there is still beauty in seeing yourself
there is still beauty in seeing yourself as part of a grander process that leads
as part of a grander process that leads
as part of a grander process that leads the cosmos from low complexity towards
the cosmos from low complexity towards
the cosmos from low complexity towards higher complexity it’s a privilege to
higher complexity it’s a privilege to
higher complexity it’s a privilege to live at a time where we can witness the
live at a time where we can witness the
live at a time where we can witness the beginnings of that and where we can
beginnings of that and where we can
beginnings of that and where we can contribute something to that thank you
contribute something to that thank you
contribute something to that thank you for your patience
”
Be First to Comment