we’re in an era now of technologies advancing faster than they ever have
advancing faster than they ever have
advancing faster than they ever have before raising important questions about
before raising important questions about
before raising important questions about labor market disruptions but I’m going
labor market disruptions but I’m going
labor market disruptions but I’m going to talk to you instead about how we can
to talk to you instead about how we can
to talk to you instead about how we can leverage these technologies to learn
leverage these technologies to learn
leverage these technologies to learn skills and ways that were never before
skills and ways that were never before
skills and ways that were never before possible and unlock our full potential
possible and unlock our full potential
possible and unlock our full potential and in particular I’m going to talk
and in particular I’m going to talk
and in particular I’m going to talk about how we can digitize the master
about how we can digitize the master
about how we can digitize the master apprentice relationship so it’s hard to
apprentice relationship so it’s hard to
apprentice relationship so it’s hard to avoid today’s stories about AI and
avoid today’s stories about AI and
avoid today’s stories about AI and machine learning advancing in their
machine learning advancing in their
machine learning advancing in their capabilities and really reaching
capabilities and really reaching
capabilities and really reaching unprecedented levels at the same time
unprecedented levels at the same time
unprecedented levels at the same time robotics and automation systems have the
robotics and automation systems have the
robotics and automation systems have the potential to streamline physical tasks
potential to streamline physical tasks
potential to streamline physical tasks as you see here in this amazing video
as you see here in this amazing video
as you see here in this amazing video from amazon fulfillment warehouse now in
from amazon fulfillment warehouse now in
from amazon fulfillment warehouse now in some cases these change the nature of
some cases these change the nature of
some cases these change the nature of the work that humans do but in other
the work that humans do but in other
the work that humans do but in other cases they completely remove the humans
cases they completely remove the humans
cases they completely remove the humans from the equation so with autonomous
from the equation so with autonomous
from the equation so with autonomous vehicles alone it’s projected that this
vehicles alone it’s projected that this
vehicles alone it’s projected that this is going to cost approximately 300,000
is going to cost approximately 300,000
is going to cost approximately 300,000 jobs to be lost per year once they reach
jobs to be lost per year once they reach
jobs to be lost per year once they reach their peak yeah that’s a scary number
their peak yeah that’s a scary number
their peak yeah that’s a scary number and it’s natural to be frightened about
and it’s natural to be frightened about
and it’s natural to be frightened about this but we’re still quite a ways away
this but we’re still quite a ways away
this but we’re still quite a ways away from AI and machine learning replacing
from AI and machine learning replacing
from AI and machine learning replacing everything we do and robots taking over
everything we do and robots taking over
everything we do and robots taking over our jobs so we’re not quite off the hook
our jobs so we’re not quite off the hook
our jobs so we’re not quite off the hook yet instead and what we’ve learned in
yet instead and what we’ve learned in
yet instead and what we’ve learned in the past is that these disruptive
the past is that these disruptive
the past is that these disruptive technologies don’t take away our jobs
technologies don’t take away our jobs
technologies don’t take away our jobs they change the nature of the work that
they change the nature of the work that
they change the nature of the work that we do and in essence they create these
we do and in essence they create these
we do and in essence they create these hybrid systems where humans and
hybrid systems where humans and
hybrid systems where humans and technologies work together so in the
technologies work together so in the
technologies work together so in the context of architecture which is where I
context of architecture which is where I
context of architecture which is where I work at Autodesk research architects now
work at Autodesk research architects now
work at Autodesk research architects now learn to use computer-aided design
learn to use computer-aided design
learn to use computer-aided design software they no longer need to become
software they no longer need to become
software they no longer need to become masters of physical drafting skills so
masters of physical drafting skills so
masters of physical drafting skills so now is a time more than ever when we
now is a time more than ever when we
now is a time more than ever when we need to have adaptable expertise often
need to have adaptable expertise often
need to have adaptable expertise often learning new skills on the job and I’m
learning new skills on the job and I’m
learning new skills on the job and I’m going to talk to you about how we can
going to talk to you about how we can
going to talk to you about how we can digitize the master apprentice
digitize the master apprentice
digitize the master apprentice relationship to do this now technology
relationship to do this now technology
relationship to do this now technology guided learning is not
guided learning is not
guided learning is not new and in fact even before computers
new and in fact even before computers
new and in fact even before computers existed psychologists were developing
existed psychologists were developing
existed psychologists were developing what they call teaching machines that
what they call teaching machines that
what they call teaching machines that were thought to lead to an industrial
were thought to lead to an industrial
were thought to lead to an industrial revolution of teaching it was to the
revolution of teaching it was to the
revolution of teaching it was to the point where is even question whether the
point where is even question whether the
point where is even question whether the learning experience would become fully
learning experience would become fully
learning experience would become fully automated and our teachers would be
automated and our teachers would be
automated and our teachers would be replaced by robots of course it’s a
replaced by robots of course it’s a
replaced by robots of course it’s a vision that was never realized although
vision that was never realized although
vision that was never realized although I always questioned about my own grade
I always questioned about my own grade
I always questioned about my own grade for science teacher but the capabilities
for science teacher but the capabilities
for science teacher but the capabilities of today’s technologies do allow us to
of today’s technologies do allow us to
of today’s technologies do allow us to learn in ways that were never before
learn in ways that were never before
learn in ways that were never before possible like I said I work at Autodesk
possible like I said I work at Autodesk
possible like I said I work at Autodesk which makes some of the world’s most
which makes some of the world’s most
which makes some of the world’s most advanced technology and software for
advanced technology and software for
advanced technology and software for doing 3d design from towering
doing 3d design from towering
doing 3d design from towering skyscrapers to special effects and
skyscrapers to special effects and
skyscrapers to special effects and Hollywood movies it also happens to be
Hollywood movies it also happens to be
Hollywood movies it also happens to be some of the most complex software to
some of the most complex software to
some of the most complex software to learn in the world having hundreds and
learn in the world having hundreds and
learn in the world having hundreds and often thousands of commands and
often thousands of commands and
often thousands of commands and workflows that you need to become a
workflows that you need to become a
workflows that you need to become a master of now with my colleagues at
master of now with my colleagues at
master of now with my colleagues at Autodesk research in the user-interface
Autodesk research in the user-interface
Autodesk research in the user-interface research group I’ve spent the last 10
research group I’ve spent the last 10
research group I’ve spent the last 10 years of my research trying to
years of my research trying to
years of my research trying to understand how technology can aid in the
understand how technology can aid in the
understand how technology can aid in the learning process in these complex
learning process in these complex
learning process in these complex environments for example we can
environments for example we can
environments for example we can instrument the software to track
instrument the software to track
instrument the software to track everything you’re doing and by analyzing
everything you’re doing and by analyzing
everything you’re doing and by analyzing what the experts are doing and what the
what the experts are doing and what the
what the experts are doing and what the rest of the community is doing we can
rest of the community is doing we can
rest of the community is doing we can recommend new commands and workflows for
recommend new commands and workflows for
recommend new commands and workflows for you to use so with this project which we
you to use so with this project which we
you to use so with this project which we called community commands the learning
called community commands the learning
called community commands the learning is occurring in real time as you’re
is occurring in real time as you’re
is occurring in real time as you’re working with the system personalized
working with the system personalized
working with the system personalized recommendations are coming to you just
recommendations are coming to you just
recommendations are coming to you just as if you were working right next to an
as if you were working right next to an
as if you were working right next to an expert but I ask myself can we extend
expert but I ask myself can we extend
expert but I ask myself can we extend this paradigm of what we call
this paradigm of what we call
this paradigm of what we call over-the-shoulder learning beyond just
over-the-shoulder learning beyond just
over-the-shoulder learning beyond just computer-based tasks can we use
computer-based tasks can we use
computer-based tasks can we use technology to achieve mastery in the
technology to achieve mastery in the
technology to achieve mastery in the physical realm and unlock our potential
physical realm and unlock our potential
physical realm and unlock our potential in essence what I was asking is can we
in essence what I was asking is can we
in essence what I was asking is can we digitize the master apprentice
digitize the master apprentice
digitize the master apprentice relationship let’s look at a couple
relationship let’s look at a couple
relationship let’s look at a couple examples to motivate this here’s an
examples to motivate this here’s an
examples to motivate this here’s an athlete using a training apparatus to
athlete using a training apparatus to
athlete using a training apparatus to improve their baseball swing now surely
improve their baseball swing now surely
improve their baseball swing now surely this type of repetitive practice is
this type of repetitive practice is
this type of repetitive practice is gonna have some benefits but he might be
gonna have some benefits but he might be
gonna have some benefits but he might be making some errors and a device like
making some errors and a device like
making some errors and a device like this is completely passive it’s
this is completely passive it’s
this is completely passive it’s providing no feedback on the errors he’s
providing no feedback on the errors he’s
providing no feedback on the errors he’s making and no guidance on specific
making and no guidance on specific
making and no guidance on specific things that he could improve upon what I
things that he could improve upon what I
things that he could improve upon what I want to do is to provide ways for
want to do is to provide ways for
want to do is to provide ways for systems like this to provide a model of
systems like this to provide a model of
systems like this to provide a model of mentorship so people can learn in the
mentorship so people can learn in the
mentorship so people can learn in the midst of an activity let’s consider a
midst of an activity let’s consider a
midst of an activity let’s consider a best-in-class example of this so what
best-in-class example of this so what
best-in-class example of this so what this person needs is a mr. Miyagi this
this person needs is a mr. Miyagi this
this person needs is a mr. Miyagi this is of course from the the movie The
is of course from the the movie The
is of course from the the movie The Karate Kid and if you go back and watch
Karate Kid and if you go back and watch
Karate Kid and if you go back and watch the scene or mr. Miyagi is teaching
the scene or mr. Miyagi is teaching
the scene or mr. Miyagi is teaching Danielle all these all these different
Danielle all these all these different
Danielle all these all these different movements it’s actually a really great
movements it’s actually a really great
movements it’s actually a really great example of the rich interactions that
example of the rich interactions that
example of the rich interactions that occur between a master and an apprentice
occur between a master and an apprentice
occur between a master and an apprentice mr. Miyagi doesn’t just show the
mr. Miyagi doesn’t just show the
mr. Miyagi doesn’t just show the movements he’s constantly providing
movements he’s constantly providing
movements he’s constantly providing feedback and guidance to Daniel he’s
feedback and guidance to Daniel he’s
feedback and guidance to Daniel he’s also creating this close personal
also creating this close personal
also creating this close personal connection so we can truly understand
connection so we can truly understand
connection so we can truly understand what it is that Daniel needs to learn
what it is that Daniel needs to learn
what it is that Daniel needs to learn best now to add a personal touch to this
best now to add a personal touch to this
best now to add a personal touch to this story I had my own mr. Miyagi perhaps a
story I had my own mr. Miyagi perhaps a
story I had my own mr. Miyagi perhaps a unique hobby for a computer scientist I
unique hobby for a computer scientist I
unique hobby for a computer scientist I trained in Muay Thai so this is the Thai
trained in Muay Thai so this is the Thai
trained in Muay Thai so this is the Thai equivalent to kickboxing and there with
equivalent to kickboxing and there with
equivalent to kickboxing and there with me is Simon Marcus who is my master or
me is Simon Marcus who is my master or
me is Simon Marcus who is my master or anti crew now I was incredibly fortunate
anti crew now I was incredibly fortunate
anti crew now I was incredibly fortunate to work with Simon he’s actually a
to work with Simon he’s actually a
to work with Simon he’s actually a two-time world champion in kickboxing
two-time world champion in kickboxing
two-time world champion in kickboxing and still an undefeated professional my
and still an undefeated professional my
and still an undefeated professional my Thai fighter so Simon provided me all
Thai fighter so Simon provided me all
Thai fighter so Simon provided me all the necessary things I needed to know to
the necessary things I needed to know to
the necessary things I needed to know to learn the physical skills surrounding
learn the physical skills surrounding
learn the physical skills surrounding MoIT I was also very clear from working
MoIT I was also very clear from working
MoIT I was also very clear from working with Simon who seems very good at
with Simon who seems very good at
with Simon who seems very good at understanding each individual student
understanding each individual student
understanding each individual student and what their strengths and weaknesses
and what their strengths and weaknesses
and what their strengths and weaknesses were and to teach accordingly and that’s
were and to teach accordingly and that’s
were and to teach accordingly and that’s a fundamental aspect of the master
a fundamental aspect of the master
a fundamental aspect of the master apprentice relationship and something
apprentice relationship and something
apprentice relationship and something that doesn’t always occur and the
that doesn’t always occur and the
that doesn’t always occur and the one-size-fits-all approach to education
one-size-fits-all approach to education
one-size-fits-all approach to education and I’m sure every one of you here can
and I’m sure every one of you here can
and I’m sure every one of you here can think about a mentor that you’ve had in
think about a mentor that you’ve had in
think about a mentor that you’ve had in your life whether it was as a child or
your life whether it was as a child or
your life whether it was as a child or even as an adult that’s had a
even as an adult that’s had a
even as an adult that’s had a significant impact on who you are today
significant impact on who you are today
significant impact on who you are today for me I was fortunate to work with
for me I was fortunate to work with
for me I was fortunate to work with Simon now not everyone has their mr.
Simon now not everyone has their mr.
Simon now not everyone has their mr. Miyagi or their Simon Marcus
Miyagi or their Simon Marcus
Miyagi or their Simon Marcus they need them the most so the question
they need them the most so the question
they need them the most so the question remains can we digitize and by doing so
remains can we digitize and by doing so
remains can we digitize and by doing so democratize this master apprentice
democratize this master apprentice
democratize this master apprentice relationship and I would argue that with
relationship and I would argue that with
relationship and I would argue that with today’s emerging technologies we can do
today’s emerging technologies we can do
today’s emerging technologies we can do just that so here’s some examples this
just that so here’s some examples this
just that so here’s some examples this is a project we did call you move where
is a project we did call you move where
is a project we did call you move where we used an augmented reality mirror to
we used an augmented reality mirror to
we used an augmented reality mirror to teach physical skills such as dance and
teach physical skills such as dance and
teach physical skills such as dance and athletics and get the feedback that you
athletics and get the feedback that you
athletics and get the feedback that you typically typically get from a trainer
typically typically get from a trainer
typically typically get from a trainer the system would capture your movements
the system would capture your movements
the system would capture your movements with a 3d depth camera and then compare
with a 3d depth camera and then compare
with a 3d depth camera and then compare them to the recorded movements of an
them to the recorded movements of an
them to the recorded movements of an expert personalized feedback was then
expert personalized feedback was then
expert personalized feedback was then projected back on the mirror and you’d
projected back on the mirror and you’d
projected back on the mirror and you’d see it overlaid on your own physical
see it overlaid on your own physical
see it overlaid on your own physical reflection applying this to the trades
reflection applying this to the trades
reflection applying this to the trades this is a project we called the smart
this is a project we called the smart
this is a project we called the smart makerspace which emerged digitized
makerspace which emerged digitized
makerspace which emerged digitized learning content within a physical
learning content within a physical
learning content within a physical makerspace environment as you worked on
makerspace environment as you worked on
makerspace environment as you worked on a physical task the system would guide
a physical task the system would guide
a physical task the system would guide you on how to use the required tools and
you on how to use the required tools and
you on how to use the required tools and materials it would track your progress
materials it would track your progress
materials it would track your progress through the tasks and update the
through the tasks and update the
through the tasks and update the instructions in real time but can we
instructions in real time but can we
instructions in real time but can we take this even further what about
take this even further what about
take this even further what about large-scale construction like this can
large-scale construction like this can
large-scale construction like this can we take the traditional mentorship model
we take the traditional mentorship model
we take the traditional mentorship model and digitize it within this context with
and digitize it within this context with
and digitize it within this context with our crowdsource fabrication project we
our crowdsource fabrication project we
our crowdsource fabrication project we did exactly that so with this project we
did exactly that so with this project we
did exactly that so with this project we had over a hundred volunteers
had over a hundred volunteers
had over a hundred volunteers spontaneously come together and build
spontaneously come together and build
spontaneously come together and build this 12-foot tall Pavilion structure
this 12-foot tall Pavilion structure
this 12-foot tall Pavilion structure what was interesting about this project
what was interesting about this project
what was interesting about this project is that the volunteers had no prior
is that the volunteers had no prior
is that the volunteers had no prior knowledge of the of the project the
knowledge of the of the project the
knowledge of the of the project the structure or the tools and materials
structure or the tools and materials
structure or the tools and materials that were needed to build it instead we
that were needed to build it instead we
that were needed to build it instead we use the combination of wearable devices
use the combination of wearable devices
use the combination of wearable devices and sensors integrated in the
and sensors integrated in the
and sensors integrated in the environment to provide real-time
environment to provide real-time
environment to provide real-time learning here’s a brief overview of the
learning here’s a brief overview of the
learning here’s a brief overview of the experience so when volunteers came to
experience so when volunteers came to
experience so when volunteers came to the environment they were given a
the environment they were given a
the environment they were given a SmartWatch and the SmartWatch provided
SmartWatch and the SmartWatch provided
SmartWatch and the SmartWatch provided just-in-time
just-in-time
just-in-time and location-aware instructions they
and location-aware instructions they
and location-aware instructions they were first directed to gather three
were first directed to gather three
were first directed to gather three pieces of bamboo and then take those
pieces of bamboo and then take those
pieces of bamboo and then take those bamboo pieces to a robotic device where
bamboo pieces to a robotic device where
bamboo pieces to a robotic device where they’d load them onto
they’d load them onto
they’d load them onto special tool they would tie a string to
special tool they would tie a string to
special tool they would tie a string to the bamboo and then the robot would
the bamboo and then the robot would
the bamboo and then the robot would begin a special fabrication method to
begin a special fabrication method to
begin a special fabrication method to create individual modules there were two
create individual modules there were two
create individual modules there were two hundred and twenty four unique modules
hundred and twenty four unique modules
hundred and twenty four unique modules in the construction of this pavilion now
in the construction of this pavilion now
in the construction of this pavilion now this is showing a powerful concept where
this is showing a powerful concept where
this is showing a powerful concept where again humans and technology are working
again humans and technology are working
again humans and technology are working in a collaborative way together to do
in a collaborative way together to do
in a collaborative way together to do something that would be difficult for
something that would be difficult for
something that would be difficult for either of them to do independently once
either of them to do independently once
either of them to do independently once a module was completed the human worker
a module was completed the human worker
a module was completed the human worker would remove it from the robot and have
would remove it from the robot and have
would remove it from the robot and have to figure out where it goes because each
to figure out where it goes because each
to figure out where it goes because each module was unique so what we did was we
module was unique so what we did was we
module was unique so what we did was we embedded wireless LEDs within the actual
embedded wireless LEDs within the actual
embedded wireless LEDs within the actual materials that would illuminate and show
materials that would illuminate and show
materials that would illuminate and show the worker exactly where their module
the worker exactly where their module
the worker exactly where their module had to go this entire process was
had to go this entire process was
had to go this entire process was coordinated by a back-end intelligent
coordinated by a back-end intelligent
coordinated by a back-end intelligent agent which we called a digital Foreman
agent which we called a digital Foreman
agent which we called a digital Foreman engine and in the end we had the world’s
engine and in the end we had the world’s
engine and in the end we had the world’s first example of what we called
first example of what we called
first example of what we called crowdsource fabrication where a group of
crowdsource fabrication where a group of
crowdsource fabrication where a group of unskilled volunteers came came together
unskilled volunteers came came together
unskilled volunteers came came together to build an architectural scale
to build an architectural scale
to build an architectural scale structure using a combination of user
structure using a combination of user
structure using a combination of user interface technologies now here’s where
interface technologies now here’s where
interface technologies now here’s where this gets really exciting there’s
this gets really exciting there’s
this gets really exciting there’s volunteer efforts across the world such
volunteer efforts across the world such
volunteer efforts across the world such as Habitat for Humanity where volunteers
as Habitat for Humanity where volunteers
as Habitat for Humanity where volunteers come together to construct housing for
come together to construct housing for
come together to construct housing for low-income families imagine the
low-income families imagine the
low-income families imagine the implications of each worker came to the
implications of each worker came to the
implications of each worker came to the site and received the learning and
site and received the learning and
site and received the learning and training resources they needed in real
training resources they needed in real
training resources they needed in real time while they contributed to the build
time while they contributed to the build
time while they contributed to the build another example that’s very relevant is
another example that’s very relevant is
another example that’s very relevant is the construction and assembly of
the construction and assembly of
the construction and assembly of shelters for refugees and asylum seekers
shelters for refugees and asylum seekers
shelters for refugees and asylum seekers with new fabrication techniques today
with new fabrication techniques today
with new fabrication techniques today there’s companies like better shelter
there’s companies like better shelter
there’s companies like better shelter that are creating shelters that are more
that are creating shelters that are more
that are creating shelters that are more robust and can provide families with
robust and can provide families with
robust and can provide families with more security but the assembly is still
more security but the assembly is still
more security but the assembly is still a major bottleneck and let’s just say
a major bottleneck and let’s just say
a major bottleneck and let’s just say the instructions for assembling these
the instructions for assembling these
the instructions for assembling these are suboptimal they’re literally like
are suboptimal they’re literally like
are suboptimal they’re literally like following IKEA instructions to try to
following IKEA instructions to try to
following IKEA instructions to try to build a house so the technologies I’ve
build a house so the technologies I’ve
build a house so the technologies I’ve been showing you could potentially lead
been showing you could potentially lead
been showing you could potentially lead to builds like this occurring in
to builds like this occurring in
to builds like this occurring in fractions of the time and also giving
fractions of the time and also giving
fractions of the time and also giving the workers lifelong skills that they
the workers lifelong skills that they
the workers lifelong skills that they can carry forward the master apprentice
can carry forward the master apprentice
can carry forward the master apprentice relationship is something that has been
relationship is something that has been
relationship is something that has been around with us for centuries but with
around with us for centuries but with
around with us for centuries but with these new technology
these new technology
these new technology we have opportunities to learn in ways
we have opportunities to learn in ways
we have opportunities to learn in ways that were never before possible we can
that were never before possible we can
that were never before possible we can digitize the master apprentice
digitize the master apprentice
digitize the master apprentice relationship and unlock our full
relationship and unlock our full
relationship and unlock our full potential now my challenge to you going
potential now my challenge to you going
potential now my challenge to you going forward for developers designers and
forward for developers designers and
forward for developers designers and computer scientists is to think about
computer scientists is to think about
computer scientists is to think about ways we can incorporate the personal
ways we can incorporate the personal
ways we can incorporate the personal aspects of these master apprentice
aspects of these master apprentice
aspects of these master apprentice relationships into our broader
relationships into our broader
relationships into our broader interactions with technology thank you
interactions with technology thank you
interactions with technology thank you [Applause]
”
Be First to Comment