Augmented Creativity | Bob Sumner | TEDxZurich

I’ve been working for several years now

I’ve been working for several years now on a collaborative project together with

on a collaborative project together with

on a collaborative project together with the ETH Zurich game Technology Center

the ETH Zurich game Technology Center

the ETH Zurich game Technology Center and Disney Research about augmented

and Disney Research about augmented

and Disney Research about augmented reality

reality

reality so what’s augmented reality you’ve

so what’s augmented reality you’ve

so what’s augmented reality you’ve probably heard this term together with

probably heard this term together with

probably heard this term together with virtual reality these usually go

virtual reality these usually go

virtual reality these usually go hand-in-hand

hand-in-hand

hand-in-hand virtual reality is the idea where you

virtual reality is the idea where you

virtual reality is the idea where you put on a virtual reality headset and

put on a virtual reality headset and

put on a virtual reality headset and you’re transformed into the virtual

you’re transformed into the virtual

you’re transformed into the virtual world augmented reality is kind of the

world augmented reality is kind of the

world augmented reality is kind of the flipside of this idea augmented reality

flipside of this idea augmented reality

flipside of this idea augmented reality takes virtual objects and brings them

takes virtual objects and brings them

takes virtual objects and brings them into your world these are very powerful

into your world these are very powerful

into your world these are very powerful concepts and what I’ve come to learn

concepts and what I’ve come to learn

concepts and what I’ve come to learn over the past years with our experiments

over the past years with our experiments

over the past years with our experiments is that the real winning proposition

is that the real winning proposition

is that the real winning proposition here the real transformative aspect of

here the real transformative aspect of

here the real transformative aspect of this technology is the ability it has to

this technology is the ability it has to

this technology is the ability it has to enhance creativity so why is that that’s

enhance creativity so why is that that’s

enhance creativity so why is that that’s really my core message and if you think

really my core message and if you think

really my core message and if you think about it over the past thousands of

about it over the past thousands of

about it over the past thousands of years you know humans we’ve developed

years you know humans we’ve developed

years you know humans we’ve developed amazing tools to interact and create in

amazing tools to interact and create in

amazing tools to interact and create in the real world and in more recent

the real world and in more recent

the real world and in more recent history since the advent of computers we

history since the advent of computers we

history since the advent of computers we have built interfaces within the

have built interfaces within the

have built interfaces within the computers digital tools that allow us to

computers digital tools that allow us to

computers digital tools that allow us to do things that seem like magic compared

do things that seem like magic compared

do things that seem like magic compared to what was possible only 50 years ago

to what was possible only 50 years ago

to what was possible only 50 years ago and the real promise of augmented

and the real promise of augmented

and the real promise of augmented reality is to merge these two worlds to

reality is to merge these two worlds to

reality is to merge these two worlds to merge the virtual and the physical and

merge the virtual and the physical and

merge the virtual and the physical and it allows us to take the magic of what’s

it allows us to take the magic of what’s

it allows us to take the magic of what’s possible in a digital tool and bring it

possible in a digital tool and bring it

possible in a digital tool and bring it into the world around us and when you

into the world around us and when you

into the world around us and when you combine that power of digital processing

combine that power of digital processing

combine that power of digital processing with the intuitiveness of working in the

with the intuitiveness of working in the

with the intuitiveness of working in the real world then there’s this

real world then there’s this

real world then there’s this exponential increase and what’s possible

exponential increase and what’s possible

exponential increase and what’s possible to create and that’s a really exciting

to create and that’s a really exciting

to create and that’s a really exciting concept so in this bigger spectrum I

concept so in this bigger spectrum I

concept so in this bigger spectrum I want to talk about one aspect that I’m

want to talk about one aspect that I’m

want to talk about one aspect that I’m especially interested in which is the

especially interested in which is the

especially interested in which is the idea of creative play this is the

idea of creative play this is the

idea of creative play this is the concept of children interacting with

concept of children interacting with

concept of children interacting with their physical environment in a playful

their physical environment in a playful

their physical environment in a playful and creative way this is an important

and creative way this is an important

and creative way this is an important part of early childhood and childhood

part of early childhood and childhood

part of early childhood and childhood development but in fact because we have

development but in fact because we have

development but in fact because we have such amazing content ready for

such amazing content ready for

such amazing content ready for consumption there’s a risk and a fear

consumption there’s a risk and a fear

consumption there’s a risk and a fear that children in the end become passive

that children in the end become passive

that children in the end become passive consumers and they lose that direct

consumers and they lose that direct

consumers and they lose that direct physical interaction with the

physical interaction with the

physical interaction with the environment around us and I really

environment around us and I really

environment around us and I really believe that augmented reality holds a

believe that augmented reality holds a

believe that augmented reality holds a unique potential to impact a situation

unique potential to impact a situation

unique potential to impact a situation by providing magical digital overlays on

by providing magical digital overlays on

by providing magical digital overlays on top of traditional real world creative

top of traditional real world creative

top of traditional real world creative activities and this is the concept we’re

activities and this is the concept we’re

activities and this is the concept we’re calling augmented creativity where we

calling augmented creativity where we

calling augmented creativity where we use augmented reality to enhance

use augmented reality to enhance

use augmented reality to enhance creative play so to illustrate this idea

creative play so to illustrate this idea

creative play so to illustrate this idea I want to show you several live demos on

I want to show you several live demos on

I want to show you several live demos on stage using my iPad before I get started

stage using my iPad before I get started

stage using my iPad before I get started I want to give a big personal thanks to

I want to give a big personal thanks to

I want to give a big personal thanks to the team who helped put this together so

the team who helped put this together so

the team who helped put this together so with that we are switching to my iPad

with that we are switching to my iPad

with that we are switching to my iPad screen so what you see behind me is

screen so what you see behind me is

screen so what you see behind me is actually exactly what I see on my screen

actually exactly what I see on my screen

actually exactly what I see on my screen and I want to start with coloring so

and I want to start with coloring so

and I want to start with coloring so this is one of childhoods most early

this is one of childhoods most early

this is one of childhoods most early creative endeavors I have a coloring

creative endeavors I have a coloring

creative endeavors I have a coloring book page here and when I view that page

book page here and when I view that page

book page here and when I view that page our software recognizes the line art

our software recognizes the line art

our software recognizes the line art that’s printed on it and displays this

that’s printed on it and displays this

that’s printed on it and displays this digital character that matches the line

digital character that matches the line

digital character that matches the line art that’s pretty cool already however

art that’s pretty cool already however

art that’s pretty cool already however at the moment she’s blank because we

at the moment she’s blank because we

at the moment she’s blank because we haven’t colored her so I’m going to do

haven’t colored her so I’m going to do

haven’t colored her so I’m going to do some live coloring on her dress right

some live coloring on her dress right

some live coloring on her dress right now just to add a little stripe there

now just to add a little stripe there

now just to add a little stripe there and now

and now

and now I tap the screen this software we’ve

I tap the screen this software we’ve

I tap the screen this software we’ve written dynamically captures that

written dynamically captures that

written dynamically captures that texture that I just colored and applies

texture that I just colored and applies

texture that I just colored and applies it to the dress so in fact I have a

it to the dress so in fact I have a

it to the dress so in fact I have a fully colored version that I made just

fully colored version that I made just

fully colored version that I made just this morning

this morning

this morning so let me capture this one

so let me capture this one

so let me capture this one [Music]

[Music]

[Music] and what’s colored on the front what you

and what’s colored on the front what you

and what’s colored on the front what you see on the front is copied directly

see on the front is copied directly

see on the front is copied directly however we also have an algorithm that

however we also have an algorithm that

however we also have an algorithm that invents what I might have colored on the

invents what I might have colored on the

invents what I might have colored on the back based on what I colored on the

back based on what I colored on the

back based on what I colored on the front now coloring books also often are

front now coloring books also often are

front now coloring books also often are accompanied with puzzles and I have here

accompanied with puzzles and I have here

accompanied with puzzles and I have here maze and you see our system actually

maze and you see our system actually

maze and you see our system actually validates that the maze was solved

validates that the maze was solved

validates that the maze was solved correctly and displays this digital

correctly and displays this digital

correctly and displays this digital character who follows the path that I’ve

character who follows the path that I’ve

character who follows the path that I’ve drawn on the page if I had made a

drawn on the page if I had made a

drawn on the page if I had made a mistake he would have in fact stop and

mistake he would have in fact stop and

mistake he would have in fact stop and wait we also have a circle the word

wait we also have a circle the word

wait we also have a circle the word puzzle so it checks that I’ve circled

puzzle so it checks that I’ve circled

puzzle so it checks that I’ve circled the proper words and delivers this

the proper words and delivers this

the proper words and delivers this reward for completing it correctly so

reward for completing it correctly so

reward for completing it correctly so we’re not restricted to you know things

we’re not restricted to you know things

we’re not restricted to you know things right in front of me on this table

right in front of me on this table

right in front of me on this table there’s a painting beside me which is

there’s a painting beside me which is

there’s a painting beside me which is something you might see at a museum

something you might see at a museum

something you might see at a museum museums have a lot of cultural

museums have a lot of cultural

museums have a lot of cultural importance but for children they can be

importance but for children they can be

importance but for children they can be super boring they’re not the most

super boring they’re not the most

super boring they’re not the most exciting experience Augmented creativity

exciting experience Augmented creativity

exciting experience Augmented creativity can help with that so when I view this

can help with that so when I view this

can help with that so when I view this painting with my app while we see it but

painting with my app while we see it but

painting with my app while we see it but because we’re using the system we can do

because we’re using the system we can do

because we’re using the system we can do some magical things so if I tap on the

some magical things so if I tap on the

some magical things so if I tap on the screen we throw balls of paint and are

screen we throw balls of paint and are

screen we throw balls of paint and are able to recolor this painting to create

able to recolor this painting to create

able to recolor this painting to create something more exciting and I can even

something more exciting and I can even

something more exciting and I can even go farther than that if I swipe we can

go farther than that if I swipe we can

go farther than that if I swipe we can explore some new hairstyles

explore some new hairstyles

explore some new hairstyles for this character let’s find one that’s

for this character let’s find one that’s

for this character let’s find one that’s pretty good I can even move the nose

pretty good I can even move the nose

pretty good I can even move the nose around a little bit oops let’s try to

around a little bit oops let’s try to

around a little bit oops let’s try to make the eyes a little bigger and I

make the eyes a little bigger and I

make the eyes a little bigger and I would say you know with these changes

would say you know with these changes

would say you know with these changes he’s a pretty happy guy so this allows a

he’s a pretty happy guy so this allows a

he’s a pretty happy guy so this allows a child to become their own Picasso when

child to become their own Picasso when

child to become their own Picasso when they visit a museum we’ve also explored

they visit a museum we’ve also explored

they visit a museum we’ve also explored bringing gameplay into this suite of

bringing gameplay into this suite of

bringing gameplay into this suite of demos and for that I have some rubber

demos and for that I have some rubber

demos and for that I have some rubber stamps so these are actually ones we

stamps so these are actually ones we

stamps so these are actually ones we created in our office with our laser

created in our office with our laser

created in our office with our laser cutter and I’m gonna show you how we can

cutter and I’m gonna show you how we can

cutter and I’m gonna show you how we can meld games into the world around us

meld games into the world around us

meld games into the world around us using these stamps and for that I’m

using these stamps and for that I’m

using these stamps and for that I’m gonna just stamp right here on the paper

gonna just stamp right here on the paper

gonna just stamp right here on the paper and when I create that stamp the system

and when I create that stamp the system

and when I create that stamp the system actually recognizes what I’ve just done

actually recognizes what I’ve just done

actually recognizes what I’ve just done and creates this corresponding character

and creates this corresponding character

and creates this corresponding character that matches the stamp it even knew that

that matches the stamp it even knew that

that matches the stamp it even knew that the ink I used was blue and colored the

the ink I used was blue and colored the

the ink I used was blue and colored the character blue gonna make a green one

character blue gonna make a green one

character blue gonna make a green one right now

right now

right now and once this is done we actually can

and once this is done we actually can

and once this is done we actually can implement a simple game where these two

implement a simple game where these two

implement a simple game where these two characters battle one another

and the full interaction really comes

and the full interaction really comes from this physical kind of tangible

from this physical kind of tangible

from this physical kind of tangible movement in the real world now these are

movement in the real world now these are

movement in the real world now these are two different concepts while we have

two different concepts while we have

two different concepts while we have visuals we saw we can create new

visuals we saw we can create new

visuals we saw we can create new interaction what about other sensing

interaction what about other sensing

interaction what about other sensing modalities so how about listening or

modalities so how about listening or

modalities so how about listening or hearing and for that I have a number of

hearing and for that I have a number of

hearing and for that I have a number of a number of cards on the table and these

a number of cards on the table and these

a number of cards on the table and these cards have different instruments and

cards have different instruments and

cards have different instruments and musical styles printed on them so the

musical styles printed on them so the

musical styles printed on them so the first one is the vocals and when I view

first one is the vocals and when I view

first one is the vocals and when I view this card with my iPad we display a

this card with my iPad we display a

this card with my iPad we display a caring for my baby

caring for my baby

caring for my baby how they visit around who’s now tree

how they visit around who’s now tree

how they visit around who’s now tree forming for us he’s singing solo right

forming for us he’s singing solo right

forming for us he’s singing solo right now so let’s give him some instruments

now so let’s give him some instruments

now so let’s give him some instruments to accompany so just like that we have a

to accompany so just like that we have a

to accompany so just like that we have a rock version of this song but we can

rock version of this song but we can

rock version of this song but we can also bring in some other some other

also bring in some other some other

also bring in some other some other instruments so now I’ve actually created

instruments so now I’ve actually created

instruments so now I’ve actually created a reggae track pretty interesting what

a reggae track pretty interesting what

a reggae track pretty interesting what about something a bit more classic so

I’m gonna add this add this string

I’m gonna add this add this string quartet so we have a more classic feel

quartet so we have a more classic feel

quartet so we have a more classic feel which is interesting but I think for

which is interesting but I think for

which is interesting but I think for this group we should rather go full-on

this group we should rather go full-on

this group we should rather go full-on techno

techno

techno [Music]

now the only thing missing is some

now the only thing missing is some lighting effects really limitless

lighting effects really limitless

lighting effects really limitless possibilities we can do with this

possibilities we can do with this

possibilities we can do with this technology I can even for example bring

technology I can even for example bring

technology I can even for example bring in the character that I colored earlier

in the character that I colored earlier

in the character that I colored earlier to join the performance in a duet or

to join the performance in a duet or

to join the performance in a duet or allow our friend Picasso to join the fun

the theme today is people first but we

the theme today is people first but we had a whole lot of fun

had a whole lot of fun

had a whole lot of fun developing this technology over the past

developing this technology over the past

developing this technology over the past years and putting together these demos

years and putting together these demos

years and putting together these demos and I think it shows that you know

and I think it shows that you know

and I think it shows that you know sometimes the most satisfying endeavor

sometimes the most satisfying endeavor

sometimes the most satisfying endeavor is when you really think about putting

is when you really think about putting

is when you really think about putting the children first thank you very much

the children first thank you very much

the children first thank you very much [Applause]

[Applause]

[Applause] [Music]