[Applause] so hard digital world imposes demands on
so hard digital world imposes demands on
so hard digital world imposes demands on us that are very different from the kind
us that are very different from the kind
us that are very different from the kind of context in which our brain evolved
of context in which our brain evolved
of context in which our brain evolved and this difference has serious
and this difference has serious
and this difference has serious implications for how effectively and
implications for how effectively and
implications for how effectively and efficiently we’re able to perform tasks
efficiently we’re able to perform tasks
efficiently we’re able to perform tasks in the real world so let’s make this
in the real world so let’s make this
in the real world so let’s make this idea concrete by exploring an example of
idea concrete by exploring an example of
idea concrete by exploring an example of how our world has changed and the effect
how our world has changed and the effect
how our world has changed and the effect it has on us we look at the evolution of
it has on us we look at the evolution of
it has on us we look at the evolution of the road trip so for many many thousands
the road trip so for many many thousands
the road trip so for many many thousands of years
of years
of years we traveled relatively slowly we either
we traveled relatively slowly we either
we traveled relatively slowly we either walk to places or we rode in semi
walk to places or we rode in semi
walk to places or we rode in semi autonomous vehicles like this
autonomous vehicles like this
autonomous vehicles like this horse-drawn carriage and the task of
horse-drawn carriage and the task of
horse-drawn carriage and the task of getting this carriage from one place to
getting this carriage from one place to
getting this carriage from one place to another was fairly straightforward you
another was fairly straightforward you
another was fairly straightforward you kept the horse pointing in the right
kept the horse pointing in the right
kept the horse pointing in the right direction looked out for robbers made
direction looked out for robbers made
direction looked out for robbers made sure you didn’t run into obstacles there
sure you didn’t run into obstacles there
sure you didn’t run into obstacles there were very few intense processing
were very few intense processing
were very few intense processing requirements that the way vehicle would
requirements that the way vehicle would
requirements that the way vehicle would demand of the of the driver you caught
demand of the of the driver you caught
demand of the of the driver you caught up with news when you got to your
up with news when you got to your
up with news when you got to your destination and the only communication
destination and the only communication
destination and the only communication you engaged in was with people in your
you engaged in was with people in your
you engaged in was with people in your immediate proximity so fast forward to
immediate proximity so fast forward to
immediate proximity so fast forward to the 1960s and you see the automobile
the 1960s and you see the automobile
the 1960s and you see the automobile firmly entrenched in our lives and in
firmly entrenched in our lives and in
firmly entrenched in our lives and in our culture but we get to places faster
our culture but we get to places faster
our culture but we get to places faster but our roles as a driver has changed
but our roles as a driver has changed
but our roles as a driver has changed you now control a high-velocity
you now control a high-velocity
you now control a high-velocity projectile and often you’re driving in a
projectile and often you’re driving in a
projectile and often you’re driving in a projectile stream so the consequences in
projectile stream so the consequences in
projectile stream so the consequences in this context of a moment of inattention
this context of a moment of inattention
this context of a moment of inattention or distraction are severe but in many
or distraction are severe but in many
or distraction are severe but in many ways if you can just focus on the task
ways if you can just focus on the task
ways if you can just focus on the task of driving the vehicle imposes very few
of driving the vehicle imposes very few
of driving the vehicle imposes very few additional demands on you you have a
additional demands on you you have a
additional demands on you you have a passive AM radio that’ll break the
passive AM radio that’ll break the
passive AM radio that’ll break the monotony of your drive and
monotony of your drive and
monotony of your drive and communications occur when you pull into
communications occur when you pull into
communications occur when you pull into town and that’s where you catch up with
town and that’s where you catch up with
town and that’s where you catch up with your news it’s just you and the road
your news it’s just you and the road
your news it’s just you and the road fast forward to today and you’re driving
fast forward to today and you’re driving
fast forward to today and you’re driving a vehicle with an active collision
a vehicle with an active collision
a vehicle with an active collision avoidance system lasers in the front and
avoidance system lasers in the front and
avoidance system lasers in the front and on the side an engine monitoring system
on the side an engine monitoring system
on the side an engine monitoring system you’ve got satellite navigation onboard
you’ve got satellite navigation onboard
you’ve got satellite navigation onboard entertainment multi-channel
entertainment multi-channel
entertainment multi-channel communications now you could be forgiven
communications now you could be forgiven
communications now you could be forgiven for thinking I was describing the
for thinking I was describing the
for thinking I was describing the systems on a Boeing 787 or Airbus 350
systems on a Boeing 787 or Airbus 350
systems on a Boeing 787 or Airbus 350 that are operated by a crew of two that
that are operated by a crew of two that
that are operated by a crew of two that are highly trained but I’m talking about
are highly trained but I’m talking about
are highly trained but I’m talking about mom and dad
mom and dad
mom and dad operating the family car and as if this
operating the family car and as if this
operating the family car and as if this wasn’t demanding enough we have these
wasn’t demanding enough we have these
wasn’t demanding enough we have these auxilary devices that we bring into the
auxilary devices that we bring into the
auxilary devices that we bring into the vehicle that are completely
vehicle that are completely
vehicle that are completely unconstrained and introduced a thousand
unconstrained and introduced a thousand
unconstrained and introduced a thousand distractions and if you let yourself do
distractions and if you let yourself do
distractions and if you let yourself do it you can take every single available
it you can take every single available
it you can take every single available cognitive cycle and apply it to this
cognitive cycle and apply it to this
cognitive cycle and apply it to this task now the problem is we really can’t
task now the problem is we really can’t
task now the problem is we really can’t get away from the effects of this
get away from the effects of this
get away from the effects of this technology no matter what we try to do
technology no matter what we try to do
technology no matter what we try to do as this hapless man is trying so the
as this hapless man is trying so the
as this hapless man is trying so the problem is research has shown that even
problem is research has shown that even
problem is research has shown that even the mere awareness of a text
the mere awareness of a text
the mere awareness of a text notification can have a serious impact
notification can have a serious impact
notification can have a serious impact on your task performance even a simple
on your task performance even a simple
on your task performance even a simple laboratory tasks so why is all this an
laboratory tasks so why is all this an
laboratory tasks so why is all this an issue I mean the the reason this is a
issue I mean the the reason this is a
issue I mean the the reason this is a problem is that the the environment
problem is that the the environment
problem is that the the environment around us is imposing demands that are
around us is imposing demands that are
around us is imposing demands that are really reaching the limits of our
really reaching the limits of our
really reaching the limits of our cognitive capabilities so if you
cognitive capabilities so if you
cognitive capabilities so if you consider our working memory system for
consider our working memory system for
consider our working memory system for instance we can handle about seven plus
instance we can handle about seven plus
instance we can handle about seven plus or minus two units of information our
or minus two units of information our
or minus two units of information our attentional systems which are necessary
attentional systems which are necessary
attentional systems which are necessary for taking our cognitive system and
for taking our cognitive system and
for taking our cognitive system and applying it focused on particular
applying it focused on particular
applying it focused on particular aspects of the task environment are now
aspects of the task environment are now
aspects of the task environment are now splitting attention by so many different
splitting attention by so many different
splitting attention by so many different concurrent demands so the question for
concurrent demands so the question for
concurrent demands so the question for all of us is how do we keep up and this
all of us is how do we keep up and this
all of us is how do we keep up and this is where our research at Honeywell comes
is where our research at Honeywell comes
is where our research at Honeywell comes in we’ve tried to address this issue
in we’ve tried to address this issue
in we’ve tried to address this issue with three broad thrusts of research the
with three broad thrusts of research the
with three broad thrusts of research the first is creating systems that are
first is creating systems that are
first is creating systems that are sensitive to cognitive state so these
sensitive to cognitive state so these
sensitive to cognitive state so these are systems that like a polite office
are systems that like a polite office
are systems that like a polite office co-worker who first checks to see
co-worker who first checks to see
co-worker who first checks to see whether you’re on the phone and you are
whether you’re on the phone and you are
whether you’re on the phone and you are engaged in a busy task before engaging
engaged in a busy task before engaging
engaged in a busy task before engaging in conversation is able to modulate its
in conversation is able to modulate its
in conversation is able to modulate its interaction with you depending on what’s
interaction with you depending on what’s
interaction with you depending on what’s going on in your head the second thrust
going on in your head the second thrust
going on in your head the second thrust of research is to come up with novel
of research is to come up with novel
of research is to come up with novel ways of interacting with computer
ways of interacting with computer
ways of interacting with computer systems and the idea here is to come up
systems and the idea here is to come up
systems and the idea here is to come up with
with
with ways of interaction that are more
ways of interaction that are more
ways of interaction that are more natural perhaps less effortful that you
natural perhaps less effortful that you
natural perhaps less effortful that you could then apply those comfort of cycles
could then apply those comfort of cycles
could then apply those comfort of cycles to other tasks or to handle more and the
to other tasks or to handle more and the
to other tasks or to handle more and the third thrust of research is enhancing
third thrust of research is enhancing
third thrust of research is enhancing cognitive capacity so that we’re able to
cognitive capacity so that we’re able to
cognitive capacity so that we’re able to take on much much more so let’s tackle
take on much much more so let’s tackle
take on much much more so let’s tackle the first thrust here of creating
the first thrust here of creating
the first thrust here of creating systems that are sensitive to cognitive
systems that are sensitive to cognitive
systems that are sensitive to cognitive state now in the example that I
state now in the example that I
state now in the example that I described a moment ago of somebody
described a moment ago of somebody
described a moment ago of somebody coming into your office and being able
coming into your office and being able
coming into your office and being able to peer over your shoulder and figure
to peer over your shoulder and figure
to peer over your shoulder and figure out what’s going on
out what’s going on
out what’s going on that was enabled by an inference that a
that was enabled by an inference that a
that was enabled by an inference that a person could make now what we’re trying
person could make now what we’re trying
person could make now what we’re trying to do is to give a computer the
to do is to give a computer the
to do is to give a computer the capability of making just such an
capability of making just such an
capability of making just such an inference and one way of doing it is to
inference and one way of doing it is to
inference and one way of doing it is to look at neural activity which can
look at neural activity which can
look at neural activity which can provide us with a lot of insight into
provide us with a lot of insight into
provide us with a lot of insight into how hard the brain is working so I’ll
how hard the brain is working so I’ll
how hard the brain is working so I’ll give you a couple of examples here you
give you a couple of examples here you
give you a couple of examples here you have two images of a brain performing an
have two images of a brain performing an
have two images of a brain performing an easy task on the left and a harder
easy task on the left and a harder
easy task on the left and a harder version of the task on the right same
version of the task on the right same
version of the task on the right same task and what’s happening here on the
task and what’s happening here on the
task and what’s happening here on the image on the right is that you see much
image on the right is that you see much
image on the right is that you see much broader crota cortical recruitment and
broader crota cortical recruitment and
broader crota cortical recruitment and higher amplitude of activity you can see
higher amplitude of activity you can see
higher amplitude of activity you can see similar effects when you’re looking at
similar effects when you’re looking at
similar effects when you’re looking at evolving expertise so with the image on
evolving expertise so with the image on
evolving expertise so with the image on the bottom if you go left to right you
the bottom if you go left to right you
the bottom if you go left to right you can see somebody learning a motor
can see somebody learning a motor
can see somebody learning a motor tracking tasks early on in the skill
tracking tasks early on in the skill
tracking tasks early on in the skill acquisition process where performance is
acquisition process where performance is
acquisition process where performance is effortful you can see a lot more areas
effortful you can see a lot more areas
effortful you can see a lot more areas of the brain are lit up so it’s not just
of the brain are lit up so it’s not just
of the brain are lit up so it’s not just the regions that are essential for
the regions that are essential for
the regions that are essential for performing the tasks you have a whole
performing the tasks you have a whole
performing the tasks you have a whole bunch of supporting functions that are
bunch of supporting functions that are
bunch of supporting functions that are concurrently active but as the person
concurrently active but as the person
concurrently active but as the person gains proficiency and the task is almost
gains proficiency and the task is almost
gains proficiency and the task is almost automatic and effortless you just see
automatic and effortless you just see
automatic and effortless you just see the parts of the brain that were
the parts of the brain that were
the parts of the brain that were responsible for the automatic execution
responsible for the automatic execution
responsible for the automatic execution of the task so brain activity and
of the task so brain activity and
of the task so brain activity and measurements of brain activity can tell
measurements of brain activity can tell
measurements of brain activity can tell us a lot about underlying cognitive
us a lot about underlying cognitive
us a lot about underlying cognitive effort but unfortunately these images
effort but unfortunately these images
effort but unfortunately these images were gathered using devices like this a
were gathered using devices like this a
were gathered using devices like this a magnetic resonance imaging tool which is
magnetic resonance imaging tool which is
magnetic resonance imaging tool which is excellent for laboratory research but
excellent for laboratory research but
excellent for laboratory research but impractical for use in the kind of
impractical for use in the kind of
impractical for use in the kind of context that we’ve been talking about
context that we’ve been talking about
context that we’ve been talking about which is why our efforts have focused on
which is why our efforts have focused on
which is why our efforts have focused on EEG so here’s how you G systems work
EEG so here’s how you G systems work
EEG so here’s how you G systems work when we are performing tasks our brain
when we are performing tasks our brain
when we are performing tasks our brain engages billions of neurons and as these
engages billions of neurons and as these
engages billions of neurons and as these neurons fire in a synchronous way the
neurons fire in a synchronous way the
neurons fire in a synchronous way the generate electric fields and you can
generate electric fields and you can
generate electric fields and you can sense these electric fields on the
sense these electric fields on the
sense these electric fields on the surface of the head as small voltage
surface of the head as small voltage
surface of the head as small voltage fluctuations and you can read these
fluctuations and you can read these
fluctuations and you can read these voltage fluctuations and make inferences
voltage fluctuations and make inferences
voltage fluctuations and make inferences about all kinds of processes that are
about all kinds of processes that are
about all kinds of processes that are happening in the brain reasoning
happening in the brain reasoning
happening in the brain reasoning processes motor processes attention etc
processes motor processes attention etc
processes motor processes attention etc just to name a few now the practicality
just to name a few now the practicality
just to name a few now the practicality of the sensors used to make these kind
of the sensors used to make these kind
of the sensors used to make these kind of inferences are changing all the time
of inferences are changing all the time
of inferences are changing all the time they’re coming down and cost becoming
they’re coming down and cost becoming
they’re coming down and cost becoming much more practical and this is a
much more practical and this is a
much more practical and this is a commercially available headset pictured
commercially available headset pictured
commercially available headset pictured in this image here in the near future
in this image here in the near future
in this image here in the near future they’ll be in your headphones
they’ll be in your headphones
they’ll be in your headphones they’ll be in your eyewear in caps
they’ll be in your eyewear in caps
they’ll be in your eyewear in caps helmets etc so our focus has not been on
helmets etc so our focus has not been on
helmets etc so our focus has not been on hardware it’s been on developing
hardware it’s been on developing
hardware it’s been on developing software components that work on the
software components that work on the
software components that work on the data coming out of these hardware
data coming out of these hardware
data coming out of these hardware systems our focus is on building
systems our focus is on building
systems our focus is on building software that can reliably work with the
software that can reliably work with the
software that can reliably work with the data coming in to make inferences about
data coming in to make inferences about
data coming in to make inferences about cognitive state and so we clean up the
cognitive state and so we clean up the
cognitive state and so we clean up the signals as they come from the sources
signals as they come from the sources
signals as they come from the sources from these Hardware sources using
from these Hardware sources using
from these Hardware sources using digital signal processing techniques we
digital signal processing techniques we
digital signal processing techniques we then implement dimensionality reduction
then implement dimensionality reduction
then implement dimensionality reduction techniques that identify features of the
techniques that identify features of the
techniques that identify features of the data that are most informative with
data that are most informative with
data that are most informative with respect to the estimation that you’re
respect to the estimation that you’re
respect to the estimation that you’re trying to make and finally we use
trying to make and finally we use
trying to make and finally we use machine learning models that are
machine learning models that are
machine learning models that are individualized to be able to look at
individualized to be able to look at
individualized to be able to look at your brain and identify the features
your brain and identify the features
your brain and identify the features that allow us to estimate whether you’re
that allow us to estimate whether you’re
that allow us to estimate whether you’re in a high cognitive load situation or
in a high cognitive load situation or
in a high cognitive load situation or low cognitive load situation high
low cognitive load situation high
low cognitive load situation high attention low attention etc now we’ve
attention low attention etc now we’ve
attention low attention etc now we’ve validated this kind of an approach in a
validated this kind of an approach in a
validated this kind of an approach in a variety of different contexts our
variety of different contexts our
variety of different contexts our efforts began with the DARPA funded
efforts began with the DARPA funded
efforts began with the DARPA funded augmented cognition program where we
augmented cognition program where we
augmented cognition program where we worked with a group of industry and
worked with a group of industry and
worked with a group of industry and academic researchers to address this
academic researchers to address this
academic researchers to address this problem that the government is facing
problem that the government is facing
problem that the government is facing the military of being able to tell when
the military of being able to tell when
the military of being able to tell when soldiers whose job is supported by all
soldiers whose job is supported by all
soldiers whose job is supported by all kinds of electronic communication
kinds of electronic communication
kinds of electronic communication devices were interrupted so not only
devices were interrupted so not only
devices were interrupted so not only were we able to demonstrate this at
were we able to demonstrate this at
were we able to demonstrate this at Aberdeen Proving Grounds in a high
Aberdeen Proving Grounds in a high
Aberdeen Proving Grounds in a high fidelity training exercise we were also
fidelity training exercise we were also
fidelity training exercise we were also able to show that cognitive state
able to show that cognitive state
able to show that cognitive state estimation in real-world contexts
estimation in real-world contexts
estimation in real-world contexts outside the laboratory was feasible
outside the laboratory was feasible
outside the laboratory was feasible but one of the things about this
but one of the things about this
but one of the things about this environment is that the extremes of
environment is that the extremes of
environment is that the extremes of workload that you experience at the
workload that you experience at the
workload that you experience at the battlefield
battlefield
battlefield aren’t representative of the kind of
aren’t representative of the kind of
aren’t representative of the kind of variation you might have in everyday
variation you might have in everyday
variation you might have in everyday context and so since this effort we’ve
context and so since this effort we’ve
context and so since this effort we’ve looked at trying to classify or estimate
looked at trying to classify or estimate
looked at trying to classify or estimate variations in cognitive load that are
variations in cognitive load that are
variations in cognitive load that are less pronounced like corporate pilots
less pronounced like corporate pilots
less pronounced like corporate pilots being able to control an aircraft and
being able to control an aircraft and
being able to control an aircraft and cruise versus an approach to landing or
cruise versus an approach to landing or
cruise versus an approach to landing or helicopter pilots trying to land a
helicopter pilots trying to land a
helicopter pilots trying to land a helicopter under varying conditions of
helicopter under varying conditions of
helicopter under varying conditions of difficulty even the difference between
difficulty even the difference between
difficulty even the difference between someone reading a difficult piece of
someone reading a difficult piece of
someone reading a difficult piece of text versus an easy piece of text and so
text versus an easy piece of text and so
text versus an easy piece of text and so what this research along with research
what this research along with research
what this research along with research to be able to estimate people’s
to be able to estimate people’s
to be able to estimate people’s attention allows us to do is to create
attention allows us to do is to create
attention allows us to do is to create systems that are more sensitive to human
systems that are more sensitive to human
systems that are more sensitive to human cognitive states so instead of a drill
cognitive states so instead of a drill
cognitive states so instead of a drill sergeant
sergeant
sergeant barking information and expecting the
barking information and expecting the
barking information and expecting the user to keep up we are creating systems
user to keep up we are creating systems
user to keep up we are creating systems that are like a cooperative passenger
that are like a cooperative passenger
that are like a cooperative passenger who sees that you’re engaged while
who sees that you’re engaged while
who sees that you’re engaged while you’re passing a truck and suppresses
you’re passing a truck and suppresses
you’re passing a truck and suppresses conversation and when you begin to fade
conversation and when you begin to fade
conversation and when you begin to fade on a long drive they engage in lively
on a long drive they engage in lively
on a long drive they engage in lively conversation and more engaging music to
conversation and more engaging music to
conversation and more engaging music to keep your performance in a zone where
keep your performance in a zone where
keep your performance in a zone where you’re able to perform effectively so
you’re able to perform effectively so
you’re able to perform effectively so our second thrust of research that I’d
our second thrust of research that I’d
our second thrust of research that I’d like to discuss is to come up with novel
like to discuss is to come up with novel
like to discuss is to come up with novel means of human-computer interaction
means of human-computer interaction
means of human-computer interaction using the neuro technology that are more
using the neuro technology that are more
using the neuro technology that are more natural and require less effort of the
natural and require less effort of the
natural and require less effort of the of the user and I’ll present two
of the user and I’ll present two
of the user and I’ll present two examples the first is of an image
examples the first is of an image
examples the first is of an image processing application and the second is
processing application and the second is
processing application and the second is of a brain computer interface that’s
of a brain computer interface that’s
of a brain computer interface that’s used to control an airplane so the first
used to control an airplane so the first
used to control an airplane so the first of these application is addresses the
of these application is addresses the
of these application is addresses the problem that we all face when you try to
problem that we all face when you try to
problem that we all face when you try to look for a specific image in the
look for a specific image in the
look for a specific image in the collection of photos you have on your
collection of photos you have on your
collection of photos you have on your home computer it’s a difficult task it’s
home computer it’s a difficult task it’s
home computer it’s a difficult task it’s difficult especially if you don’t have a
difficult especially if you don’t have a
difficult especially if you don’t have a lot of time and this is the kind of
lot of time and this is the kind of
lot of time and this is the kind of problem that image and analysts in a
problem that image and analysts in a
problem that image and analysts in a variety of domains medical image
variety of domains medical image
variety of domains medical image analysts geospatial image analysts face
analysts geospatial image analysts face
analysts geospatial image analysts face all the time now the problem is that the
all the time now the problem is that the
all the time now the problem is that the tools these analysts use are designed
tools these analysts use are designed
tools these analysts use are designed for slow and deliberate processing you
for slow and deliberate processing you
for slow and deliberate processing you bring up large images and search them
bring up large images and search them
bring up large images and search them systematically to find the tumor or some
systematically to find the tumor or some
systematically to find the tumor or some kind of a target that’s out there
kind of a target that’s out there
kind of a target that’s out there but what we were able to do every day
but what we were able to do every day
but what we were able to do every day almost automatically is to make robust
almost automatically is to make robust
almost automatically is to make robust split-second perceptual judgments around
split-second perceptual judgments around
split-second perceptual judgments around us like the kind of processes that we
us like the kind of processes that we
us like the kind of processes that we invoke when you detect and return a
invoke when you detect and return a
invoke when you detect and return a tennis serve or hit respond to a
tennis serve or hit respond to a
tennis serve or hit respond to a baseball pitch or take an evasive
baseball pitch or take an evasive
baseball pitch or take an evasive maneuver to avoid a absent-minded
maneuver to avoid a absent-minded
maneuver to avoid a absent-minded squirrel along your path so all of these
squirrel along your path so all of these
squirrel along your path so all of these things happen automatically and we’re
things happen automatically and we’re
things happen automatically and we’re interested in being able to identify the
interested in being able to identify the
interested in being able to identify the neural signatures associated with these
neural signatures associated with these
neural signatures associated with these kinds of events and one signature in
kinds of events and one signature in
kinds of events and one signature in particular is the event related
particular is the event related
particular is the event related potential it’s a brain Zaha signal I’ve
potential it’s a brain Zaha signal I’ve
potential it’s a brain Zaha signal I’ve seen something of interest here now do
seen something of interest here now do
seen something of interest here now do something about it and so the way this
something about it and so the way this
something about it and so the way this works in an image analysis application
works in an image analysis application
works in an image analysis application is we take large satellite images that
is we take large satellite images that
is we take large satellite images that are tens of thousands of pictures wide
are tens of thousands of pictures wide
are tens of thousands of pictures wide and tall and break them into chips that
and tall and break them into chips that
and tall and break them into chips that we present at high speed rates you know
we present at high speed rates you know
we present at high speed rates you know bursts of imagery flowing past tube and
bursts of imagery flowing past tube and
bursts of imagery flowing past tube and as these images are being screened by
as these images are being screened by
as these images are being screened by the user we look at neural activity that
the user we look at neural activity that
the user we look at neural activity that follows each image just a split-second
follows each image just a split-second
follows each image just a split-second of neural data to identify the event
of neural data to identify the event
of neural data to identify the event related potential signal that I just
related potential signal that I just
related potential signal that I just described and what you could do then is
described and what you could do then is
described and what you could do then is create a probability map of areas that
create a probability map of areas that
create a probability map of areas that grab the analyst attention and you can
grab the analyst attention and you can
grab the analyst attention and you can take the probability map overlaid on the
take the probability map overlaid on the
take the probability map overlaid on the original image and you get a bunch of
original image and you get a bunch of
original image and you get a bunch of hot spots with the target is most likely
hot spots with the target is most likely
hot spots with the target is most likely so instead of slow deliberate search of
so instead of slow deliberate search of
so instead of slow deliberate search of this image that would have taken tens of
this image that would have taken tens of
this image that would have taken tens of minutes you’re able to reduce it to a
minutes you’re able to reduce it to a
minutes you’re able to reduce it to a high speed scan that just takes about a
high speed scan that just takes about a
high speed scan that just takes about a minute or two so turning the human into
minute or two so turning the human into
minute or two so turning the human into this high speed sensor and bypassing our
this high speed sensor and bypassing our
this high speed sensor and bypassing our slow and deliberate ways of interacting
slow and deliberate ways of interacting
slow and deliberate ways of interacting with these systems now another
with these systems now another
with these systems now another application that we’ve used with the
application that we’ve used with the
application that we’ve used with the event-related potential signal is to
event-related potential signal is to
event-related potential signal is to have explore its use as a potential way
have explore its use as a potential way
have explore its use as a potential way to interact with computer systems so
to interact with computer systems so
to interact with computer systems so what you’ll see in this video is a
what you’ll see in this video is a
what you’ll see in this video is a journalist from Wired Jack Stewart
journalist from Wired Jack Stewart
journalist from Wired Jack Stewart controlling a multi-engine corporate
controlling a multi-engine corporate
controlling a multi-engine corporate aircraft using the event-related
aircraft using the event-related
aircraft using the event-related potential so in front of Jack is a
potential so in front of Jack is a
potential so in front of Jack is a display that has a variety of commands
display that has a variety of commands
display that has a variety of commands and these commands are highlighted at
and these commands are highlighted at
and these commands are highlighted at random and all he has to do to control
random and all he has to do to control
random and all he has to do to control it is to glance at a command of interest
it is to glance at a command of interest
it is to glance at a command of interest and make a mental note when it
and make a mental note when it
and make a mental note when it highlights and that generates the event
highlights and that generates the event
highlights and that generates the event related potential that we can detect
related potential that we can detect
related potential that we can detect infer intent and the next
infer intent and the next
infer intent and the next so let’s watch jack control the airplane
so let’s watch jack control the airplane
so let’s watch jack control the airplane using neural signals all right Jack
using neural signals all right Jack
using neural signals all right Jack you’re flying this thing Oh No where
you’re flying this thing Oh No where
you’re flying this thing Oh No where would you like me to point how about a
would you like me to point how about a
would you like me to point how about a right turn right frankly we spend the
right turn right frankly we spend the
right turn right frankly we spend the next 30 minutes going through some
next 30 minutes going through some
next 30 minutes going through some maneuvers until I’m completely convinced
maneuvers until I’m completely convinced
maneuvers until I’m completely convinced that system work so you could learn more
that system work so you could learn more
that system work so you could learn more about this flight by going to wire comm
about this flight by going to wire comm
about this flight by going to wire comm it’s on YouTube as well but Jack flies
it’s on YouTube as well but Jack flies
it’s on YouTube as well but Jack flies around Seattle using this brain computer
around Seattle using this brain computer
around Seattle using this brain computer interface with minimal training so we’ve
interface with minimal training so we’ve
interface with minimal training so we’ve looked at two avenues of enhancing human
looked at two avenues of enhancing human
looked at two avenues of enhancing human performance and augmenting it in a way
performance and augmenting it in a way
performance and augmenting it in a way that people can keep up with all the
that people can keep up with all the
that people can keep up with all the demands that are being thrown at them
demands that are being thrown at them
demands that are being thrown at them the last trust of research that I could
the last trust of research that I could
the last trust of research that I could talk about is to enhance cognitive
talk about is to enhance cognitive
talk about is to enhance cognitive capacity and so what we’ve been doing is
capacity and so what we’ve been doing is
capacity and so what we’ve been doing is to try to enhance people’s reasoning and
to try to enhance people’s reasoning and
to try to enhance people’s reasoning and problem-solving skills which is tied to
problem-solving skills which is tied to
problem-solving skills which is tied to a construct known as fluid intelligence
a construct known as fluid intelligence
a construct known as fluid intelligence which you measure through IQ tests and
which you measure through IQ tests and
which you measure through IQ tests and the way we’ve done it is to try to
the way we’ve done it is to try to
the way we’ve done it is to try to identify the cognitive processes
identify the cognitive processes
identify the cognitive processes associated with fluid intelligence
associated with fluid intelligence
associated with fluid intelligence it’s a brain’s executive function system
it’s a brain’s executive function system
it’s a brain’s executive function system and so we target the executive function
and so we target the executive function
and so we target the executive function system in two ways one is to build a
system in two ways one is to build a
system in two ways one is to build a cognitive training paradigm in the form
cognitive training paradigm in the form
cognitive training paradigm in the form of a game that exercises the components
of a game that exercises the components
of a game that exercises the components of the executive function system and
of the executive function system and
of the executive function system and what we do in the game is not only vary
what we do in the game is not only vary
what we do in the game is not only vary the complexity of the game but also
the complexity of the game but also
the complexity of the game but also variety which is crucial for the
variety which is crucial for the
variety which is crucial for the training to transfer to real-world
training to transfer to real-world
training to transfer to real-world contexts the other thing we do as a
contexts the other thing we do as a
contexts the other thing we do as a person is playing this game is to
person is playing this game is to
person is playing this game is to stimulate the cortical substrate that
stimulate the cortical substrate that
stimulate the cortical substrate that supports these kind of tasks using mild
supports these kind of tasks using mild
supports these kind of tasks using mild electrical stimulation and we’ve then
electrical stimulation and we’ve then
electrical stimulation and we’ve then evaluated this combination in
evaluated this combination in
evaluated this combination in large-scale studies with collaborators
large-scale studies with collaborators
large-scale studies with collaborators at Harvard Medical School Northeastern
at Harvard Medical School Northeastern
at Harvard Medical School Northeastern University Oxford University using a
University Oxford University using a
University Oxford University using a game that was built with our
game that was built with our
game that was built with our collaborators at sim coach games and
collaborators at sim coach games and
collaborators at sim coach games and what we’ve shown in independent
what we’ve shown in independent
what we’ve shown in independent assessments administered by the US
assessments administered by the US
assessments administered by the US government is that three weeks of
government is that three weeks of
government is that three weeks of training 30 minutes with this game and
training 30 minutes with this game and
training 30 minutes with this game and the combination of electrical
the combination of electrical
the combination of electrical stimulation can produce Cystic reliable
stimulation can produce Cystic reliable
stimulation can produce Cystic reliable improvements in IQ scores
improvements in IQ scores
improvements in IQ scores so we have a large-scale validation
so we have a large-scale validation
so we have a large-scale validation underway now to try to replicate the
underway now to try to replicate the
underway now to try to replicate the results that we obtained with this study
results that we obtained with this study
results that we obtained with this study so I’ve shown you a few different
so I’ve shown you a few different
so I’ve shown you a few different avenues that we’re exploring in our lab
avenues that we’re exploring in our lab
avenues that we’re exploring in our lab to try to augment human performance to
to try to augment human performance to
to try to augment human performance to keep up with the demands that we
keep up with the demands that we
keep up with the demands that we surrounded with now each of these areas
surrounded with now each of these areas
surrounded with now each of these areas of research is at its infancy but
of research is at its infancy but
of research is at its infancy but hopefully it provided you with a glimpse
hopefully it provided you with a glimpse
hopefully it provided you with a glimpse of what the future might be like to help
of what the future might be like to help
of what the future might be like to help us all scale up to keep up with the
us all scale up to keep up with the
us all scale up to keep up with the increasing demands of the world
increasing demands of the world
increasing demands of the world thank you
thank you
thank you [Applause]
”
Be First to Comment