Press "Enter" to skip to content

Why We Need A New, Algorithmic Social Contract | Iyad Rahwan | TEDxCambridgeSalon


[Applause]

[Applause] today I want to talk about something

today I want to talk about something

today I want to talk about something that’s been in the news almost every day

that’s been in the news almost every day

that’s been in the news almost every day probably multiple times and it’s

probably multiple times and it’s

probably multiple times and it’s artificial intelligence or AI so I’m

artificial intelligence or AI so I’m

artificial intelligence or AI so I’m going to use this little cartoon that I

going to use this little cartoon that I

going to use this little cartoon that I drew of a little robot I’m going to use

drew of a little robot I’m going to use

drew of a little robot I’m going to use this guy to depict an AI which would be

this guy to depict an AI which would be

this guy to depict an AI which would be a robot or a driverless car or just a an

a robot or a driverless car or just a an

a robot or a driverless car or just a an algorithm running somewhere allocating

algorithm running somewhere allocating

algorithm running somewhere allocating resources and I’m going to ask the

resources and I’m going to ask the

resources and I’m going to ask the question of how do we handle the

question of how do we handle the

question of how do we handle the challenges that come with this

challenges that come with this

challenges that come with this technology so on one hand a AI promises

technology so on one hand a AI promises

technology so on one hand a AI promises many benefits from better

many benefits from better

many benefits from better recommendations for restaurants and

recommendations for restaurants and

recommendations for restaurants and movies and even romantic partners to

movies and even romantic partners to

movies and even romantic partners to safer cars to more accurate medical

safer cars to more accurate medical

safer cars to more accurate medical diagnosis and a whole bunch of other

diagnosis and a whole bunch of other

diagnosis and a whole bunch of other benefits but I also raises many

benefits but I also raises many

benefits but I also raises many challenges that people are talking about

challenges that people are talking about

challenges that people are talking about very frequently in the media from

very frequently in the media from

very frequently in the media from algorithms that filter news putting us

algorithms that filter news putting us

algorithms that filter news putting us in filter bubbles to unfair matching in

in filter bubbles to unfair matching in

in filter bubbles to unfair matching in markets to mass unemployment of course

markets to mass unemployment of course

markets to mass unemployment of course and a whole bunch of other challenges so

and a whole bunch of other challenges so

and a whole bunch of other challenges so how do we balance the benefits and the

how do we balance the benefits and the

how do we balance the benefits and the risks now one idea that is commonly

risks now one idea that is commonly

risks now one idea that is commonly discussed is that we always need a human

discussed is that we always need a human

discussed is that we always need a human in the loop of these systems so it would

in the loop of these systems so it would

in the loop of these systems so it would look something like this

look something like this

look something like this you’ve got an AI that is performing a

you’ve got an AI that is performing a

you’ve got an AI that is performing a function let’s say it’s flying a plane

function let’s say it’s flying a plane

function let’s say it’s flying a plane you always need to put a human in the

you always need to put a human in the

you always need to put a human in the loop like a captain in the cockpit and

loop like a captain in the cockpit and

loop like a captain in the cockpit and this person would ensure that the system

this person would ensure that the system

this person would ensure that the system behaves in accordance with our plans and

behaves in accordance with our plans and

behaves in accordance with our plans and desires and also is able to intervene in

desires and also is able to intervene in

desires and also is able to intervene in case something goes wrong and maybe more

case something goes wrong and maybe more

case something goes wrong and maybe more importantly we have somebody to blame

importantly we have somebody to blame

importantly we have somebody to blame something goes wrong now an interesting

something goes wrong now an interesting

something goes wrong now an interesting thing to notice about this picture is

thing to notice about this picture is

thing to notice about this picture is that everybody here has the same goal

that everybody here has the same goal

that everybody here has the same goal so most passengers I would hope would

so most passengers I would hope would

so most passengers I would hope would require getting from A to B safely maybe

require getting from A to B safely maybe

require getting from A to B safely maybe there are exceptions but in many cases

there are exceptions but in many cases

there are exceptions but in many cases our systems with

our systems with

our systems with perform functions in which there are

perform functions in which there are

perform functions in which there are multiple goals and these goals may be in

multiple goals and these goals may be in

multiple goals and these goals may be in conflict so somebody may be interested

conflict so somebody may be interested

conflict so somebody may be interested in fairness so another person cares more

in fairness so another person cares more

in fairness so another person cares more about efficiency and yet another person

about efficiency and yet another person

about efficiency and yet another person cares more about safety and these goals

cares more about safety and these goals

cares more about safety and these goals may not be simultaneously achievable for

may not be simultaneously achievable for

may not be simultaneously achievable for whatever reason now so how do we agree

whatever reason now so how do we agree

whatever reason now so how do we agree on these goals first before we tell the

on these goals first before we tell the

on these goals first before we tell the Machine what to do and have the

Machine what to do and have the

Machine what to do and have the human-in-the-loop intervene so this is

human-in-the-loop intervene so this is

human-in-the-loop intervene so this is less like a human in the loop challenge

less like a human in the loop challenge

less like a human in the loop challenge and more like a society in the loop

and more like a society in the loop

and more like a society in the loop challenge whereby society in the loop

challenge whereby society in the loop

challenge whereby society in the loop here I mean human in the loop plus a

here I mean human in the loop plus a

here I mean human in the loop plus a social contract between the stakeholders

social contract between the stakeholders

social contract between the stakeholders who care about different things and the

who care about different things and the

who care about different things and the social contract is meant to provide the

social contract is meant to provide the

social contract is meant to provide the agreement in which into which people

agreement in which into which people

agreement in which into which people would opt in in order to cooperate to

would opt in in order to cooperate to

would opt in in order to cooperate to their collective benefit it’s a

their collective benefit it’s a

their collective benefit it’s a necessity and the social contract is not

necessity and the social contract is not

necessity and the social contract is not really a new idea it’s been around for a

really a new idea it’s been around for a

really a new idea it’s been around for a while at least 350 years next year this

while at least 350 years next year this

while at least 350 years next year this is the book Leviathan by political

is the book Leviathan by political

is the book Leviathan by political philosopher and economist Thomas Hobbes

philosopher and economist Thomas Hobbes

philosopher and economist Thomas Hobbes and Thomas Hobbes is innovation was to

and Thomas Hobbes is innovation was to

and Thomas Hobbes is innovation was to identify this notion of legitimacy of

identify this notion of legitimacy of

identify this notion of legitimacy of the sovereign power that doesn’t come

the sovereign power that doesn’t come

the sovereign power that doesn’t come from above it doesn’t come out of

from above it doesn’t come out of

from above it doesn’t come out of nothing but it comes out of the mutual

nothing but it comes out of the mutual

nothing but it comes out of the mutual consent of those who are governed so if

consent of those who are governed so if

consent of those who are governed so if you zoom in on this sovereign this

you zoom in on this sovereign this

you zoom in on this sovereign this all-powerful sovereign looking over the

all-powerful sovereign looking over the

all-powerful sovereign looking over the land here you see that it’s made up of

land here you see that it’s made up of

land here you see that it’s made up of hundreds of little people who have opted

hundreds of little people who have opted

hundreds of little people who have opted into the social contract by giving up

into the social contract by giving up

into the social contract by giving up some of their individual rights and

some of their individual rights and

some of their individual rights and freedoms in order to maintain this

freedoms in order to maintain this

freedoms in order to maintain this sovereign power to enforce property

sovereign power to enforce property

sovereign power to enforce property rights as well as other rights and of

rights as well as other rights and of

rights as well as other rights and of course if these people would opt out of

course if these people would opt out of

course if these people would opt out of this contract this very powerful thing

this contract this very powerful thing

this contract this very powerful thing would immediately cease to exist

would immediately cease to exist

would immediately cease to exist now Hobbes made this sovereign very

now Hobbes made this sovereign very

now Hobbes made this sovereign very powerful he said that this sovereign

powerful he said that this sovereign

powerful he said that this sovereign should have almost all power short of

should have almost all power short of

should have almost all power short of taking people’s lives but then we spent

taking people’s lives but then we spent

taking people’s lives but then we spent many decades actually centuries taming

many decades actually centuries taming

many decades actually centuries taming this Leviathan and we did this by

this Leviathan and we did this by

this Leviathan and we did this by introducing many checks and balances on

introducing many checks and balances on

introducing many checks and balances on its power introducing fundamental rights

its power introducing fundamental rights

its power introducing fundamental rights that it cannot

that it cannot

that it cannot violate and a whole bunch of other

violate and a whole bunch of other

violate and a whole bunch of other political developments since then and

political developments since then and

political developments since then and today as we think about technology it’s

today as we think about technology it’s

today as we think about technology it’s important to realize that we’ve created

important to realize that we’ve created

important to realize that we’ve created and maintained and tamed the Leviathan

and maintained and tamed the Leviathan

and maintained and tamed the Leviathan through a social contract so today as we

through a social contract so today as we

through a social contract so today as we build a technology on a kind of cyborg

build a technology on a kind of cyborg

build a technology on a kind of cyborg sovereign that is made up of humans as

sovereign that is made up of humans as

sovereign that is made up of humans as well as algorithms making decisions that

well as algorithms making decisions that

well as algorithms making decisions that impact us we also need a kind of social

impact us we also need a kind of social

impact us we also need a kind of social contract that is also algorithmic and I

contract that is also algorithmic and I

contract that is also algorithmic and I want to talk a little bit about the

want to talk a little bit about the

want to talk a little bit about the nature of this thing now how do and how

nature of this thing now how do and how

nature of this thing now how do and how do we tame it so before we discuss this

do we tame it so before we discuss this

do we tame it so before we discuss this we need to talk about what things

we need to talk about what things

we need to talk about what things regulates and I can’t think of a better

regulates and I can’t think of a better

regulates and I can’t think of a better source of or classification of things

source of or classification of things

source of or classification of things that regulate then Larry Lessig’s

that regulate then Larry Lessig’s

that regulate then Larry Lessig’s regulation architecture so there is well

regulation architecture so there is well

regulation architecture so there is well Larry in the middle he’s being regulated

Larry in the middle he’s being regulated

Larry in the middle he’s being regulated and he’s regulated by four major forces

and he’s regulated by four major forces

and he’s regulated by four major forces this is from his book code and other

this is from his book code and other

this is from his book code and other laws of cyberspace which was very

laws of cyberspace which was very

laws of cyberspace which was very influential in the 90s as the internet

influential in the 90s as the internet

influential in the 90s as the internet was starting so law is one of the main

was starting so law is one of the main

was starting so law is one of the main regulators this is what governments do

regulators this is what governments do

regulators this is what governments do but also social norms markets that set

but also social norms markets that set

but also social norms markets that set prices on different behaviors and

prices on different behaviors and

prices on different behaviors and objects but also architecture and by

objects but also architecture and by

objects but also architecture and by this he means both the physical

this he means both the physical

this he means both the physical architecture the built architecture but

architecture the built architecture but

architecture the built architecture but also the information architecture things

also the information architecture things

also the information architecture things like the design of cyberspace so how is

like the design of cyberspace so how is

like the design of cyberspace so how is algorithmic social contract changing

algorithmic social contract changing

algorithmic social contract changing this picture I think on one hand it’s

this picture I think on one hand it’s

this picture I think on one hand it’s introducing a new kind of actor that

introducing a new kind of actor that

introducing a new kind of actor that requires regulation so we are humans

requires regulation so we are humans

requires regulation so we are humans still but we’re regulating new kinds of

still but we’re regulating new kinds of

still but we’re regulating new kinds of actors so instead of Larry in the middle

actors so instead of Larry in the middle

actors so instead of Larry in the middle here we have our little robot and this

here we have our little robot and this

here we have our little robot and this robot is being regulated by similar

robot is being regulated by similar

robot is being regulated by similar forces this could be for example a bunch

forces this could be for example a bunch

forces this could be for example a bunch of programs trading in an algorithmic

of programs trading in an algorithmic

of programs trading in an algorithmic trading market or it could be a

trading market or it could be a

trading market or it could be a driverless car that is moving around

driverless car that is moving around

driverless car that is moving around urban spaces and so on now on the other

urban spaces and so on now on the other

urban spaces and so on now on the other hand we have now a I driven regulation

hand we have now a I driven regulation

hand we have now a I driven regulation of human behavior as well

of human behavior as well

of human behavior as well cause again going back to this picture

cause again going back to this picture

cause again going back to this picture and putting Larry again in the middle

and putting Larry again in the middle

and putting Larry again in the middle the law is now becoming AI Augmented

the law is now becoming AI Augmented

the law is now becoming AI Augmented predictive policing is telling police

predictive policing is telling police

predictive policing is telling police where two police judges are using

where two police judges are using

where two police judges are using algorithms to inform therefore all

algorithms to inform therefore all

algorithms to inform therefore all decisions and so on the markets are now

decisions and so on the markets are now

decisions and so on the markets are now using algorithms to allocate buyers and

using algorithms to allocate buyers and

using algorithms to allocate buyers and sellers for instance in peer-to-peer

sellers for instance in peer-to-peer

sellers for instance in peer-to-peer markets or ride-sharing markets and the

markets or ride-sharing markets and the

markets or ride-sharing markets and the information architecture is being

information architecture is being

information architecture is being mediated by algorithms as well deciding

mediated by algorithms as well deciding

mediated by algorithms as well deciding what information we see out of the sea

what information we see out of the sea

what information we see out of the sea of of information out there and even

of of information out there and even

of of information out there and even norms are being altered by these

norms are being altered by these

norms are being altered by these algorithms for instance think of dating

algorithms for instance think of dating

algorithms for instance think of dating websites and how they alter the kinds of

websites and how they alter the kinds of

websites and how they alter the kinds of relationships with humans form so I

relationships with humans form so I

relationships with humans form so I think it’s important to think as as we

think it’s important to think as as we

think it’s important to think as as we think about how this picture is changing

think about how this picture is changing

think about how this picture is changing is important to think about what has

is important to think about what has

is important to think about what has remained the same and what has changed

remained the same and what has changed

remained the same and what has changed because I think things that remain the

because I think things that remain the

because I think things that remain the same means mean that we can still use

same means mean that we can still use

same means mean that we can still use some of our old tools to regulate this

some of our old tools to regulate this

some of our old tools to regulate this new environment so one of the things

new environment so one of the things

new environment so one of the things that remains the same is opacity and

that remains the same is opacity and

that remains the same is opacity and think consider for example teaching a

think consider for example teaching a

think consider for example teaching a machine right from wrong like what to do

machine right from wrong like what to do

machine right from wrong like what to do of course you know on one hand we have

of course you know on one hand we have

of course you know on one hand we have difficulty articulating our goals to

difficulty articulating our goals to

difficulty articulating our goals to machines but we’ve always had difficulty

machines but we’ve always had difficulty

machines but we’ve always had difficulty articulating goals to humans we also had

articulating goals to humans we also had

articulating goals to humans we also had difficulty trusting other humans and now

difficulty trusting other humans and now

difficulty trusting other humans and now we have difficulty trusting other

we have difficulty trusting other

we have difficulty trusting other machines to comply with these goals and

machines to comply with these goals and

machines to comply with these goals and constraints and of course even when even

constraints and of course even when even

constraints and of course even when even well-intentioned goals can sometimes go

well-intentioned goals can sometimes go

well-intentioned goals can sometimes go out of control you know you could tell a

out of control you know you could tell a

out of control you know you could tell a robot to minimize misery and it kills

robot to minimize misery and it kills

robot to minimize misery and it kills all people because that’s one way of

all people because that’s one way of

all people because that’s one way of achieving that goal now so there is

achieving that goal now so there is

achieving that goal now so there is something wrong with this picture

something wrong with this picture

something wrong with this picture of requiring programmers to ensure that

of requiring programmers to ensure that

of requiring programmers to ensure that their machines are ethical before they

their machines are ethical before they

their machines are ethical before they let them into the wild and I think it’s

let them into the wild and I think it’s

let them into the wild and I think it’s because this goal is actually quite a

because this goal is actually quite a

because this goal is actually quite a tall order to ask of programmers you

tall order to ask of programmers you

tall order to ask of programmers you don’t ask people to open up other

don’t ask people to open up other

don’t ask people to open up other people’s brains to check if they are

people’s brains to check if they are

people’s brains to check if they are ethical by looking at their brain

ethical by looking at their brain

ethical by looking at their brain architecture this is just unrealistic

architecture this is just unrealistic

architecture this is just unrealistic instead what we do is we

instead what we do is we

instead what we do is we observe people’s behavior and then we

observe people’s behavior and then we

observe people’s behavior and then we hold it up we evaluated against set

hold it up we evaluated against set

hold it up we evaluated against set standards of conduct the second thing

standards of conduct the second thing

standards of conduct the second thing that hasn’t really changed much is the

that hasn’t really changed much is the

that hasn’t really changed much is the presence of unsolvable dilemmas so very

presence of unsolvable dilemmas so very

presence of unsolvable dilemmas so very common one that is I’ve been partly

common one that is I’ve been partly

common one that is I’ve been partly responsible for popularizing is the

responsible for popularizing is the

responsible for popularizing is the ethical dilemma that a an autonomous car

ethical dilemma that a an autonomous car

ethical dilemma that a an autonomous car may face so the story goes the car is

may face so the story goes the car is

may face so the story goes the car is somehow lost control is going to kill

somehow lost control is going to kill

somehow lost control is going to kill some pedestrians but it’s able to swerve

some pedestrians but it’s able to swerve

some pedestrians but it’s able to swerve and kill maybe its own passengers in

and kill maybe its own passengers in

and kill maybe its own passengers in order to save more pedestrians again

order to save more pedestrians again

order to save more pedestrians again many people think that you know AI

many people think that you know AI

many people think that you know AI should solve this problem but this

should solve this problem but this

should solve this problem but this problem is fundamentally unsolvable

problem is fundamentally unsolvable

problem is fundamentally unsolvable that’s that’s exactly why it’s called a

that’s that’s exactly why it’s called a

that’s that’s exactly why it’s called a dilemma it’s by definition unsolvable

dilemma it’s by definition unsolvable

dilemma it’s by definition unsolvable and it’s really a tall order to expect

and it’s really a tall order to expect

and it’s really a tall order to expect machines to solve a problem that defied

machines to solve a problem that defied

machines to solve a problem that defied the best legal minds and ethical

the best legal minds and ethical

the best legal minds and ethical philosophical minds for thousands of

philosophical minds for thousands of

philosophical minds for thousands of years before we add up the technology

years before we add up the technology

years before we add up the technology but that is not to say that we can solve

but that is not to say that we can solve

but that is not to say that we can solve these problems we can’t have a measured

these problems we can’t have a measured

these problems we can’t have a measured or a reasonable way forward for instance

or a reasonable way forward for instance

or a reasonable way forward for instance just think of these moose bar dilemma

just think of these moose bar dilemma

just think of these moose bar dilemma right I’ll call it which is this

right I’ll call it which is this

right I’ll call it which is this metallic bar that is placed in the front

metallic bar that is placed in the front

metallic bar that is placed in the front of the car if you remove these bars this

of the car if you remove these bars this

of the car if you remove these bars this most parts also called the kangaroo bar

most parts also called the kangaroo bar

most parts also called the kangaroo bar or a rule bar in Australia depends on

or a rule bar in Australia depends on

or a rule bar in Australia depends on what kind of animal you might hit and

what kind of animal you might hit and

what kind of animal you might hit and these things removing them would reduce

these things removing them would reduce

these things removing them would reduce the risk of injury by 21% to pedestrians

the risk of injury by 21% to pedestrians

the risk of injury by 21% to pedestrians and the risk of death by 6% which is why

and the risk of death by 6% which is why

and the risk of death by 6% which is why they are banned in many parts of Europe

they are banned in many parts of Europe

they are banned in many parts of Europe they’re banned in Australia but they’re

they’re banned in Australia but they’re

they’re banned in Australia but they’re not banned in the u.s. so there we are

not banned in the u.s. so there we are

not banned in the u.s. so there we are constantly making trade-offs you know

constantly making trade-offs you know

constantly making trade-offs you know about the risks that we’re willing to

about the risks that we’re willing to

about the risks that we’re willing to put up with relative risks in this case

put up with relative risks in this case

put up with relative risks in this case that are defined by some kind of social

that are defined by some kind of social

that are defined by some kind of social contract so we can continue to do the

contract so we can continue to do the

contract so we can continue to do the same with machines that are intelligent

same with machines that are intelligent

same with machines that are intelligent a third thing that is the same is our

a third thing that is the same is our

a third thing that is the same is our own psychological bias you know we’re

own psychological bias you know we’re

own psychological bias you know we’re we’re still human

we’re still human

we’re still human we still reason about the world in

we still reason about the world in

we still reason about the world in particular ways and so recently in our

particular ways and so recently in our

particular ways and so recently in our research with problem

research with problem

research with problem research that is trying to overcome

research that is trying to overcome

research that is trying to overcome these kinds of dilemmas so we’ve shown

these kinds of dilemmas so we’ve shown

these kinds of dilemmas so we’ve shown people their lifetime odds of dying from

people their lifetime odds of dying from

people their lifetime odds of dying from being in a regular car versus an

being in a regular car versus an

being in a regular car versus an autonomous car as you know according to

autonomous car as you know according to

autonomous car as you know according to the projections of the safety benefits

the projections of the safety benefits

the projections of the safety benefits of those cars and people say okay that’s

of those cars and people say okay that’s

of those cars and people say okay that’s good III like that but that does nothing

good III like that but that does nothing

good III like that but that does nothing to alleviate people’s concern about

to alleviate people’s concern about

to alleviate people’s concern about being sacrificed by the car okay and and

being sacrificed by the car okay and and

being sacrificed by the car okay and and these concerns alone despite of the

these concerns alone despite of the

these concerns alone despite of the absolute reduction in risk may dissipate

absolute reduction in risk may dissipate

absolute reduction in risk may dissipate people from adopting the cars and we’ve

people from adopting the cars and we’ve

people from adopting the cars and we’ve seen this before right the lifetime odds

seen this before right the lifetime odds

seen this before right the lifetime odds of dying from a car accident are

of dying from a car accident are

of dying from a car accident are astronomically higher than dying from a

astronomically higher than dying from a

astronomically higher than dying from a plane crash yet many more people are

plane crash yet many more people are

plane crash yet many more people are afraid of flying air travel compared to

afraid of flying air travel compared to

afraid of flying air travel compared to getting in a car and again we need to

getting in a car and again we need to

getting in a car and again we need to educate people about reasoning about

educate people about reasoning about

educate people about reasoning about risk in order to overcome these kinds of

risk in order to overcome these kinds of

risk in order to overcome these kinds of psychological biases so these are things

psychological biases so these are things

psychological biases so these are things that are similar that are not new I

that are similar that are not new I

that are similar that are not new I think so what’s different one thing that

think so what’s different one thing that

think so what’s different one thing that is different is the theory of mind so in

is different is the theory of mind so in

is different is the theory of mind so in in the current regulation in the current

in the current regulation in the current

in the current regulation in the current social contract we have we’re regulating

social contract we have we’re regulating

social contract we have we’re regulating entities that are humans we have some

entities that are humans we have some

entities that are humans we have some sort of theory of mind of how they might

sort of theory of mind of how they might

sort of theory of mind of how they might behave what they’re interested in and

behave what they’re interested in and

behave what they’re interested in and and what they dislike and like and we do

and what they dislike and like and we do

and what they dislike and like and we do the same thing with those who govern but

the same thing with those who govern but

the same thing with those who govern but with machines we have a new kind of

with machines we have a new kind of

with machines we have a new kind of agent that is slightly different so

agent that is slightly different so

agent that is slightly different so psychologists gray and colleagues have

psychologists gray and colleagues have

psychologists gray and colleagues have done a research on how we perceive other

done a research on how we perceive other

done a research on how we perceive other minds including the minds of humans the

minds including the minds of humans the

minds including the minds of humans the minds of pets and the minds of robots

minds of pets and the minds of robots

minds of pets and the minds of robots and as you can see here we perceive

and as you can see here we perceive

and as you can see here we perceive humans as being high on agency the

humans as being high on agency the

humans as being high on agency the ability to make autonomous decisions as

ability to make autonomous decisions as

ability to make autonomous decisions as well as high on experience the ability

well as high on experience the ability

well as high on experience the ability to experience pain pleasure and so on

to experience pain pleasure and so on

to experience pain pleasure and so on but you can see here that the robots are

but you can see here that the robots are

but you can see here that the robots are high on agency to some degree and

high on agency to some degree and

high on agency to some degree and actually moving further along the x-axis

actually moving further along the x-axis

actually moving further along the x-axis here but they’re very low on experience

here but they’re very low on experience

here but they’re very low on experience we don’t know how to reason about these

we don’t know how to reason about these

we don’t know how to reason about these kinds of agents as we regulate them for

kinds of agents as we regulate them for

kinds of agents as we regulate them for one

one

one we can’t punish them because they don’t

we can’t punish them because they don’t

we can’t punish them because they don’t care about pain and I think this is a

care about pain and I think this is a

care about pain and I think this is a fundamental shift that we have to think

fundamental shift that we have to think

fundamental shift that we have to think about as we form this new social

about as we form this new social

about as we form this new social contract a second thing that is

contract a second thing that is

contract a second thing that is different is adaptation beyond the

different is adaptation beyond the

different is adaptation beyond the initial intent in a completely

initial intent in a completely

initial intent in a completely unpredictable manner so for example the

unpredictable manner so for example the

unpredictable manner so for example the US Securities and Exchange Commission

US Securities and Exchange Commission

US Securities and Exchange Commission defines market manipulation as an

defines market manipulation as an

defines market manipulation as an intentional conduct designed to deceive

intentional conduct designed to deceive

intentional conduct designed to deceive investors but attribute an intent to a

investors but attribute an intent to a

investors but attribute an intent to a synthesized strategy that includes

synthesized strategy that includes

synthesized strategy that includes machine learning that adapts the

machine learning that adapts the

machine learning that adapts the strategy over time it’s an extremely

strategy over time it’s an extremely

strategy over time it’s an extremely difficult thing to do which makes it

difficult thing to do which makes it

difficult thing to do which makes it very difficult for us to point to the

very difficult for us to point to the

very difficult for us to point to the cause behind or the intent behind some

cause behind or the intent behind some

cause behind or the intent behind some behaviors a third thing that is

behaviors a third thing that is

behaviors a third thing that is different is speed so as systems become

different is speed so as systems become

different is speed so as systems become faster and faster for instance

faster and faster for instance

faster and faster for instance algorithmic trading systems the speed-up

algorithmic trading systems the speed-up

algorithmic trading systems the speed-up can generate new entirely new entirely

can generate new entirely new entirely

can generate new entirely new entirely new behavioral regimes for example much

new behavioral regimes for example much

new behavioral regimes for example much more frequent flash crashes and this

more frequent flash crashes and this

more frequent flash crashes and this happens precisely at the point where

happens precisely at the point where

happens precisely at the point where these where humans lose the ability to

these where humans lose the ability to

these where humans lose the ability to intervene in real time in these kinds of

intervene in real time in these kinds of

intervene in real time in these kinds of systems which brings up a whole lot of

systems which brings up a whole lot of

systems which brings up a whole lot of new challenges so we could slow down the

new challenges so we could slow down the

new challenges so we could slow down the machines somehow find ways to to keep

machines somehow find ways to to keep

machines somehow find ways to to keep them running at the speed that we can

them running at the speed that we can

them running at the speed that we can keep keep up with but obviously this

keep keep up with but obviously this

keep keep up with but obviously this type of solution means that we have to

type of solution means that we have to

type of solution means that we have to slow down our ability to benefit from

slow down our ability to benefit from

slow down our ability to benefit from those machines as well and if we were if

those machines as well and if we were if

those machines as well and if we were if we took this to the extreme apply this

we took this to the extreme apply this

we took this to the extreme apply this precautionary principle I would be

precautionary principle I would be

precautionary principle I would be sitting here and we will all be cavemen

sitting here and we will all be cavemen

sitting here and we will all be cavemen women and I’ll be telling you about the

women and I’ll be telling you about the

women and I’ll be telling you about the catastrophic consequences of using fire

catastrophic consequences of using fire

catastrophic consequences of using fire holding holding civilization obviously

holding holding civilization obviously

holding holding civilization obviously from progressing forward so one way of

from progressing forward so one way of

from progressing forward so one way of handling this challenge of speed is to

handling this challenge of speed is to

handling this challenge of speed is to use other programs to use programs to

use other programs to use programs to

use other programs to use programs to watch other programs so obviously

watch other programs so obviously

watch other programs so obviously instead of having a human here in the

instead of having a human here in the

instead of having a human here in the loop watching the algorithm or the AI we

loop watching the algorithm or the AI we

loop watching the algorithm or the AI we can have another AI watching

can have another AI watching

can have another AI watching this is what some people have referred

this is what some people have referred

this is what some people have referred to as oversight programs which i think

to as oversight programs which i think

to as oversight programs which i think is going to be increasingly common of

is going to be increasingly common of

is going to be increasingly common of course there are always limits to this

course there are always limits to this

course there are always limits to this because then who watches the algorithm

because then who watches the algorithm

because then who watches the algorithm who is certifying the first algorithm

who is certifying the first algorithm

who is certifying the first algorithm and obviously this can go only so far

and obviously this can go only so far

and obviously this can go only so far until you run out of material to build

until you run out of material to build

until you run out of material to build more powerful machines but I think it

more powerful machines but I think it

more powerful machines but I think it gives us some room to move so just kind

gives us some room to move so just kind

gives us some room to move so just kind of wrapping up the algorithmic social

of wrapping up the algorithmic social

of wrapping up the algorithmic social contract is going to change the our

contract is going to change the our

contract is going to change the our picture of regulation in two ways it’s

picture of regulation in two ways it’s

picture of regulation in two ways it’s going to change the kinds of actors that

going to change the kinds of actors that

going to change the kinds of actors that we regulate and is going to change the

we regulate and is going to change the

we regulate and is going to change the way that we regulate humans as well so

way that we regulate humans as well so

way that we regulate humans as well so the picture looks something like this

the picture looks something like this

the picture looks something like this where you have algorithmic actors that

where you have algorithmic actors that

where you have algorithmic actors that are being regulated by algorithm

are being regulated by algorithm

are being regulated by algorithm Augmented regulatory forces which kind

Augmented regulatory forces which kind

Augmented regulatory forces which kind of put brings a question of what is left

of put brings a question of what is left

of put brings a question of what is left for us to do but that’s what a different

for us to do but that’s what a different

for us to do but that’s what a different talk now one more thing super

talk now one more thing super

talk now one more thing super intelligence is this challenge that some

intelligence is this challenge that some

intelligence is this challenge that some people are talking about well what if

people are talking about well what if

people are talking about well what if machines become just so vastly more

machines become just so vastly more

machines become just so vastly more intelligent that all of humans combined

intelligent that all of humans combined

intelligent that all of humans combined how do we handle these types of

how do we handle these types of

how do we handle these types of challenges and is this the most urgent

challenges and is this the most urgent

challenges and is this the most urgent catastrophic risk that could end the

catastrophic risk that could end the

catastrophic risk that could end the human civilization altogether I think

human civilization altogether I think

human civilization altogether I think these kinds of challenges are not insane

these kinds of challenges are not insane

these kinds of challenges are not insane they are real but I think they’re very

they are real but I think they’re very

they are real but I think they’re very hard to predict and prepare for but I do

hard to predict and prepare for but I do

hard to predict and prepare for but I do think that by building technologies and

think that by building technologies and

think that by building technologies and institutions for creating an algorithmic

institutions for creating an algorithmic

institutions for creating an algorithmic social contract now we’re actually in a

social contract now we’re actually in a

social contract now we’re actually in a better position to be ready for for the

better position to be ready for for the

better position to be ready for for the broader challenge the longer-term

broader challenge the longer-term

broader challenge the longer-term challenge so thank you

you

you [Applause]

”

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *