my husband John and I are philanthropist that word means different things to
that word means different things to
that word means different things to different people to us it means
different people to us it means
different people to us it means aggressive investments in a number of
aggressive investments in a number of
aggressive investments in a number of issue areas from evidence-based policy
issue areas from evidence-based policy
issue areas from evidence-based policy criminal justice education healthcare
criminal justice education healthcare
criminal justice education healthcare pension reform public accountability and
pension reform public accountability and
pension reform public accountability and many more I get asked a lot of questions
many more I get asked a lot of questions
many more I get asked a lot of questions about our work why did you choose the
about our work why did you choose the
about our work why did you choose the shoe areas that you did and how do you
shoe areas that you did and how do you
shoe areas that you did and how do you choose new ones I always feel that when
choose new ones I always feel that when
choose new ones I always feel that when responding to that question people are
responding to that question people are
responding to that question people are expecting me to tell a story about some
expecting me to tell a story about some
expecting me to tell a story about some transformative event that happened in my
transformative event that happened in my
transformative event that happened in my life for John’s life that led us to
life for John’s life that led us to
life for John’s life that led us to embark on this path maybe they want to
embark on this path maybe they want to
embark on this path maybe they want to hear that we invest in health care
hear that we invest in health care
hear that we invest in health care because we’ve had family members who
because we’ve had family members who
because we’ve had family members who unfortunately have had cancer maybe they
unfortunately have had cancer maybe they
unfortunately have had cancer maybe they want to hear story about my life growing
want to hear story about my life growing
want to hear story about my life growing up in Puerto Rico and how that fueled my
up in Puerto Rico and how that fueled my
up in Puerto Rico and how that fueled my interest in and passion for fighting
interest in and passion for fighting
interest in and passion for fighting social justice issues and addressing
social justice issues and addressing
social justice issues and addressing poverty it said we’ve come to expect
poverty it said we’ve come to expect
poverty it said we’ve come to expect that every talk will start with a story
that every talk will start with a story
that every talk will start with a story like that something intensely moving and
like that something intensely moving and
like that something intensely moving and personal anything that starts with when
personal anything that starts with when
personal anything that starts with when I was a kid but I’m not going to tell a
I was a kid but I’m not going to tell a
I was a kid but I’m not going to tell a story like that although I could I think
story like that although I could I think
story like that although I could I think personal stories are so important they
personal stories are so important they
personal stories are so important they are what fuels our passion for many of
are what fuels our passion for many of
are what fuels our passion for many of us they’re the reason why we do the work
us they’re the reason why we do the work
us they’re the reason why we do the work we do but I believe that they’re
we do but I believe that they’re
we do but I believe that they’re precisely the wrong place to look for
precisely the wrong place to look for
precisely the wrong place to look for insight in fact I believe that we as
insight in fact I believe that we as
insight in fact I believe that we as policymakers government officials
policymakers government officials
policymakers government officials philanthropist even as individuals spend
philanthropist even as individuals spend
philanthropist even as individuals spend entirely too much time on anecdote and
entirely too much time on anecdote and
entirely too much time on anecdote and not enough time when evidence I’m here
not enough time when evidence I’m here
not enough time when evidence I’m here today to argue that we need to change
today to argue that we need to change
today to argue that we need to change that that we’re routinely making all
that that we’re routinely making all
that that we’re routinely making all kinds of decisions based on incomplete
kinds of decisions based on incomplete
kinds of decisions based on incomplete inconsistent flawed or even non-existent
inconsistent flawed or even non-existent
inconsistent flawed or even non-existent data based on anecdote that this is
data based on anecdote that this is
data based on anecdote that this is harming all of us in ways that we’re not
harming all of us in ways that we’re not
harming all of us in ways that we’re not appreciating and that we can and should
appreciating and that we can and should
appreciating and that we can and should do better now I’ve just made a bunch of
do better now I’ve just made a bunch of
do better now I’ve just made a bunch of provocative statements so let me get to
provocative statements so let me get to
provocative statements so let me get to work at convincing it I’ll start with
work at convincing it I’ll start with
work at convincing it I’ll start with one example from the Ted global stage in
one example from the Ted global stage in
one example from the Ted global stage in 2012 the second most viewed TED talk
2012 the second most viewed TED talk
2012 the second most viewed TED talk of all time Amy Cuddy psychologist
of all time Amy Cuddy psychologist
of all time Amy Cuddy psychologist Harvard Business School professor famous
Harvard Business School professor famous
Harvard Business School professor famous in large part for her TED talk for her
in large part for her TED talk for her
in large part for her TED talk for her work on power posing so she basically
work on power posing so she basically
work on power posing so she basically argues that standing in a power post
argues that standing in a power post
argues that standing in a power post position like Wonder Woman like this or
position like Wonder Woman like this or
position like Wonder Woman like this or like this you know whatever it makes you
like this you know whatever it makes you
like this you know whatever it makes you feel super powerful changes the hormone
feel super powerful changes the hormone
feel super powerful changes the hormone levels in your body and might even be
levels in your body and might even be
levels in your body and might even be conducive to greater success it’s
conducive to greater success it’s
conducive to greater success it’s amazing right I mean that’s so cool I
amazing right I mean that’s so cool I
amazing right I mean that’s so cool I should just stand like this the entire
should just stand like this the entire
should just stand like this the entire time the Ted community eighths is up
time the Ted community eighths is up
time the Ted community eighths is up over 38 million views over the last four
over 38 million views over the last four
over 38 million views over the last four and a half years the talk is been
and a half years the talk is been
and a half years the talk is been subtitled into 48 languages and Counting
subtitled into 48 languages and Counting
subtitled into 48 languages and Counting here’s how the Ted website today
here’s how the Ted website today
here’s how the Ted website today describes professor Cuddy’s work it says
describes professor Cuddy’s work it says
describes professor Cuddy’s work it says that professor Cuddy’s work reveals the
that professor Cuddy’s work reveals the
that professor Cuddy’s work reveals the benefit of power posing and it makes a
benefit of power posing and it makes a
benefit of power posing and it makes a reference to this hormone level issue
reference to this hormone level issue
reference to this hormone level issue that I just talked about national news
that I just talked about national news
that I just talked about national news outlets gushed about the work and many
outlets gushed about the work and many
outlets gushed about the work and many of us rushed to buy her best-selling
of us rushed to buy her best-selling
of us rushed to buy her best-selling book now I need to be a downer but I
book now I need to be a downer but I
book now I need to be a downer but I think for LG powerful thing doesn’t work
think for LG powerful thing doesn’t work
think for LG powerful thing doesn’t work at least not as advertised look at the
at least not as advertised look at the
at least not as advertised look at the myriad studies that have been conducted
myriad studies that have been conducted
myriad studies that have been conducted on power posing ever since professor
on power posing ever since professor
on power posing ever since professor Cuddy released their findings they point
Cuddy released their findings they point
Cuddy released their findings they point the glaring errors in her methodology
the glaring errors in her methodology
the glaring errors in her methodology they questioned the entire theory of
they questioned the entire theory of
they questioned the entire theory of power posing some researchers tried to
power posing some researchers tried to
power posing some researchers tried to reproduce her work and couldn’t others
reproduce her work and couldn’t others
reproduce her work and couldn’t others tried to reproduce her work and got
tried to reproduce her work and got
tried to reproduce her work and got exactly the opposite result that power
exactly the opposite result that power
exactly the opposite result that power posing reduces feelings of power and it
posing reduces feelings of power and it
posing reduces feelings of power and it gets worse even professor Cuddy’s
gets worse even professor Cuddy’s
gets worse even professor Cuddy’s co-author who herself as a prominent
co-author who herself as a prominent
co-author who herself as a prominent academic has completely disavow the
academic has completely disavow the
academic has completely disavow the study and she did so in no uncertain
study and she did so in no uncertain
study and she did so in no uncertain terms here’s where she said I do not
terms here’s where she said I do not
terms here’s where she said I do not believe the powerful effects are real
believe the powerful effects are real
believe the powerful effects are real and she goes farther than that she
and she goes farther than that she
and she goes farther than that she confesses to glaring errors in the way
confesses to glaring errors in the way
confesses to glaring errors in the way that they conducted their study tiny
that they conducted their study tiny
that they conducted their study tiny sample sizes fluency data selectively
sample sizes fluency data selectively
sample sizes fluency data selectively reported findings even professor Cuddy
reported findings even professor Cuddy
reported findings even professor Cuddy herself who continues to stand by the
herself who continues to stand by the
herself who continues to stand by the power post Theory now has sort of
power post Theory now has sort of
power post Theory now has sort of amended her story and now she says that
amended her story and now she says that
amended her story and now she says that she’s agnostic about power poses effects
she’s agnostic about power poses effects
she’s agnostic about power poses effects on hormone levels
on hormone levels
on hormone levels Gnostic that was the whole reason she
Gnostic that was the whole reason she
Gnostic that was the whole reason she went on the Ted stage into this and did
went on the Ted stage into this and did
went on the Ted stage into this and did this and made us all do this and feel
this and made us all do this and feel
this and made us all do this and feel powerful you might think this is a trite
powerful you might think this is a trite
powerful you might think this is a trite example and it’s by no means meant to be
example and it’s by no means meant to be
example and it’s by no means meant to be a personal attack who cares if I stand
a personal attack who cares if I stand
a personal attack who cares if I stand in front of you like this if that makes
in front of you like this if that makes
in front of you like this if that makes me feel good or not and what does it
me feel good or not and what does it
me feel good or not and what does it have to do with solving the country’s
have to do with solving the country’s
have to do with solving the country’s problems and the world’s problems which
problems and the world’s problems which
problems and the world’s problems which is why we’re all here well a lot because
is why we’re all here well a lot because
is why we’re all here well a lot because this example is endemic or something
this example is endemic or something
this example is endemic or something that we see throughout academic research
that we see throughout academic research
that we see throughout academic research and throughout research in general
and throughout research in general
and throughout research in general virtually everywhere you look you’ll
virtually everywhere you look you’ll
virtually everywhere you look you’ll find researchers many of them prominent
find researchers many of them prominent
find researchers many of them prominent and most of them well-intentioned
and most of them well-intentioned
and most of them well-intentioned actively misleading us into believing
actively misleading us into believing
actively misleading us into believing that bad research is proven fact we’ve
that bad research is proven fact we’ve
that bad research is proven fact we’ve seen this time and again in our work and
seen this time and again in our work and
seen this time and again in our work and when John and I started the foundation
when John and I started the foundation
when John and I started the foundation we set out to solve the country’s
we set out to solve the country’s
we set out to solve the country’s problems by attacking root causes so we
problems by attacking root causes so we
problems by attacking root causes so we were pretty new at this so we figured
were pretty new at this so we figured
were pretty new at this so we figured that the best place to start was to try
that the best place to start was to try
that the best place to start was to try to get our head around what we did and
to get our head around what we did and
to get our head around what we did and didn’t know as a community collectively
didn’t know as a community collectively
didn’t know as a community collectively as a country what do we know about what
as a country what do we know about what
as a country what do we know about what doesn’t doesn’t work after all
doesn’t doesn’t work after all
doesn’t doesn’t work after all philanthropist has been trying to save
philanthropist has been trying to save
philanthropist has been trying to save the country and make the country better
the country and make the country better
the country and make the country better for many many years and have spent
for many many years and have spent
for many many years and have spent billions of dollars doing it governments
billions of dollars doing it governments
billions of dollars doing it governments state governments local governments have
state governments local governments have
state governments local governments have spent exponentially more on health care
spent exponentially more on health care
spent exponentially more on health care programs social programs anti-poverty
programs social programs anti-poverty
programs social programs anti-poverty programs job training you name it so we
programs job training you name it so we
programs job training you name it so we assumed that there had to be a massive
assumed that there had to be a massive
assumed that there had to be a massive body of evidence that could help guide
body of evidence that could help guide
body of evidence that could help guide our investment decisions what we found
our investment decisions what we found
our investment decisions what we found was really alarming we saw bad research
was really alarming we saw bad research
was really alarming we saw bad research everywhere it didn’t matter where we
everywhere it didn’t matter where we
everywhere it didn’t matter where we looked for example we turned to
looked for example we turned to
looked for example we turned to nutrition as potentially an avenue to
nutrition as potentially an avenue to
nutrition as potentially an avenue to address our health epidemics we wanted
address our health epidemics we wanted
address our health epidemics we wanted to get smart about what factors and
to get smart about what factors and
to get smart about what factors and foods might be conducive to chronic
foods might be conducive to chronic
foods might be conducive to chronic diseases obesity heart disease diabetes
diseases obesity heart disease diabetes
diseases obesity heart disease diabetes and we found some very good research but
and we found some very good research but
and we found some very good research but we also found an abundance of studies
we also found an abundance of studies
we also found an abundance of studies like these chocolate makes you skinny Oh
like these chocolate makes you skinny Oh
like these chocolate makes you skinny Oh beer helps you work out in case you
beer helps you work out in case you
beer helps you work out in case you didn’t know cause knee prevents
didn’t know cause knee prevents
didn’t know cause knee prevents Alzheimer’s super catchy
Alzheimer’s super catchy
Alzheimer’s super catchy loses a researcher notices the people
loses a researcher notices the people
loses a researcher notices the people under study who are eating chocolate are
under study who are eating chocolate are
under study who are eating chocolate are also skinny and summarily comes to the
also skinny and summarily comes to the
also skinny and summarily comes to the conclusion or at least tells us that
conclusion or at least tells us that
conclusion or at least tells us that they’re skinny because eat chocolate
they’re skinny because eat chocolate
they’re skinny because eat chocolate same thing with a Alzheimer study shoddy
same thing with a Alzheimer study shoddy
same thing with a Alzheimer study shoddy research bad methodologies small sample
research bad methodologies small sample
research bad methodologies small sample sizes correlations powdered as
sizes correlations powdered as
sizes correlations powdered as causations selectively reported findings
causations selectively reported findings
causations selectively reported findings just like the Amy Cuddy study my way
just like the Amy Cuddy study my way
just like the Amy Cuddy study my way there’s a lot of these one might say a
there’s a lot of these one might say a
there’s a lot of these one might say a lot of alternative facts in 2012 a group
lot of alternative facts in 2012 a group
lot of alternative facts in 2012 a group of researchers randomly chose 50
of researchers randomly chose 50
of researchers randomly chose 50 ingredients from a cookbook normal stuff
ingredients from a cookbook normal stuff
ingredients from a cookbook normal stuff like milk eggs and they took a look at
like milk eggs and they took a look at
like milk eggs and they took a look at the body of research relating to these
the body of research relating to these
the body of research relating to these ingredients to see what they could learn
ingredients to see what they could learn
ingredients to see what they could learn about whether these ingredients did or
about whether these ingredients did or
about whether these ingredients did or did not cause cancer seems like a
did not cause cancer seems like a
did not cause cancer seems like a worthwhile exercise here’s what they
worthwhile exercise here’s what they
worthwhile exercise here’s what they found for every ingredient that you see
found for every ingredient that you see
found for every ingredient that you see listed in the slide lying Tomatoes milk
listed in the slide lying Tomatoes milk
listed in the slide lying Tomatoes milk eggs corn coffee butter there was at
eggs corn coffee butter there was at
eggs corn coffee butter there was at least one research study that argued
least one research study that argued
least one research study that argued that the ingredient causes cancer and at
that the ingredient causes cancer and at
that the ingredient causes cancer and at least one research study that argued
least one research study that argued
least one research study that argued that the ingredient prevented cancer
that the ingredient prevented cancer
that the ingredient prevented cancer what are we supposed to do with this
what are we supposed to do with this
what are we supposed to do with this information we turn to health care same
information we turn to health care same
information we turn to health care same thing we saw that the authors of the
thing we saw that the authors of the
thing we saw that the authors of the vast majority of clinical trials
vast majority of clinical trials
vast majority of clinical trials reported and top medical journals
reported and top medical journals
reported and top medical journals silently changed the outcomes that they
silently changed the outcomes that they
silently changed the outcomes that they were reporting so they said they were
were reporting so they said they were
were reporting so they said they were going to study one thing but they
going to study one thing but they
going to study one thing but they reported on another now why would they
reported on another now why would they
reported on another now why would they do that now that I don’t claim to be an
do that now that I don’t claim to be an
do that now that I don’t claim to be an expert scientific research is enormously
expert scientific research is enormously
expert scientific research is enormously complicated certainly clinical
complicated certainly clinical
complicated certainly clinical researchers as well but one theory might
researchers as well but one theory might
researchers as well but one theory might be that the original studies didn’t pan
be that the original studies didn’t pan
be that the original studies didn’t pan out the way they wanted so they cherry
out the way they wanted so they cherry
out the way they wanted so they cherry picked positive findings from those same
picked positive findings from those same
picked positive findings from those same studies on secondary outcomes and
studies on secondary outcomes and
studies on secondary outcomes and reported on those instead so they could
reported on those instead so they could
reported on those instead so they could get published now you don’t have to have
get published now you don’t have to have
get published now you don’t have to have a PhD to start wondering whether maybe
a PhD to start wondering whether maybe
a PhD to start wondering whether maybe there could be something fishy going on
there could be something fishy going on
there could be something fishy going on here and these shenanigans happen
here and these shenanigans happen
here and these shenanigans happen everywhere take a recent project that we
everywhere take a recent project that we
everywhere take a recent project that we did a reproducibility project so we
did a reproducibility project so we
did a reproducibility project so we asked researchers to reproduce one
asked researchers to reproduce one
asked researchers to reproduce one hundred psychology experiments that had
hundred psychology experiments that had
hundred psychology experiments that had been published in top psychology
been published in top psychology
been published in top psychology journals in 2008 so go do them again if
journals in 2008 so go do them again if
journals in 2008 so go do them again if you do them again will you find the same
you do them again will you find the same
you do them again will you find the same result that’s what we wanted to know you
result that’s what we wanted to know you
result that’s what we wanted to know you know often they could find the same
know often they could find the same
know often they could find the same results one third to one half of the
results one third to one half of the
results one third to one half of the time now I’m not claiming that
time now I’m not claiming that
time now I’m not claiming that scientists and researchers are actively
scientists and researchers are actively
scientists and researchers are actively and intentionally committing fraud I’m
and intentionally committing fraud I’m
and intentionally committing fraud I’m saying there’s something broken in our
saying there’s something broken in our
saying there’s something broken in our system where scientists are feeling the
system where scientists are feeling the
system where scientists are feeling the need to report only positive findings at
need to report only positive findings at
need to report only positive findings at the expense of the whole story how would
the expense of the whole story how would
the expense of the whole story how would they do that well both ways so let’s say
they do that well both ways so let’s say
they do that well both ways so let’s say that a researcher wants to prove that
that a researcher wants to prove that
that a researcher wants to prove that there’s a relationship between eating
there’s a relationship between eating
there’s a relationship between eating blue M&Ms and prostate cancer because
blue M&Ms and prostate cancer because
blue M&Ms and prostate cancer because that would be newsworthy so he designs
that would be newsworthy so he designs
that would be newsworthy so he designs the conducts an experiment but he
the conducts an experiment but he
the conducts an experiment but he doesn’t find a relationship okay so he
doesn’t find a relationship okay so he
doesn’t find a relationship okay so he does it again the same experiment does
does it again the same experiment does
does it again the same experiment does it again no relationship
it again no relationship
it again no relationship he doesn’t 17 more times in our
he doesn’t 17 more times in our
he doesn’t 17 more times in our relationship but on the 20th time he
relationship but on the 20th time he
relationship but on the 20th time he does find a relationship boom he can
does find a relationship boom he can
does find a relationship boom he can publish blue M&Ms cause prostate cancer
publish blue M&Ms cause prostate cancer
publish blue M&Ms cause prostate cancer new study shows he doesn’t tell us about
new study shows he doesn’t tell us about
new study shows he doesn’t tell us about the 19 other times that he conducted
the 19 other times that he conducted
the 19 other times that he conducted that same study and failed he just tells
that same study and failed he just tells
that same study and failed he just tells us about the one time that he succeeded
us about the one time that he succeeded
us about the one time that he succeeded that’s called the Fowler effect the
that’s called the Fowler effect the
that’s called the Fowler effect the researcher could also tweak the
researcher could also tweak the
researcher could also tweak the statistical analysis until he gets the
statistical analysis until he gets the
statistical analysis until he gets the results that he wants that’s healthy
results that he wants that’s healthy
results that he wants that’s healthy hacking or he could report is to a very
hacking or he could report is to a very
hacking or he could report is to a very narrowly defined group when in fact the
narrowly defined group when in fact the
narrowly defined group when in fact the original research study was meant to
original research study was meant to
original research study was meant to address a much much larger group why
address a much much larger group why
address a much much larger group why does it happen well there could be lots
does it happen well there could be lots
does it happen well there could be lots of reasons but I would argue that a
of reasons but I would argue that a
of reasons but I would argue that a reason is it the incentive system in
reason is it the incentive system in
reason is it the incentive system in science and in research is broken in an
science and in research is broken in an
science and in research is broken in an ideal world scientists and researchers
ideal world scientists and researchers
ideal world scientists and researchers will be motivated by one thing the
will be motivated by one thing the
will be motivated by one thing the pursuit of truth but in the real world
pursuit of truth but in the real world
pursuit of truth but in the real world scientists and researchers are equally
scientists and researchers are equally
scientists and researchers are equally motivated by the desire to publish
motivated by the desire to publish
motivated by the desire to publish because that’s the vehicle for achieving
because that’s the vehicle for achieving
because that’s the vehicle for achieving tenure that’s the vehicle for getting
tenure that’s the vehicle for getting
tenure that’s the vehicle for getting funding and for achieving notoriety
funding and for achieving notoriety
funding and for achieving notoriety scientific journals are clamoring for
scientific journals are clamoring for
scientific journals are clamoring for articles that report flashy results so
articles that report flashy results so
articles that report flashy results so that those articles can get cited they
that those articles can get cited they
that those articles can get cited they would have a term for this is called the
would have a term for this is called the
would have a term for this is called the impact factor then of course there’s the
impact factor then of course there’s the
impact factor then of course there’s the media and we as consumers we all want to
media and we as consumers we all want to
media and we as consumers we all want to hear those flashy results as a
hear those flashy results as a
hear those flashy results as a researchers deliver even if it’s at the
researchers deliver even if it’s at the
researchers deliver even if it’s at the expense of full transparency and rigor
expense of full transparency and rigor
expense of full transparency and rigor and while the worse off for it our
and while the worse off for it our
and while the worse off for it our foundation we try to break this cycle
foundation we try to break this cycle
foundation we try to break this cycle and reform the system by funding
and reform the system by funding
and reform the system by funding organizations that are promoting
organizations that are promoting
organizations that are promoting transparency in good practices and
transparency in good practices and
transparency in good practices and collaboration and data sharing the
collaboration and data sharing the
collaboration and data sharing the Center for open science the Center for
Center for open science the Center for
Center for open science the Center for evidence-based medicine at Oxford the
evidence-based medicine at Oxford the
evidence-based medicine at Oxford the metric Center at Stanford health news
metric Center at Stanford health news
metric Center at Stanford health news organizations that are holding the
organizations that are holding the
organizations that are holding the media’s feet to the fire as to what they
media’s feet to the fire as to what they
media’s feet to the fire as to what they report on research studies and they’re
report on research studies and they’re
report on research studies and they’re all doing terrific work but we need to
all doing terrific work but we need to
all doing terrific work but we need to do so much more we didn’t become
do so much more we didn’t become
do so much more we didn’t become philanthropists to hang out with
philanthropists to hang out with
philanthropists to hang out with academics and role in the hallowed halls
academics and role in the hallowed halls
academics and role in the hallowed halls of universities we became
of universities we became
of universities we became philanthropists to change the world but
philanthropists to change the world but
philanthropists to change the world but how can we even think about what to
how can we even think about what to
how can we even think about what to change if we don’t know what works and
change if we don’t know what works and
change if we don’t know what works and how are we supposed to know what works
how are we supposed to know what works
how are we supposed to know what works if we can’t trust research so this is
if we can’t trust research so this is
if we can’t trust research so this is our problem
our problem
our problem this is our issue this goes to the core
this is our issue this goes to the core
this is our issue this goes to the core of who we are and what we do we can’t
of who we are and what we do we can’t
of who we are and what we do we can’t function if we don’t have a healthy
function if we don’t have a healthy
function if we don’t have a healthy research ecosystem this problem of bad
research ecosystem this problem of bad
research ecosystem this problem of bad research and bad science isn’t limited
research and bad science isn’t limited
research and bad science isn’t limited track edenia this is just as bad in
track edenia this is just as bad in
track edenia this is just as bad in public policy and remember the Scared
public policy and remember the Scared
public policy and remember the Scared Straight program you know at-risk youth
Straight program you know at-risk youth
Straight program you know at-risk youth would go to maximum-security prisons or
would go to maximum-security prisons or
would go to maximum-security prisons or meet prisoners and the prisoners would
meet prisoners and the prisoners would
meet prisoners and the prisoners would yell at them and tell them about their
yell at them and tell them about their
yell at them and tell them about their bad choices and how they have to live
bad choices and how they have to live
bad choices and how they have to live their life on the straight and narrow we
their life on the straight and narrow we
their life on the straight and narrow we spent millions of dollars on these
spent millions of dollars on these
spent millions of dollars on these programs and they made intuitive sense
programs and they made intuitive sense
programs and they made intuitive sense except that they didn’t work research
except that they didn’t work research
except that they didn’t work research showed that these programs actually
showed that these programs actually
showed that these programs actually increase the likelihood that these kids
increase the likelihood that these kids
increase the likelihood that these kids would commit criminal acts we should
would commit criminal acts we should
would commit criminal acts we should have known this much earlier we would
have known this much earlier we would
have known this much earlier we would have saved millions of dollars and more
have saved millions of dollars and more
have saved millions of dollars and more importantly maybe we would have saved
importantly maybe we would have saved
importantly maybe we would have saved some of these
some of these
some of these kids three federal departments the
kids three federal departments the
kids three federal departments the Department of Labor HHS and the
Department of Labor HHS and the
Department of Labor HHS and the Department of Education have funded
Department of Education have funded
Department of Education have funded randomized control trials on social
randomized control trials on social
randomized control trials on social programs that have either been
programs that have either been
programs that have either been administered by the government or by the
administered by the government or by the
administered by the government or by the private sector
private sector
private sector now randomized control trials are the
now randomized control trials are the
now randomized control trials are the gold standard in research here’s what
gold standard in research here’s what
gold standard in research here’s what they found
they found
they found 70% of the employment and job training
70% of the employment and job training
70% of the employment and job training programs that the Department of Labor
programs that the Department of Labor
programs that the Department of Labor looked at had weak or no positive
looked at had weak or no positive
looked at had weak or no positive effects of the 28 teen pregnancy
effects of the 28 teen pregnancy
effects of the 28 teen pregnancy prevention programs that HHS looked at
prevention programs that HHS looked at
prevention programs that HHS looked at only three were worthwhile and eighty to
only three were worthwhile and eighty to
only three were worthwhile and eighty to 90 percent of the education programs
90 percent of the education programs
90 percent of the education programs that the Department of Education looked
that the Department of Education looked
that the Department of Education looked at had weak or no positive effects
at had weak or no positive effects
at had weak or no positive effects relative to what squares were doing
relative to what squares were doing
relative to what squares were doing already so we know so little about what
already so we know so little about what
already so we know so little about what works in education and in job training
works in education and in job training
works in education and in job training and in policy in general even when we
and in policy in general even when we
and in policy in general even when we think we do take the federal
think we do take the federal
think we do take the federal government’s what works Clearinghouse
government’s what works Clearinghouse
government’s what works Clearinghouse which was established as a resource for
which was established as a resource for
which was established as a resource for practitioners to determine what works in
practitioners to determine what works in
practitioners to determine what works in education it purports to rely on
education it purports to rely on
education it purports to rely on rigorous research let’s look at this
rigorous research let’s look at this
rigorous research let’s look at this research there is a study that had only
research there is a study that had only
research there is a study that had only a few dozen participants there’s another
a few dozen participants there’s another
a few dozen participants there’s another study that was conducted over the course
study that was conducted over the course
study that was conducted over the course of around 12 weeks then there’s a bunch
of around 12 weeks then there’s a bunch
of around 12 weeks then there’s a bunch of positive labels on secondary outcomes
of positive labels on secondary outcomes
of positive labels on secondary outcomes that ultimately don’t matter
that ultimately don’t matter
that ultimately don’t matter for example the website labels is
for example the website labels is
for example the website labels is positive a reading preparation program
positive a reading preparation program
positive a reading preparation program because research showed that after going
because research showed that after going
because research showed that after going through the program kids were able to
through the program kids were able to
through the program kids were able to recognize letters of the alphabet well
recognize letters of the alphabet well
recognize letters of the alphabet well you might say that’s a stepping stone to
you might say that’s a stepping stone to
you might say that’s a stepping stone to reading so that sounds pretty reasonable
reading so that sounds pretty reasonable
reading so that sounds pretty reasonable to me and it does except that the
to me and it does except that the
to me and it does except that the website doesn’t tell us that those same
website doesn’t tell us that those same
website doesn’t tell us that those same researchers studied that same program
researchers studied that same program
researchers studied that same program and found that the program had no
and found that the program had no
and found that the program had no effects on kids ultimate ability to read
effects on kids ultimate ability to read
effects on kids ultimate ability to read well isn’t that what we care about so
well isn’t that what we care about so
well isn’t that what we care about so we’re spending millions and millions of
we’re spending millions and millions of
we’re spending millions and millions of dollars on programs that at worst don’t
dollars on programs that at worst don’t
dollars on programs that at worst don’t work and at best we don’t know well we
work and at best we don’t know well we
work and at best we don’t know well we don’t have sufficient data or we don’t
don’t have sufficient data or we don’t
don’t have sufficient data or we don’t have reliable data so this needs to
have reliable data so this needs to
have reliable data so this needs to change we need to stop as a
change we need to stop as a
change we need to stop as a philanthropic community as a
philanthropic community as a
philanthropic community as a policymaking community we need to stop
policymaking community we need to stop
policymaking community we need to stop funding these programs and relying on
funding these programs and relying on
funding these programs and relying on this ecosystem that isn’t giving us the
this ecosystem that isn’t giving us the
this ecosystem that isn’t giving us the results that we need
results that we need
results that we need and I’ve got some ideas on how to do
and I’ve got some ideas on how to do
and I’ve got some ideas on how to do that how to reform the system first more
that how to reform the system first more
that how to reform the system first more randomized control trials those of us
randomized control trials those of us
randomized control trials those of us who work with governments for
who work with governments for
who work with governments for governments collaborate with governments
governments collaborate with governments
governments collaborate with governments need to demand more randomized control
need to demand more randomized control
need to demand more randomized control trials so that we can get evidence and
trials so that we can get evidence and
trials so that we can get evidence and understand what doesn’t doesn’t work
understand what doesn’t doesn’t work
understand what doesn’t doesn’t work second we all need to follow the
second we all need to follow the
second we all need to follow the evidence this is on all of us
evidence this is on all of us
evidence this is on all of us governments and philanthropist need to
governments and philanthropist need to
governments and philanthropist need to stop funding what doesn’t work and start
stop funding what doesn’t work and start
stop funding what doesn’t work and start funding what does work we need to hold
funding what does work we need to hold
funding what does work we need to hold our own selves accountable for these
our own selves accountable for these
our own selves accountable for these results and for this problem this isn’t
results and for this problem this isn’t
results and for this problem this isn’t something that was imposed upon us –
something that was imposed upon us –
something that was imposed upon us – something that we’ve created and that’s
something that we’ve created and that’s
something that we’ve created and that’s on us to fix and third we need better
on us to fix and third we need better
on us to fix and third we need better data we can’t do research if we don’t
data we can’t do research if we don’t
data we can’t do research if we don’t have healthy data systems that are
have healthy data systems that are
have healthy data systems that are speaking to each other that are
speaking to each other that are
speaking to each other that are harmonized that researchers can access
harmonized that researchers can access
harmonized that researchers can access to give us the answers that we need now
to give us the answers that we need now
to give us the answers that we need now here – there are excellent organizations
here – there are excellent organizations
here – there are excellent organizations both within and outside the government
both within and outside the government
both within and outside the government that are working on these issues within
that are working on these issues within
that are working on these issues within the government the Commission on
the government the Commission on
the government the Commission on evidence-based policymaking the Social
evidence-based policymaking the Social
evidence-based policymaking the Social and Behavioral Sciences team are
and Behavioral Sciences team are
and Behavioral Sciences team are conducting randomized control trials
conducting randomized control trials
conducting randomized control trials pushing an evidence-based policymaking
pushing an evidence-based policymaking
pushing an evidence-based policymaking agenda within government and they’re
agenda within government and they’re
agenda within government and they’re doing terrific work in collaboration
doing terrific work in collaboration
doing terrific work in collaboration with stayed in local governments in the
with stayed in local governments in the
with stayed in local governments in the nonprofit sector the rhode island policy
nonprofit sector the rhode island policy
nonprofit sector the rhode island policy innovation labs j-pal results for
innovation labs j-pal results for
innovation labs j-pal results for America are all contributing to
America are all contributing to
America are all contributing to reforming this ecosystem by either
reforming this ecosystem by either
reforming this ecosystem by either conducting randomized control trials or
conducting randomized control trials or
conducting randomized control trials or working at promoting evidence and thanks
working at promoting evidence and thanks
working at promoting evidence and thanks to the work of these organizations and
to the work of these organizations and
to the work of these organizations and many others we now have some answers we
many others we now have some answers we
many others we now have some answers we now know which programs do move the
now know which programs do move the
now know which programs do move the needle on things that we care about like
needle on things that we care about like
needle on things that we care about like child welfare and education and job
child welfare and education and job
child welfare and education and job training and recidivism armed with that
training and recidivism armed with that
training and recidivism armed with that information aren’t we in a better spot
information aren’t we in a better spot
information aren’t we in a better spot as people who care as concerned citizens
as people who care as concerned citizens
as people who care as concerned citizens as philanthropists as policymakers to
as philanthropists as policymakers to
as philanthropists as policymakers to make good decisions we need to follow
make good decisions we need to follow
make good decisions we need to follow data we need to focus more on research
data we need to focus more on research
data we need to focus more on research and we need to demand that of
and we need to demand that of
and we need to demand that of governments and of ourselves because
governments and of ourselves because
governments and of ourselves because policy ideas that truly are we’re
policy ideas that truly are we’re
policy ideas that truly are we’re spreading
spreading
spreading or the ones at work thank you
or the ones at work thank you
or the ones at work thank you [Applause]
[Applause]
[Applause] [Music]
”
Be First to Comment