hi so imagine that you are a head of a development organization and many of you
development organization and many of you
development organization and many of you actually are indeed and so you are
actually are indeed and so you are
actually are indeed and so you are listening to a pitch from three
listening to a pitch from three
listening to a pitch from three different teams and the idea that they
different teams and the idea that they
different teams and the idea that they are pitching for is an effective support
are pitching for is an effective support
are pitching for is an effective support package for the poor to be able to
package for the poor to be able to
package for the poor to be able to increase their income and the first team
increase their income and the first team
increase their income and the first team says that you should invest in cows
says that you should invest in cows
says that you should invest in cows because cows generates very high return
because cows generates very high return
because cows generates very high return on investment but second team says that
on investment but second team says that
on investment but second team says that no no no actually issue give cash
no no no actually issue give cash
no no no actually issue give cash because with cash the poor can make
because with cash the poor can make
because with cash the poor can make their own decisions on what to invest
their own decisions on what to invest
their own decisions on what to invest and the third group says well actually
and the third group says well actually
and the third group says well actually you should invest in chickens because
you should invest in chickens because
you should invest in chickens because you can reach even more people with the
you can reach even more people with the
you can reach even more people with the same amount of money that you’re
same amount of money that you’re
same amount of money that you’re providing for cows for example so how do
providing for cows for example so how do
providing for cows for example so how do you decide this I think is a very
you decide this I think is a very
you decide this I think is a very important question for all of us because
important question for all of us because
important question for all of us because we want to make a maximum impact in a
we want to make a maximum impact in a
we want to make a maximum impact in a very very effective manner so a sensible
very very effective manner so a sensible
very very effective manner so a sensible answer might be to say that well let’s
answer might be to say that well let’s
answer might be to say that well let’s compare these options and see which one
compare these options and see which one
compare these options and see which one would generate the highest return on
would generate the highest return on
would generate the highest return on investment and based on that you will
investment and based on that you will
investment and based on that you will make a decision
make a decision
make a decision it sounds no-brainer right but are we as
it sounds no-brainer right but are we as
it sounds no-brainer right but are we as a development sector doing that ready
testing and comparing different options
testing and comparing different options with evidence oh let’s call
with evidence oh let’s call
with evidence oh let’s call experimentation doesn’t seem to happen
experimentation doesn’t seem to happen
experimentation doesn’t seem to happen all that often in the development sector
all that often in the development sector
all that often in the development sector but if you think about it
but if you think about it
but if you think about it experimentation is everywhere we humans
experimentation is everywhere we humans
experimentation is everywhere we humans want to understand how the world works
want to understand how the world works
want to understand how the world works and we want to understand how we can get
and we want to understand how we can get
and we want to understand how we can get the result that we want and take an
the result that we want and take an
the result that we want and take an example
example
example this is my sixth year old daughter and
this is my sixth year old daughter and
this is my sixth year old daughter and she often conducts an experiment so she
she often conducts an experiment so she
she often conducts an experiment so she will come to me and ask a questions like
will come to me and ask a questions like
will come to me and ask a questions like this can I watch something on my on my
this can I watch something on my on my
this can I watch something on my on my iPad and if I say no she would go to her
iPad and if I say no she would go to her
iPad and if I say no she would go to her mother and asked the exact same question
mother and asked the exact same question
mother and asked the exact same question to see if she would get the most
to see if she would get the most
to see if she would get the most satisfactory answer
satisfactory answer
satisfactory answer so through this experiment she’s
so through this experiment she’s
so through this experiment she’s figuring out how she can get what she
figuring out how she can get what she
figuring out how she can get what she wants an experimentation seems to be
wants an experimentation seems to be
wants an experimentation seems to be also a commonplace in the business
also a commonplace in the business
also a commonplace in the business sector so suppose that you are renewing
sector so suppose that you are renewing
sector so suppose that you are renewing your company’s website in a big way and
your company’s website in a big way and
your company’s website in a big way and before you make a huge investment you
before you make a huge investment you
before you make a huge investment you might want to compare right so this
might want to compare right so this
might want to compare right so this design a generated this percentage of
design a generated this percentage of
design a generated this percentage of the click rate and the other design
the click rate and the other design
the click rate and the other design created generated even higher click
created generated even higher click
created generated even higher click rates so based on this test you might
rates so based on this test you might
rates so based on this test you might want to make a decision on which one you
want to make a decision on which one you
want to make a decision on which one you want to invest in so this
want to invest in so this
want to invest in so this experimentation is a commonplace in
experimentation is a commonplace in
experimentation is a commonplace in business and in our everyday life
business and in our everyday life
business and in our everyday life but it does not seems to be the norm in
but it does not seems to be the norm in
but it does not seems to be the norm in the development sector and this is
the development sector and this is
the development sector and this is despite the fact that there’s so much at
despite the fact that there’s so much at
despite the fact that there’s so much at stake in the development sector because
stake in the development sector because
stake in the development sector because it touches the lives and well-being of
it touches the lives and well-being of
it touches the lives and well-being of so many people in living in a very
so many people in living in a very
so many people in living in a very difficult conditions and I must admit
difficult conditions and I must admit
difficult conditions and I must admit that I myself was not thinking about
that I myself was not thinking about
that I myself was not thinking about experiment until recently so the large
experiment until recently so the large
experiment until recently so the large development programs are often designed
development programs are often designed
development programs are often designed without really comparing different
without really comparing different
without really comparing different options and without really looking at
options and without really looking at
options and without really looking at the evidence and the reason for this
the evidence and the reason for this
the evidence and the reason for this might be because of the way that
might be because of the way that
might be because of the way that planning and funding process are set up
planning and funding process are set up
planning and funding process are set up so funders often ask the recipient
so funders often ask the recipient
so funders often ask the recipient organization to spell out all the
organization to spell out all the
organization to spell out all the detailed activities for the next four
detailed activities for the next four
detailed activities for the next four years and there is also a cumbersome
years and there is also a cumbersome
years and there is also a cumbersome grant amendment process if you want to
grant amendment process if you want to
grant amendment process if you want to change the project design which is quite
change the project design which is quite
change the project design which is quite important when you are experimenting
but the good news is that this is
but the good news is that this is starting to change there is more and
starting to change there is more and
starting to change there is more and more interest in experimentation and
more interest in experimentation and
more interest in experimentation and evidence-based decisions and pioneering
evidence-based decisions and pioneering
evidence-based decisions and pioneering work is being done by institutions such
work is being done by institutions such
work is being done by institutions such as MIT is j-pal or innovations for
as MIT is j-pal or innovations for
as MIT is j-pal or innovations for poverty action so this is an example so
poverty action so this is an example so
poverty action so this is an example so out of a 2,400 candidates to become a
out of a 2,400 candidates to become a
out of a 2,400 candidates to become a community health workers 300 were
community health workers 300 were
community health workers 300 were recruited through two different methods
recruited through two different methods
recruited through two different methods and one recruiting message emphasize the
and one recruiting message emphasize the
and one recruiting message emphasize the social incentive so basically if you
social incentive so basically if you
social incentive so basically if you join this community health workers group
join this community health workers group
join this community health workers group you will be able to serve the community
you will be able to serve the community
you will be able to serve the community so it’s a social incentive and the other
so it’s a social incentive and the other
so it’s a social incentive and the other group was recruited through more of a
group was recruited through more of a
group was recruited through more of a carrier incentive so if you join this
carrier incentive so if you join this
carrier incentive so if you join this community health workers group you
community health workers group you
community health workers group you actually can develop your own skills and
actually can develop your own skills and
actually can develop your own skills and after a year a study was conducted and
after a year a study was conducted and
after a year a study was conducted and compared this the performance of these
compared this the performance of these
compared this the performance of these two groups of community health workers
two groups of community health workers
two groups of community health workers what they found was that those who were
what they found was that those who were
what they found was that those who were recruited through the carrier incentive
recruited through the carrier incentive
recruited through the carrier incentive messages performed a lot better
messages performed a lot better
messages performed a lot better they organized 30% more a household
they organized 30% more a household
they organized 30% more a household visit and they also organized twice as
visit and they also organized twice as
visit and they also organized twice as many community meetings compared to its
many community meetings compared to its
many community meetings compared to its counterpart now some of the
counterpart now some of the
counterpart now some of the practitioners here may ask right this is
practitioners here may ask right this is
practitioners here may ask right this is quite important but we are doers we are
quite important but we are doers we are
quite important but we are doers we are not researchers so what do we do I think
not researchers so what do we do I think
not researchers so what do we do I think the answer lies in a lean
the answer lies in a lean
the answer lies in a lean experimentation so let me rewind a bit
experimentation so let me rewind a bit
experimentation so let me rewind a bit seven years ago I left the United
seven years ago I left the United
seven years ago I left the United Nations and then started my own
Nations and then started my own
Nations and then started my own organization called Kopernik and
organization called Kopernik and
organization called Kopernik and Kopernik is named after Nicholas
Kopernik is named after Nicholas
Kopernik is named after Nicholas Copernicus who himself was very fond of
Copernicus who himself was very fond of
Copernicus who himself was very fond of experimentation and just like he changed
experimentation and just like he changed
experimentation and just like he changed the way that we see the world we wanted
the way that we see the world we wanted
the way that we see the world we wanted to
to
to the changed the way that development
the changed the way that development
the changed the way that development assistance is delivered and one of the
assistance is delivered and one of the
assistance is delivered and one of the things that we have been doing is to
things that we have been doing is to
things that we have been doing is to distribute a very simple yet
distribute a very simple yet
distribute a very simple yet life-changing technologies to the poor
life-changing technologies to the poor
life-changing technologies to the poor in the last mile so technology such as
in the last mile so technology such as
in the last mile so technology such as clean cookstoves or water filters or
clean cookstoves or water filters or
clean cookstoves or water filters or solar lights and these are distributed
solar lights and these are distributed
solar lights and these are distributed through a sales agent that we recruit
through a sales agent that we recruit
through a sales agent that we recruit and train and after a while we notice
and train and after a while we notice
and train and after a while we notice that there is a huge difference in terms
that there is a huge difference in terms
that there is a huge difference in terms of the sales performance among the sales
of the sales performance among the sales
of the sales performance among the sales agent and we wanted to find out what is
agent and we wanted to find out what is
agent and we wanted to find out what is contributing to that because we wanted
contributing to that because we wanted
contributing to that because we wanted to reach more people in a more effective
to reach more people in a more effective
to reach more people in a more effective way so we took a sample of a
way so we took a sample of a
way so we took a sample of a high-performing sales agent and compared
high-performing sales agent and compared
high-performing sales agent and compared against the general overall sales agent
against the general overall sales agent
against the general overall sales agent and we looked at the things like gender
and we looked at the things like gender
and we looked at the things like gender or age or the previous background in
or age or the previous background in
or age or the previous background in business and none of these were
business and none of these were
business and none of these were differentiating between the high
differentiating between the high
differentiating between the high performing sales agent and an ordinary
performing sales agent and an ordinary
performing sales agent and an ordinary sales agent but when we looked at the
sales agent but when we looked at the
sales agent but when we looked at the mobility we found an interesting
mobility we found an interesting
mobility we found an interesting correlations so those agent who have
correlations so those agent who have
correlations so those agent who have easy access to motorbike was selling
easy access to motorbike was selling
easy access to motorbike was selling twice as limited as many technologies
twice as limited as many technologies
twice as limited as many technologies compared to those without access to the
compared to those without access to the
compared to those without access to the motorbike so through this very simple
motorbike so through this very simple
motorbike so through this very simple experiment we changed our recruitment
experiment we changed our recruitment
experiment we changed our recruitment strategies of the sales agent and this
strategies of the sales agent and this
strategies of the sales agent and this took us only three days of our staff
took us only three days of our staff
took us only three days of our staff time another experiment that another
time another experiment that another
time another experiment that another example of a lean experimentation we
example of a lean experimentation we
example of a lean experimentation we conduct is around grain storage
conduct is around grain storage
conduct is around grain storage this is sorghum which is the fifth
this is sorghum which is the fifth
this is sorghum which is the fifth common a most common grain in the world
common a most common grain in the world
common a most common grain in the world and this is becoming quite popular in
and this is becoming quite popular in
and this is becoming quite popular in eastern part of Indonesia
eastern part of Indonesia
eastern part of Indonesia where many of our activities are
where many of our activities are
where many of our activities are concentrated around and this becomes
concentrated around and this becomes
concentrated around and this becomes very popular because it grows without
very popular because it grows without
very popular because it grows without much water so it’s appropriate for the
much water so it’s appropriate for the
much water so it’s appropriate for the dry area and it’s very very nutritious
dry area and it’s very very nutritious
dry area and it’s very very nutritious but the challenge is that after the
but the challenge is that after the
but the challenge is that after the harvest farmers typically stores these
harvest farmers typically stores these
harvest farmers typically stores these grains in a very simple bag like this
grains in a very simple bag like this
grains in a very simple bag like this this is just a wooden plastic bags and
this is just a wooden plastic bags and
this is just a wooden plastic bags and soon this grain will be infested with
soon this grain will be infested with
soon this grain will be infested with weevils and up to 70% of the grain is
weevils and up to 70% of the grain is
weevils and up to 70% of the grain is eaten up so it is a huge waste of
eaten up so it is a huge waste of
eaten up so it is a huge waste of resources and also a huge loss of
resources and also a huge loss of
resources and also a huge loss of potential income for these smallholder
potential income for these smallholder
potential income for these smallholder farmers so we wanted to do something
farmers so we wanted to do something
farmers so we wanted to do something about it and we did some quick research
about it and we did some quick research
about it and we did some quick research and found that maize has a very same
and found that maize has a very same
and found that maize has a very same issue and some organizations are
issue and some organizations are
issue and some organizations are advocating for an airtight or hermetic
advocating for an airtight or hermetic
advocating for an airtight or hermetic storage method and we were not sure if
storage method and we were not sure if
storage method and we were not sure if the same solution would work with
the same solution would work with
the same solution would work with sorghum because it’s after all it’s a
sorghum because it’s after all it’s a
sorghum because it’s after all it’s a different grain so we wanted to test if
different grain so we wanted to test if
different grain so we wanted to test if this hermetic storage indeed in
this hermetic storage indeed in
this hermetic storage indeed in decreases the number of weevils the idea
decreases the number of weevils the idea
decreases the number of weevils the idea of the hermetic storage is that you you
of the hermetic storage is that you you
of the hermetic storage is that you you basically limit the flow of oxygen so
basically limit the flow of oxygen so
basically limit the flow of oxygen so eventually the weevil will run out of
eventually the weevil will run out of
eventually the weevil will run out of oxygen so they will die and when they
oxygen so they will die and when they
oxygen so they will die and when they die of course they will stop eating the
die of course they will stop eating the
die of course they will stop eating the grain so we did this experiment so we
grain so we did this experiment so we
grain so we did this experiment so we had this bag of sorghum in a traditional
had this bag of sorghum in a traditional
had this bag of sorghum in a traditional bag and the other one is contain use the
bag and the other one is contain use the
bag and the other one is contain use the hermetic bag inside the traditional box
hermetic bag inside the traditional box
hermetic bag inside the traditional box so it’s double double is sealed and this
so it’s double double is sealed and this
so it’s double double is sealed and this each contained 25 kilogram of solvent
each contained 25 kilogram of solvent
each contained 25 kilogram of solvent grain and we put them side-by-side for
grain and we put them side-by-side for
grain and we put them side-by-side for three months and literally we go back
three months and literally we go back
three months and literally we go back every month and have three scoops out of
every month and have three scoops out of
every month and have three scoops out of and then spread it and then literally
and then spread it and then literally
and then spread it and then literally count the number of weevils by hand and
count the number of weevils by hand and
count the number of weevils by hand and after three months as expected this
after three months as expected this
after three months as expected this traditional bag was infested with
traditional bag was infested with
traditional bag was infested with weevils there are close to 400 weevils
weevils there are close to 400 weevils
weevils there are close to 400 weevils it took a long time to count and while
it took a long time to count and while
it took a long time to count and while this hermetic bag had less than a third
this hermetic bag had less than a third
this hermetic bag had less than a third of the weevils now this is not a perfect
of the weevils now this is not a perfect
of the weevils now this is not a perfect solution but it is reducing the number
solution but it is reducing the number
solution but it is reducing the number of weevils significantly and this
of weevils significantly and this
of weevils significantly and this experimentation costs less than $2,000
experimentation costs less than $2,000
experimentation costs less than $2,000 for us so I’m sharing these very
for us so I’m sharing these very
for us so I’m sharing these very seemingly mundane and simple examples
seemingly mundane and simple examples
seemingly mundane and simple examples because I believe that even a complex
because I believe that even a complex
because I believe that even a complex development programs can be broken down
development programs can be broken down
development programs can be broken down into simple activities and then can be
into simple activities and then can be
into simple activities and then can be tested through lean experimentation so
tested through lean experimentation so
tested through lean experimentation so the message today is that we should be
the message today is that we should be
the message today is that we should be doing more of experimentation be it
doing more of experimentation be it
doing more of experimentation be it rigorous or lean and really trying to
rigorous or lean and really trying to
rigorous or lean and really trying to find a smart solutions that really work
find a smart solutions that really work
find a smart solutions that really work and if we do that collectively soon we
and if we do that collectively soon we
and if we do that collectively soon we will be able to achieve the sustainable
will be able to achieve the sustainable
will be able to achieve the sustainable development goals more effectively and
development goals more effectively and
development goals more effectively and we will also be able to answer the
we will also be able to answer the
we will also be able to answer the question of which one it is cows cash or
question of which one it is cows cash or
question of which one it is cows cash or chickens that are most effective in
chickens that are most effective in
chickens that are most effective in increasing the income of the poor thank
increasing the income of the poor thank
increasing the income of the poor thank you
you
you [Applause]
”
Be First to Comment