and now we’re going to talk about how agile esters might be killing the planet
agile esters might be killing the planet
agile esters might be killing the planet who was here for Jazz’s talked earlier
who was here for Jazz’s talked earlier
who was here for Jazz’s talked earlier on value hands up most of you and who
on value hands up most of you and who
on value hands up most of you and who was here for Josh’s talk okay there’s
was here for Josh’s talk okay there’s
was here for Josh’s talk okay there’s going to be some overlap hopefully I can
going to be some overlap hopefully I can
going to be some overlap hopefully I can get into a little more depth because I
get into a little more depth because I
get into a little more depth because I came along to their sessions to make
came along to their sessions to make
came along to their sessions to make sure that we weren’t going to have too
sure that we weren’t going to have too
sure that we weren’t going to have too much overlap so similar theme let’s go
much overlap so similar theme let’s go
much overlap so similar theme let’s go through it agile esters are killing the
through it agile esters are killing the
through it agile esters are killing the planet so agile lean scrum Kanban they
planet so agile lean scrum Kanban they
planet so agile lean scrum Kanban they have all the answers do they are they
have all the answers do they are they
have all the answers do they are they really solving our problems or are they
really solving our problems or are they
really solving our problems or are they just helping us to build the wrong thing
just helping us to build the wrong thing
just helping us to build the wrong thing writer there’s been a really big focus
writer there’s been a really big focus
writer there’s been a really big focus on let’s make our development work
on let’s make our development work
on let’s make our development work better the issue of that is we’re
better the issue of that is we’re
better the issue of that is we’re delivering all these milestones we’re
delivering all these milestones we’re
delivering all these milestones we’re delivering working software so the great
delivering working software so the great
delivering working software so the great promise of agile was that hey you know
promise of agile was that hey you know
promise of agile was that hey you know what for so terrible because they look
what for so terrible because they look
what for so terrible because they look at phases we say that we’ve done
at phases we say that we’ve done
at phases we say that we’ve done planning we’ve done delivery we’re
planning we’ve done delivery we’re
planning we’ve done delivery we’re making all this progress and we sort of
making all this progress and we sort of
making all this progress and we sort of trashed that in the agile community we
trashed that in the agile community we
trashed that in the agile community we said well hey that’s that’s useless or
said well hey that’s that’s useless or
said well hey that’s that’s useless or we could have as a pile of documents we
we could have as a pile of documents we
we could have as a pile of documents we might not have anything useful to show
might not have anything useful to show
might not have anything useful to show for it so instead we’re going to quickly
for it so instead we’re going to quickly
for it so instead we’re going to quickly build working software show it to our
build working software show it to our
build working software show it to our clients let’s see how that’s turned out
clients let’s see how that’s turned out
clients let’s see how that’s turned out for us in fact all they did was push
for us in fact all they did was push
for us in fact all they did was push more through the system faster so now
more through the system faster so now
more through the system faster so now what we’ve done is we’ve gone from a
what we’ve done is we’ve gone from a
what we’ve done is we’ve gone from a model where we tracked activity right we
model where we tracked activity right we
model where we tracked activity right we track that we’d actually finished some
track that we’d actually finished some
track that we’d actually finished some work activity we’ve gone to tracking
work activity we’ve gone to tracking
work activity we’ve gone to tracking outputs we have built more stuff but is
outputs we have built more stuff but is
outputs we have built more stuff but is that actually a good thing
that actually a good thing
that actually a good thing so I hear so many people saying we get
so I hear so many people saying we get
so I hear so many people saying we get our products out on time and on budget
our products out on time and on budget
our products out on time and on budget does it matter if it’s the wrong thing
does it matter if it’s the wrong thing
does it matter if it’s the wrong thing for the wrong people
for the wrong people
for the wrong people a dog takes a machine gun to product
a dog takes a machine gun to product
a dog takes a machine gun to product development so what we’re effectively
development so what we’re effectively
development so what we’re effectively doing is taking lots of shots and hoping
doing is taking lots of shots and hoping
doing is taking lots of shots and hoping that some of them hit our target the
that some of them hit our target the
that some of them hit our target the issue of that is we take out a lot of
issue of that is we take out a lot of
issue of that is we take out a lot of innocent bystanders on the way and
innocent bystanders on the way and
innocent bystanders on the way and effectively we could be creating a whole
effectively we could be creating a whole
effectively we could be creating a whole lot of waste so output is not
lot of waste so output is not
lot of waste so output is not necessarily a good thing
necessarily a good thing
necessarily a good thing right what we’ve done is effectively by
right what we’ve done is effectively by
right what we’ve done is effectively by building more increased the odds of
building more increased the odds of
building more increased the odds of success by using the machine gun whereas
success by using the machine gun whereas
success by using the machine gun whereas what we really want is a very targeted
what we really want is a very targeted
what we really want is a very targeted sniper gun
sniper gun
sniper gun I was at a meeting once and the VP’s
I was at a meeting once and the VP’s
I was at a meeting once and the VP’s stood up and said hey team it’s so great
stood up and said hey team it’s so great
stood up and said hey team it’s so great we delivered 35 features on-time and
we delivered 35 features on-time and
we delivered 35 features on-time and on-budget and everyone’s clapping away
on-budget and everyone’s clapping away
on-budget and everyone’s clapping away now at the back saying you have it did
now at the back saying you have it did
now at the back saying you have it did we make money did we deliver results and
we make money did we deliver results and
we make money did we deliver results and in fact would to have done the same job
capers Jones had an interesting
capers Jones had an interesting statistic in one of his books on
statistic in one of his books on
statistic in one of his books on estimation he said that 30 development
estimation he said that 30 development
estimation he said that 30 development accounted for about 30% of product
accounted for about 30% of product
accounted for about 30% of product development costs which means there’s
development costs which means there’s
development costs which means there’s 70% surrounding it over the years I’ve
70% surrounding it over the years I’ve
70% surrounding it over the years I’ve been doing agile for a very long time
been doing agile for a very long time
been doing agile for a very long time and various forms I found that you know
and various forms I found that you know
and various forms I found that you know it’s not their jobs being a bad thing
it’s not their jobs being a bad thing
it’s not their jobs being a bad thing it’s cleaned up a lot of the mess in the
it’s cleaned up a lot of the mess in the
it’s cleaned up a lot of the mess in the middle zone that development zone and
middle zone that development zone and
middle zone that development zone and the great promise of frameworks like
the great promise of frameworks like
the great promise of frameworks like scrum is to make the dysfunctions
scrum is to make the dysfunctions
scrum is to make the dysfunctions visible so what we’ve now found is that
visible so what we’ve now found is that
visible so what we’ve now found is that we clean that middle mess up we’re
we clean that middle mess up we’re
we clean that middle mess up we’re making stuff get through the pipe
making stuff get through the pipe
making stuff get through the pipe quicker and instead what we found is
quicker and instead what we found is
quicker and instead what we found is that what’s coming in the requests and
that what’s coming in the requests and
that what’s coming in the requests and what’s getting out are the wrong things
so instead of building the wrong thing
so instead of building the wrong thing writer so instead of saying you know the
writer so instead of saying you know the
writer so instead of saying you know the big questions mean which is better
big questions mean which is better
big questions mean which is better Kanban or scrum completely the wrong
Kanban or scrum completely the wrong
Kanban or scrum completely the wrong question it does not matter if you’re
question it does not matter if you’re
question it does not matter if you’re building the wrong thing who are doing
building the wrong thing who are doing
building the wrong thing who are doing is gold-plating the development
is gold-plating the development
is gold-plating the development mechanisms we’re not actually improving
mechanisms we’re not actually improving
mechanisms we’re not actually improving the product itself today we want to
the product itself today we want to
the product itself today we want to build the right thing so I’m going to
build the right thing so I’m going to
build the right thing so I’m going to talk to you a lot about building
talk to you a lot about building
talk to you a lot about building outcomes over outputs now people who now
outcomes over outputs now people who now
outcomes over outputs now people who now have Tom Gill
have Tom Gill
have Tom Gill back in the 60s he’s been saying this
back in the 60s he’s been saying this
back in the 60s he’s been saying this he’s been saying hey we need to measure
he’s been saying hey we need to measure
he’s been saying hey we need to measure value and it’s taken a long time for the
value and it’s taken a long time for the
value and it’s taken a long time for the rest of the world to start catching up
rest of the world to start catching up
rest of the world to start catching up Tom’s a brilliant guy almost too smart I
Tom’s a brilliant guy almost too smart I
Tom’s a brilliant guy almost too smart I think it took this long for everyone to
think it took this long for everyone to
think it took this long for everyone to finish competitive engineering book but
finish competitive engineering book but
finish competitive engineering book but what Tom’s been saying for a long time
what Tom’s been saying for a long time
what Tom’s been saying for a long time is you need to measure value nothing
is you need to measure value nothing
is you need to measure value nothing else matters his whole models based on
else matters his whole models based on
else matters his whole models based on Evo which is little one-week iterative
Evo which is little one-week iterative
Evo which is little one-week iterative steps to get feedback and to look at
steps to get feedback and to look at
steps to get feedback and to look at your progress measures so I’ve been
your progress measures so I’ve been
your progress measures so I’ve been working Tom a long time to figure out
working Tom a long time to figure out
working Tom a long time to figure out you know how do we make this more used
you know how do we make this more used
you know how do we make this more used and more acceptable to the world so
and more acceptable to the world so
and more acceptable to the world so that’s been a bit of a lynchpin in what
that’s been a bit of a lynchpin in what
that’s been a bit of a lynchpin in what I’ve been doing to give a bit of
I’ve been doing to give a bit of
I’ve been doing to give a bit of background in the late 90s we were doing
background in the late 90s we were doing
background in the late 90s we were doing some X
some X
some X he just very technically focused 2003 I
he just very technically focused 2003 I
he just very technically focused 2003 I was in a San Francisco startup I came in
was in a San Francisco startup I came in
was in a San Francisco startup I came in to run the product team and then they
to run the product team and then they
to run the product team and then they ended up giving me everything they gave
ended up giving me everything they gave
ended up giving me everything they gave me product development operations etc
me product development operations etc
me product development operations etc and they said in six months we want to
and they said in six months we want to
and they said in six months we want to take the company public so I want to do
take the company public so I want to do
take the company public so I want to do a big IPO and at this point this is a
a big IPO and at this point this is a
a big IPO and at this point this is a company that has like six warring VPS
company that has like six warring VPS
company that has like six warring VPS this is like Afghanistan and practice we
this is like Afghanistan and practice we
this is like Afghanistan and practice we have everybody disagreeing we have
have everybody disagreeing we have
have everybody disagreeing we have separate lists think of those as six
separate lists think of those as six
separate lists think of those as six different back logs for all the agile
different back logs for all the agile
different back logs for all the agile people and we don’t have that bigger
people and we don’t have that bigger
people and we don’t have that bigger development team so we have everyone
development team so we have everyone
development team so we have everyone building lots of things lots of work and
building lots of things lots of work and
building lots of things lots of work and progress nothing actually getting done
progress nothing actually getting done
progress nothing actually getting done so when I and I said ok well let’s start
so when I and I said ok well let’s start
so when I and I said ok well let’s start cleaning up this mess and we started to
cleaning up this mess and we started to
cleaning up this mess and we started to use a bit of lightweight sort of scrum I
use a bit of lightweight sort of scrum I
use a bit of lightweight sort of scrum I started bringing in lean thinking lean
started bringing in lean thinking lean
started bringing in lean thinking lean portfolio thinking and the number one
portfolio thinking and the number one
portfolio thinking and the number one standout item was that we were on this
standout item was that we were on this
standout item was that we were on this massive feature farming Drive what we
massive feature farming Drive what we
massive feature farming Drive what we were trying to do is put so many
were trying to do is put so many
were trying to do is put so many features out at Lightspeed and we were
features out at Lightspeed and we were
features out at Lightspeed and we were going to make lots of money
going to make lots of money
going to make lots of money who seen those proposals which come
who seen those proposals which come
who seen those proposals which come through all of these features each
through all of these features each
through all of these features each features worth X amount of money I’m
features worth X amount of money I’m
features worth X amount of money I’m having a big argument with my co-authors
having a big argument with my co-authors
having a big argument with my co-authors of the scrum primer there’s a little
of the scrum primer there’s a little
of the scrum primer there’s a little column in the backgroud backlog which
column in the backgroud backlog which
column in the backgroud backlog which I’d sort of missed the first time which
I’d sort of missed the first time which
I’d sort of missed the first time which assigns value to each product backlog
assigns value to each product backlog
assigns value to each product backlog item I’m like how do you know you know
item I’m like how do you know you know
item I’m like how do you know you know what’s the value of something like a
what’s the value of something like a
what’s the value of something like a logon that’s way too detailed and so
logon that’s way too detailed and so
logon that’s way too detailed and so these people thought that each feature
these people thought that each feature
these people thought that each feature was worth money I looked at I thought
was worth money I looked at I thought
was worth money I looked at I thought that that’s not our problem at all so I
that that’s not our problem at all so I
that that’s not our problem at all so I went up to the board of directors and I
went up to the board of directors and I
went up to the board of directors and I looked at them I said ok so who goes to
looked at them I said ok so who goes to
looked at them I said ok so who goes to Amazon and eBay and I’ll ask you all
Amazon and eBay and I’ll ask you all
Amazon and eBay and I’ll ask you all this who goes and uses Amazon and eBay
this who goes and uses Amazon and eBay
this who goes and uses Amazon and eBay regularly like at least one for once a
regularly like at least one for once a
regularly like at least one for once a month right sure we do all the shopping
month right sure we do all the shopping
month right sure we do all the shopping there because we don’t like to get up
there because we don’t like to get up
there because we don’t like to get up from programming so I said okay do you
from programming so I said okay do you
from programming so I said okay do you who was one of the people who uses it a
who was one of the people who uses it a
who was one of the people who uses it a lot who would say they like a hardcore
lot who would say they like a hardcore
lot who would say they like a hardcore user of Amazon or Ebay hands up okay ask
user of Amazon or Ebay hands up okay ask
user of Amazon or Ebay hands up okay ask you do you go to Amazon or Ebay every
you do you go to Amazon or Ebay every
you do you go to Amazon or Ebay every week because you’re wanting to see
week because you’re wanting to see
week because you’re wanting to see what’s the new feature that they’re
what’s the new feature that they’re
what’s the new feature that they’re going to launch for you to use no what
going to launch for you to use no what
going to launch for you to use no what do you go there for for the books and
do you go there for for the books and
do you go there for for the books and what else
what else
what else eBay
eBay
eBay No so you go there for the books you
No so you go there for the books you
No so you go there for the books you want to buy something it’s pretty simple
want to buy something it’s pretty simple
want to buy something it’s pretty simple if your Amazon simple value statement I
if your Amazon simple value statement I
if your Amazon simple value statement I want to buy something if you’re someone
want to buy something if you’re someone
want to buy something if you’re someone like Skype simple value statement I want
like Skype simple value statement I want
like Skype simple value statement I want to make a call and yet most of these
to make a call and yet most of these
to make a call and yet most of these products need to go out of their way to
products need to go out of their way to
products need to go out of their way to put as many obstacles and features in
put as many obstacles and features in
put as many obstacles and features in your way as they possibly can
your way as they possibly can
your way as they possibly can so the more features you send out the
so the more features you send out the
so the more features you send out the worse it actually gets so this was that
worse it actually gets so this was that
worse it actually gets so this was that problem we’re on this feature farming
problem we’re on this feature farming
problem we’re on this feature farming exercise and in fact that wasn’t a
exercise and in fact that wasn’t a
exercise and in fact that wasn’t a problem at all so I said we’re not going
problem at all so I said we’re not going
problem at all so I said we’re not going to build lots of new features we’re
to build lots of new features we’re
to build lots of new features we’re going to perform improve the performance
going to perform improve the performance
going to perform improve the performance and usability all the qualities of our
and usability all the qualities of our
and usability all the qualities of our website our revenue curve went from
website our revenue curve went from
website our revenue curve went from being small hardly going up until so
being small hardly going up until so
being small hardly going up until so steeply vertical than in six months time
steeply vertical than in six months time
steeply vertical than in six months time when we were ready to go public the
when we were ready to go public the
when we were ready to go public the bankers called us in and said stop
bankers called us in and said stop
bankers called us in and said stop you’re not allowed to go public I like
you’re not allowed to go public I like
you’re not allowed to go public I like what we’ve done a great job we’re almost
what we’ve done a great job we’re almost
what we’ve done a great job we’re almost down to zero defects we’d actually
down to zero defects we’d actually
down to zero defects we’d actually halfway through being actually replacing
halfway through being actually replacing
halfway through being actually replacing legacy code where the best XP is on the
legacy code where the best XP is on the
legacy code where the best XP is on the planet everything was going so well and
planet everything was going so well and
planet everything was going so well and they said I said look at the revenue
they said I said look at the revenue
they said I said look at the revenue curve and they said that’s the problem I
curve and they said that’s the problem I
curve and they said that’s the problem I don’t know if anyone’s ever tried to
don’t know if anyone’s ever tried to
don’t know if anyone’s ever tried to take a company public why would our
take a company public why would our
take a company public why would our revenue curve be a problem because it
revenue curve be a problem because it
revenue curve be a problem because it goes up any ideas okay be careful of
goes up any ideas okay be careful of
goes up any ideas okay be careful of this if you get really good at what what
this if you get really good at what what
this if you get really good at what what you’re doing the problem is that it’s
you’re doing the problem is that it’s
you’re doing the problem is that it’s not sustainable they’re like no one goes
not sustainable they’re like no one goes
not sustainable they’re like no one goes into a vertical cliff jump looks like
into a vertical cliff jump looks like
into a vertical cliff jump looks like you’re actually fiddling the books so I
you’re actually fiddling the books so I
you’re actually fiddling the books so I had to literally go back to my team and
had to literally go back to my team and
had to literally go back to my team and say okay guys stop what you’re doing
say okay guys stop what you’re doing
say okay guys stop what you’re doing let’s actually tell that off get a
let’s actually tell that off get a
let’s actually tell that off get a sustainable curve so we could take the
sustainable curve so we could take the
sustainable curve so we could take the company public so that’s the dramatic
company public so that’s the dramatic
company public so that’s the dramatic difference that stuff we’re hearing
difference that stuff we’re hearing
difference that stuff we’re hearing today about lean startups etc that’s
today about lean startups etc that’s
today about lean startups etc that’s what matters we were lucky enough to
what matters we were lucky enough to
what matters we were lucky enough to have the chief scientist from Amazon at
have the chief scientist from Amazon at
have the chief scientist from Amazon at the time Amazon it’s a company that
the time Amazon it’s a company that
the time Amazon it’s a company that drives themselves by data everything
drives themselves by data everything
drives themselves by data everything they do they test it’s amazing Amazon in
they do they test it’s amazing Amazon in
they do they test it’s amazing Amazon in the early days if anyone used it back
the early days if anyone used it back
the early days if anyone used it back then but it used to take about three
then but it used to take about three
then but it used to take about three weeks to get a book two to three weeks
weeks to get a book two to three weeks
weeks to get a book two to three weeks right you’d order it get a book some
right you’d order it get a book some
right you’d order it get a book some went up and said well how can we get a
went up and said well how can we get a
went up and said well how can we get a book and maybe you know let’s get it
book and maybe you know let’s get it
book and maybe you know let’s get it down to a week three days one day then
down to a week three days one day then
down to a week three days one day then who had the brilliant idea you know some
who had the brilliant idea you know some
who had the brilliant idea you know some wants to get a book would you like to
wants to get a book would you like to
wants to get a book would you like to get a book in under one
get a book in under one
get a book in under one it cause right it’s the worst worst
it cause right it’s the worst worst
it cause right it’s the worst worst product in the world for me I’m sitting
product in the world for me I’m sitting
product in the world for me I’m sitting at something like a conference I’m there
at something like a conference I’m there
at something like a conference I’m there and I’m like one click by it’s
and I’m like one click by it’s
and I’m like one click by it’s downloading to my Kindle fantastic so
downloading to my Kindle fantastic so
downloading to my Kindle fantastic so they get the whole idea that’s all about
they get the whole idea that’s all about
they get the whole idea that’s all about that user experience so one thing that
that user experience so one thing that
that user experience so one thing that I’ve been working a lot with lately is
I’ve been working a lot with lately is
I’ve been working a lot with lately is guy called Range Rover in the US and
guy called Range Rover in the US and
guy called Range Rover in the US and I’ve been creating something called the
I’ve been creating something called the
I’ve been creating something called the target outcomes framework so we really
target outcomes framework so we really
target outcomes framework so we really want to drive home you’ve got to go for
want to drive home you’ve got to go for
want to drive home you’ve got to go for outcomes not outputs okay something that
outcomes not outputs okay something that
outcomes not outputs okay something that we need to re-educate our clients in and
we need to re-educate our clients in and
we need to re-educate our clients in and as you heard joshing around talking
as you heard joshing around talking
as you heard joshing around talking about earlier velocity all these
about earlier velocity all these
about earlier velocity all these estimator measures they are just getting
estimator measures they are just getting
estimator measures they are just getting in the way of what we need to do which
in the way of what we need to do which
in the way of what we need to do which is to deliver value faster so think of
is to deliver value faster so think of
is to deliver value faster so think of outcomes outcomes are the results we
outcomes outcomes are the results we
outcomes outcomes are the results we expect the the destination we want to
expect the the destination we want to
expect the the destination we want to get to options are once you create
get to options are once you create
get to options are once you create outcomes these are the results there’s
outcomes these are the results there’s
outcomes these are the results there’s many ways to get there so I’m going to
many ways to get there so I’m going to
many ways to get there so I’m going to talk through how to get started I wanna
talk through how to get started I wanna
talk through how to get started I wanna give you a little bit of practical
give you a little bit of practical
give you a little bit of practical information we’ve only got a little bit
information we’ve only got a little bit
information we’ve only got a little bit less than an hour this is something
less than an hour this is something
less than an hour this is something we’re going to have a book coming out on
we’re going to have a book coming out on
we’re going to have a book coming out on this and some more detailed guidance
this and some more detailed guidance
this and some more detailed guidance what I found
what I found
what I found I love the Lean Startup book but when
I love the Lean Startup book but when
I love the Lean Startup book but when you go through it it sounds fantastic
you go through it it sounds fantastic
you go through it it sounds fantastic until you try to do it and if you’ve got
until you try to do it and if you’ve got
until you try to do it and if you’ve got a little website that’s okay but when
a little website that’s okay but when
a little website that’s okay but when you’re running this at scale I run it
you’re running this at scale I run it
you’re running this at scale I run it with company who has literally millions
with company who has literally millions
with company who has literally millions of users and very complicated product
of users and very complicated product
of users and very complicated product sets if you’re doing this at enterprise
sets if you’re doing this at enterprise
sets if you’re doing this at enterprise levels I find that it starts falling
levels I find that it starts falling
levels I find that it starts falling down they can be little there’s not a
down they can be little there’s not a
down they can be little there’s not a lot of information out there on how to
lot of information out there on how to
lot of information out there on how to get this going big scale one thing you
get this going big scale one thing you
get this going big scale one thing you like to do there’s a lot of lean people
like to do there’s a lot of lean people
like to do there’s a lot of lean people in the room who knows about a3 thinking
in the room who knows about a3 thinking
in the room who knows about a3 thinking coming out of Toyota ok not a lot of
coming out of Toyota ok not a lot of
coming out of Toyota ok not a lot of lean people lean gets kind of
lean people lean gets kind of
lean people lean gets kind of bastardized in our community it tends to
bastardized in our community it tends to
bastardized in our community it tends to be about reducing waste it’s all about
be about reducing waste it’s all about
be about reducing waste it’s all about it in a way trying to make things more
it in a way trying to make things more
it in a way trying to make things more efficient great ideas but one of the key
efficient great ideas but one of the key
efficient great ideas but one of the key tenants is about this idea of learning
tenants is about this idea of learning
tenants is about this idea of learning now a3 thinking there’s some good books
now a3 thinking there’s some good books
now a3 thinking there’s some good books on it there’s a Sobek book on a3
on it there’s a Sobek book on a3
on it there’s a Sobek book on a3 thinking there’s another managing to
thinking there’s another managing to
thinking there’s another managing to learn book and it’s a very deep
learn book and it’s a very deep
learn book and it’s a very deep discipline it makes you think about
discipline it makes you think about
discipline it makes you think about systemic reasons and why we have
systemic reasons and why we have
systemic reasons and why we have problems or causes for things so this is
problems or causes for things so this is
problems or causes for things so this is actually something we’ve blended into
actually something we’ve blended into
actually something we’ve blended into model so that we understand the context
model so that we understand the context
model so that we understand the context the users the environment extremely
the users the environment extremely
the users the environment extremely deeply before we start deciding where to
deeply before we start deciding where to
deeply before we start deciding where to go I think sometimes we sort of leapfrog
go I think sometimes we sort of leapfrog
go I think sometimes we sort of leapfrog that we jump straight into we want to
that we jump straight into we want to
that we jump straight into we want to get here but actually we don’t know the
get here but actually we don’t know the
get here but actually we don’t know the problems who don’t know the reasons why
problems who don’t know the reasons why
problems who don’t know the reasons why so one thing you can do we’re hearing
so one thing you can do we’re hearing
so one thing you can do we’re hearing about user experience I’m going to tell
about user experience I’m going to tell
about user experience I’m going to tell you to be the method actor of user
you to be the method actor of user
you to be the method actor of user experience you know robert de niro these
experience you know robert de niro these
experience you know robert de niro these famous actors go out they actually have
famous actors go out they actually have
famous actors go out they actually have a movie where they’re going to be
a movie where they’re going to be
a movie where they’re going to be working behind the counter in
working behind the counter in
working behind the counter in supermarket they get a job in the
supermarket they get a job in the
supermarket they get a job in the counter of the supermarket so never ask
counter of the supermarket so never ask
counter of the supermarket so never ask you users what they want and in fact the
you users what they want and in fact the
you users what they want and in fact the worst thing you’d ever do is ask your
worst thing you’d ever do is ask your
worst thing you’d ever do is ask your developers what they think the users
developers what they think the users
developers what they think the users want or even your on-site customer right
want or even your on-site customer right
want or even your on-site customer right they might be million away at miles away
they might be million away at miles away
they might be million away at miles away from your actual users I always love
from your actual users I always love
from your actual users I always love that quote which you probably seen if
that quote which you probably seen if
that quote which you probably seen if I’d asked my customers what they wanted
I’d asked my customers what they wanted
I’d asked my customers what they wanted we would have built faster horses so one
we would have built faster horses so one
we would have built faster horses so one thing you never do is go and get that
thing you never do is go and get that
thing you never do is go and get that product requirements backlog list for
product requirements backlog list for
product requirements backlog list for users you’ll build the wrong thing
users you’ll build the wrong thing
users you’ll build the wrong thing almost guaranteed get out of your chair
almost guaranteed get out of your chair
almost guaranteed get out of your chair and go find out what they need I’ll talk
and go find out what they need I’ll talk
and go find out what they need I’ll talk about more about this later but this can
about more about this later but this can
about more about this later but this can be done very quickly and easily there’s
be done very quickly and easily there’s
be done very quickly and easily there’s a great story I do one of these Silicon
a great story I do one of these Silicon
a great story I do one of these Silicon Valley think tanks they build great
Valley think tanks they build great
Valley think tanks they build great products they were investigating they
products they were investigating they
products they were investigating they got a job to create children’s
got a job to create children’s
got a job to create children’s toothbrush so everybody at that time
toothbrush so everybody at that time
toothbrush so everybody at that time thought children are little people right
thought children are little people right
thought children are little people right little people have little hands we’re
little people have little hands we’re
little people have little hands we’re going to build a little toothbrush so
going to build a little toothbrush so
going to build a little toothbrush so everybody had built these little
everybody had built these little
everybody had built these little toothbrushes right thinking little hands
toothbrushes right thinking little hands
toothbrushes right thinking little hands little people little toothbrush they
little people little toothbrush they
little people little toothbrush they went out and actually watched little
went out and actually watched little
went out and actually watched little people doing their toothbrush stuff and
people doing their toothbrush stuff and
people doing their toothbrush stuff and guess what they found that everyone’s
guess what they found that everyone’s
guess what they found that everyone’s got kids little kids are little
got kids little kids are little
got kids little kids are little toothbrushes they brush their teeth like
toothbrushes they brush their teeth like
toothbrushes they brush their teeth like that right they have no fine motor
that right they have no fine motor
that right they have no fine motor skills so actually completely wrong
skills so actually completely wrong
skills so actually completely wrong solution they came up with those big
solution they came up with those big
solution they came up with those big toothbrushes they were the guys who came
toothbrushes they were the guys who came
toothbrushes they were the guys who came up that invention so the kids could
up that invention so the kids could
up that invention so the kids could grasp them easily
grasp them easily
grasp them easily they went from something like under 20
they went from something like under 20
they went from something like under 20 percent market share to 75 percent
percent market share to 75 percent
percent market share to 75 percent market share surely through going out
market share surely through going out
market share surely through going out and observing the users Yahoo we used to
and observing the users Yahoo we used to
and observing the users Yahoo we used to do a lot of the stuff we’d beg borrow
do a lot of the stuff we’d beg borrow
do a lot of the stuff we’d beg borrow steal go and get friends whatever we
steal go and get friends whatever we
steal go and get friends whatever we could do to get a sin
could do to get a sin
could do to get a sin and to understand them so remember I
and to understand them so remember I
and to understand them so remember I said earlier you want to go to Amazon to
said earlier you want to go to Amazon to
said earlier you want to go to Amazon to get a book you are on skype to make a
get a book you are on skype to make a
get a book you are on skype to make a call those are some of the things that
call those are some of the things that
call those are some of the things that when we go in we both look at those
when we go in we both look at those
when we go in we both look at those goals if you’re in existing products
goals if you’re in existing products
goals if you’re in existing products which a lot of us are could even be your
which a lot of us are could even be your
which a lot of us are could even be your business processes doesn’t matter figure
business processes doesn’t matter figure
business processes doesn’t matter figure out what it is they want to do we use
out what it is they want to do we use
out what it is they want to do we use some of your lean techniques like cycle
some of your lean techniques like cycle
some of your lean techniques like cycle times to measure those what we’re going
times to measure those what we’re going
times to measure those what we’re going to do is keep iterating to make those
to do is keep iterating to make those
to do is keep iterating to make those better and better and better so figure
better and better and better so figure
better and better and better so figure out what the core jobs are and always be
out what the core jobs are and always be
out what the core jobs are and always be very specific about making that as easy
very specific about making that as easy
very specific about making that as easy and quick for our users to do create
and quick for our users to do create
and quick for our users to do create strategies for getting to market at the
strategies for getting to market at the
strategies for getting to market at the right time I saw a Jess had a little
right time I saw a Jess had a little
right time I saw a Jess had a little target with the one person in the middle
target with the one person in the middle
target with the one person in the middle too etc great idea
too etc great idea
too etc great idea be aware that product development is a
be aware that product development is a
be aware that product development is a bit of a black card it’s far more
bit of a black card it’s far more
bit of a black card it’s far more sophisticated if you’re at a startup you
sophisticated if you’re at a startup you
sophisticated if you’re at a startup you have different users at different life
have different users at different life
have different users at different life cycles for example if I’m a startup I
cycles for example if I’m a startup I
cycles for example if I’m a startup I want to get viral thought leaders so I
want to get viral thought leaders so I
want to get viral thought leaders so I want someone like one of our personas we
want someone like one of our personas we
want someone like one of our personas we created I was in the innovation group at
created I was in the innovation group at
created I was in the innovation group at Yahoo we went out we’re looking at a
Yahoo we went out we’re looking at a
Yahoo we went out we’re looking at a youth product and we targeted the
youth product and we targeted the
youth product and we targeted the scholar called Susan because we saw in
scholar called Susan because we saw in
scholar called Susan because we saw in the breaks
the breaks
the breaks she had a tech piece of technology she’d
she had a tech piece of technology she’d
she had a tech piece of technology she’d be looking at it there’d be like eight
be looking at it there’d be like eight
be looking at it there’d be like eight kids all around here watching what Susan
kids all around here watching what Susan
kids all around here watching what Susan was doing so she was the thought leader
was doing so she was the thought leader
was doing so she was the thought leader we thought you know what if we want to
we thought you know what if we want to
we thought you know what if we want to get this product out to many kids we
get this product out to many kids we
get this product out to many kids we want her to be our target that was one
want her to be our target that was one
want her to be our target that was one of our personas after a while we then
of our personas after a while we then
of our personas after a while we then want to go for the kids who are going to
want to go for the kids who are going to
want to go for the kids who are going to be the repeat users so again a lot of
be the repeat users so again a lot of
be the repeat users so again a lot of the skits Center a lot of sophisticated
the skits Center a lot of sophisticated
the skits Center a lot of sophisticated thinking about our users I mean at the
thinking about our users I mean at the
thinking about our users I mean at the moment with some teams I pretty much
moment with some teams I pretty much
moment with some teams I pretty much start with drawing a little stick figure
start with drawing a little stick figure
start with drawing a little stick figure bubble head it’s better to have a user
bubble head it’s better to have a user
bubble head it’s better to have a user than no user so don’t want you to think
than no user so don’t want you to think
than no user so don’t want you to think you have to get super sophisticated but
you have to get super sophisticated but
you have to get super sophisticated but again create empathy and get people to
again create empathy and get people to
again create empathy and get people to drive products our software is used by
drive products our software is used by
drive products our software is used by real people um once we do this analysis
real people um once we do this analysis
real people um once we do this analysis again and going through fairly quickly
again and going through fairly quickly
again and going through fairly quickly but we’ve decided what’s really going on
but we’ve decided what’s really going on
but we’ve decided what’s really going on with our products um I do a lot of get
with our products um I do a lot of get
with our products um I do a lot of get data gathering that’s one of the first
data gathering that’s one of the first
data gathering that’s one of the first things I do at companies one of our
things I do at companies one of our
things I do at companies one of our clients
clients
clients who uses builds a product you’ve
who uses builds a product you’ve
who uses builds a product you’ve probably all have used millions people
probably all have used millions people
probably all have used millions people are using it around the world we went in
are using it around the world we went in
are using it around the world we went in and said okay what are the usage
and said okay what are the usage
and said okay what are the usage patterns what are people really doing
patterns what are people really doing
patterns what are people really doing with this product it’s really
with this product it’s really
with this product it’s really interesting three days later one of the
interesting three days later one of the
interesting three days later one of the product owners came back and said you
product owners came back and said you
product owners came back and said you know how you told me to investigate all
know how you told me to investigate all
know how you told me to investigate all the stuff and get some of this data back
the stuff and get some of this data back
the stuff and get some of this data back he said we found this one feature that’s
he said we found this one feature that’s
he said we found this one feature that’s been on the backlog for a while if we
been on the backlog for a while if we
been on the backlog for a while if we launch it now it will actually work if I
launch it now it will actually work if I
launch it now it will actually work if I said we’re about to launch it tomorrow
said we’re about to launch it tomorrow
said we’re about to launch it tomorrow it’s pretty simple that’s going to save
it’s pretty simple that’s going to save
it’s pretty simple that’s going to save us 12 million pounds a year so
us 12 million pounds a year so
us 12 million pounds a year so immediately by looking at the data they
immediately by looking at the data they
immediately by looking at the data they could figure out what’s going on that
could figure out what’s going on that
could figure out what’s going on that was great that was our first week of
was great that was our first week of
was great that was our first week of consulting at a client so that pretty
consulting at a client so that pretty
consulting at a client so that pretty much paid for us so once we’ve done this
much paid for us so once we’ve done this
much paid for us so once we’ve done this data gathering we’ve understood our
data gathering we’ve understood our
data gathering we’ve understood our users because what we’re trying to do is
users because what we’re trying to do is
users because what we’re trying to do is do deep deep thinking we can use
do deep deep thinking we can use
do deep deep thinking we can use techniques like five why’s value stream
techniques like five why’s value stream
techniques like five why’s value stream mapping what’s really going on once we
mapping what’s really going on once we
mapping what’s really going on once we understand it now we can do the impact
understand it now we can do the impact
understand it now we can do the impact mapping if you know the severity of the
mapping if you know the severity of the
mapping if you know the severity of the problem or the opportunity that helps
problem or the opportunity that helps
problem or the opportunity that helps you immediately prioritize I love those
you immediately prioritize I love those
you immediately prioritize I love those agile classes just a quick aside where
agile classes just a quick aside where
agile classes just a quick aside where they say you can do bubble sorting you
they say you can do bubble sorting you
they say you can do bubble sorting you get two stories you go let’s do a or B
get two stories you go let’s do a or B
get two stories you go let’s do a or B which one goes higher and it’s like
which one goes higher and it’s like
which one goes higher and it’s like seriously is that a way to build a
seriously is that a way to build a
seriously is that a way to build a product you know how do we that makes no
product you know how do we that makes no
product you know how do we that makes no sense to me instead where we’re going to
sense to me instead where we’re going to
sense to me instead where we’re going to get the most impact
get the most impact
get the most impact put on your 8020 thinking caps we’ve all
put on your 8020 thinking caps we’ve all
put on your 8020 thinking caps we’ve all seen those Standish Group reports but
seen those Standish Group reports but
seen those Standish Group reports but are the ones that show that about 20% of
are the ones that show that about 20% of
are the ones that show that about 20% of the features used 80% of the time this
the features used 80% of the time this
the features used 80% of the time this is what we’re doing as the agile
is what we’re doing as the agile
is what we’re doing as the agile community we have a great method for
community we have a great method for
community we have a great method for getting products out quickly optimized
getting products out quickly optimized
getting products out quickly optimized for that 20% and we’ll talk about ways
for that 20% and we’ll talk about ways
for that 20% and we’ll talk about ways you know which is the right 20% but it
you know which is the right 20% but it
you know which is the right 20% but it starts before you waste all your time
starts before you waste all your time
starts before you waste all your time building user stories understand where
building user stories understand where
building user stories understand where is the greatest upside the problems that
is the greatest upside the problems that
is the greatest upside the problems that we experienced we set the target
we experienced we set the target
we experienced we set the target outcomes so target outcomes I’m going to
outcomes so target outcomes I’m going to
outcomes so target outcomes I’m going to call them the pocoyo key of product
call them the pocoyo key of product
call them the pocoyo key of product development anyone heard that term
development anyone heard that term
development anyone heard that term Pocoyo key before yeah see if you throw
Pocoyo key before yeah see if you throw
Pocoyo key before yeah see if you throw Japanese words and he was like how cool
Japanese words and he was like how cool
Japanese words and he was like how cool is that so Pocoyo key make it easy to do
is that so Pocoyo key make it easy to do
is that so Pocoyo key make it easy to do the right thing and hard to do the wrong
the right thing and hard to do the wrong
the right thing and hard to do the wrong thing
thing
thing haysoms exam because it makes me sound
haysoms exam because it makes me sound
haysoms exam because it makes me sound like I’m about 200 but floppy disks when
like I’m about 200 but floppy disks when
like I’m about 200 but floppy disks when you opened up the little fluffy disk
you opened up the little fluffy disk
you opened up the little fluffy disk container
container
container there was one way to put it in right and
there was one way to put it in right and
there was one way to put it in right and not the way to put it and wrong right so
not the way to put it and wrong right so
not the way to put it and wrong right so they designed in idiot proofing right
they designed in idiot proofing right
they designed in idiot proofing right they made it very easy to choose and
they made it very easy to choose and
they made it very easy to choose and this is what we need to do is
this is what we need to do is
this is what we need to do is idiot-proof our products so that we can
idiot-proof our products so that we can
idiot-proof our products so that we can tell which features should go in and
tell which features should go in and
tell which features should go in and better still which features not to build
better still which features not to build
better still which features not to build remember we’re dealing with the world
remember we’re dealing with the world
remember we’re dealing with the world that we’re getting so good at building
that we’re getting so good at building
that we’re getting so good at building stuff fast that’s making us lazy and
stuff fast that’s making us lazy and
stuff fast that’s making us lazy and it’s making easy to build too many
it’s making easy to build too many
it’s making easy to build too many things my best product donors I work
things my best product donors I work
things my best product donors I work with are very good at saying no well not
with are very good at saying no well not
with are very good at saying no well not now I’m not going to do it now
now I’m not going to do it now
now I’m not going to do it now target outcomes need to be clear
target outcomes need to be clear
target outcomes need to be clear measurable we’ve kind of used a
measurable we’ve kind of used a
measurable we’ve kind of used a simplified version of Gil’s work so we
simplified version of Gil’s work so we
simplified version of Gil’s work so we have a very clear baseline again you
have a very clear baseline again you
have a very clear baseline again you need data to set a good baseline we have
need data to set a good baseline we have
need data to set a good baseline we have a very good target this is the point we
a very good target this is the point we
a very good target this is the point we want to get to for example at Amazon it
want to get to for example at Amazon it
want to get to for example at Amazon it takes two weeks to buy a book we want to
takes two weeks to buy a book we want to
takes two weeks to buy a book we want to get that down to three days let’s keep
get that down to three days let’s keep
get that down to three days let’s keep building things and features and
building things and features and
building things and features and enhancements to get that cycle down okay
enhancements to get that cycle down okay
enhancements to get that cycle down okay that’s a simple version of setting
that’s a simple version of setting
that’s a simple version of setting target outcomes have fun and if you
target outcomes have fun and if you
target outcomes have fun and if you don’t measure how will you know when you
don’t measure how will you know when you
don’t measure how will you know when you get there it is a discipline I get
get there it is a discipline I get
get there it is a discipline I get people it’s hard to get the Cystic at
people it’s hard to get the Cystic at
people it’s hard to get the Cystic at first so I get people saying well we
first so I get people saying well we
first so I get people saying well we can’t quantify things you know we can’t
can’t quantify things you know we can’t
can’t quantify things you know we can’t quantify the user experience right who
quantify the user experience right who
quantify the user experience right who thinks that who thinks it’s hard to
thinks that who thinks it’s hard to
thinks that who thinks it’s hard to quantify everything any hands yeah ok so
quantify everything any hands yeah ok so
quantify everything any hands yeah ok so I had this recently so one of the other
I had this recently so one of the other
I had this recently so one of the other things I’m doing at the moment is
things I’m doing at the moment is
things I’m doing at the moment is writing a book on legal contracts that
writing a book on legal contracts that
writing a book on legal contracts that can be used for agile development it’s
can be used for agile development it’s
can be used for agile development it’s actually why I spent a lot of time
actually why I spent a lot of time
actually why I spent a lot of time building a target outcomes framework
building a target outcomes framework
building a target outcomes framework because fundamentally legal contracts
because fundamentally legal contracts
because fundamentally legal contracts are based on a goods and service
are based on a goods and service
are based on a goods and service contract it’s based on coming up with
contract it’s based on coming up with
contract it’s based on coming up with description of the goods you want like
description of the goods you want like
description of the goods you want like you go to a shop hand over I want
you go to a shop hand over I want
you go to a shop hand over I want buttermilk eggs they give you buttermilk
buttermilk eggs they give you buttermilk
buttermilk eggs they give you buttermilk eggs you get paid for it fundamentally
eggs you get paid for it fundamentally
eggs you get paid for it fundamentally flawed for the complex development we do
flawed for the complex development we do
flawed for the complex development we do in software so now we need a way to say
in software so now we need a way to say
in software so now we need a way to say let’s be results-driven let’s come up
let’s be results-driven let’s come up
let’s be results-driven let’s come up with outcomes that you measure and based
with outcomes that you measure and based
with outcomes that you measure and based on that we have to make everything
on that we have to make everything
on that we have to make everything measurable so I was working with a big
measurable so I was working with a big
measurable so I was working with a big charity I went
charity I went
charity I went do a quick contract review just to see
do a quick contract review just to see
do a quick contract review just to see what they were looking at they had some
what they were looking at they had some
what they were looking at they had some words in there we want our system they
words in there we want our system they
words in there we want our system they are building an internal CRM so they get
are building an internal CRM so they get
are building an internal CRM so they get a lot of people contributing and
a lot of people contributing and
a lot of people contributing and donating to their cause they want to
donating to their cause they want to
donating to their cause they want to track what they’re doing at the moment
track what they’re doing at the moment
track what they’re doing at the moment everybody is using things like Excel 10
everybody is using things like Excel 10
everybody is using things like Excel 10 different products etc they want to
different products etc they want to
different products etc they want to centralize it be able to do better data
centralize it be able to do better data
centralize it be able to do better data reporting etc and so they had written
reporting etc and so they had written
reporting etc and so they had written the contract for the suppliers which
the contract for the suppliers which
the contract for the suppliers which said we want our system to be easy to
said we want our system to be easy to
said we want our system to be easy to use and to be very consistent ok so what
use and to be very consistent ok so what
use and to be very consistent ok so what is consistent mean any ideas throw some
is consistent mean any ideas throw some
is consistent mean any ideas throw some ideas out consistent first thing that
ideas out consistent first thing that
ideas out consistent first thing that pops to mind same thing not go down
pops to mind same thing not go down
pops to mind same thing not go down anyone else
anyone else
anyone else predictable what a predictor will mean
see I like that version predictable that
see I like that version predictable that should be fragile team so you can guess
should be fragile team so you can guess
should be fragile team so you can guess what they might do ok right ok so I’m
what they might do ok right ok so I’m
what they might do ok right ok so I’m going through it and I’m mentally
going through it and I’m mentally
going through it and I’m mentally assuming it consistent while it means
assuming it consistent while it means
assuming it consistent while it means that it should work like if you click on
that it should work like if you click on
that it should work like if you click on a right button here it should be the
a right button here it should be the
a right button here it should be the same right button and you should go back
same right button and you should go back
same right button and you should go back and it should act that way and I said is
and it should act that way and I said is
and it should act that way and I said is that what you mean let’s really define
that what you mean let’s really define
that what you mean let’s really define it and she said no no I mean that once
it and she said no no I mean that once
it and she said no no I mean that once you run your report the data is correct
you run your report the data is correct
you run your report the data is correct so consistent mean there was actually
so consistent mean there was actually
so consistent mean there was actually data that was validated and it was
data that was validated and it was
data that was validated and it was actually correct and more precise data
actually correct and more precise data
actually correct and more precise data whoa who knew right so you can imagine
whoa who knew right so you can imagine
whoa who knew right so you can imagine the kind of systems we were about to
the kind of systems we were about to
the kind of systems we were about to build with these people so when people
build with these people so when people
build with these people so when people say you can’t measure everything yeah
say you can’t measure everything yeah
say you can’t measure everything yeah there are some difficulties last week I
there are some difficulties last week I
there are some difficulties last week I was down with a very big video gaming
was down with a very big video gaming
was down with a very big video gaming company helping them to build their
company helping them to build their
company helping them to build their video games
video games
video games very cool industry and I was kind of
very cool industry and I was kind of
very cool industry and I was kind of thinking how do I get this working what
thinking how do I get this working what
thinking how do I get this working what a target outcomes for gamers right so
a target outcomes for gamers right so
a target outcomes for gamers right so first I said them who’s your persona who
first I said them who’s your persona who
first I said them who’s your persona who you’re building it for a gamer like ok
you’re building it for a gamer like ok
you’re building it for a gamer like ok you’ve got at least three I can think of
you’ve got at least three I can think of
you’ve got at least three I can think of you’ve got a hardcore gamer a core gamer
you’ve got a hardcore gamer a core gamer
you’ve got a hardcore gamer a core gamer and a casual gamer very different news
and a casual gamer very different news
and a casual gamer very different news cases one has to get into the game as a
cases one has to get into the game as a
cases one has to get into the game as a casual gamer has to be easy to use
casual gamer has to be easy to use
casual gamer has to be easy to use hardcore gamers need skill difficulty I
hardcore gamers need skill difficulty I
hardcore gamers need skill difficulty I said ok and what are the goals for these
said ok and what are the goals for these
said ok and what are the goals for these gamers well we’re going to build a game
gamers well we’re going to build a game
gamers well we’re going to build a game where you win at the end measure Nets
where you win at the end measure Nets
where you win at the end measure Nets whatever a racing game you hit the
whatever a racing game you hit the
whatever a racing game you hit the I’m like wow that’s gonna be a great
I’m like wow that’s gonna be a great
I’m like wow that’s gonna be a great game I can make that now I can build a
game I can make that now I can build a
game I can make that now I can build a little racing track where they hit a
little racing track where they hit a
little racing track where they hit a finish line we’re good to go
finish line we’re good to go
finish line we’re good to go well no it’s about more than that what
well no it’s about more than that what
well no it’s about more than that what is it about no it’s about fun it’s about
is it about no it’s about fun it’s about
is it about no it’s about fun it’s about what is fun okay so we started breaking
what is fun okay so we started breaking
what is fun okay so we started breaking it down it’s really interesting we
it down it’s really interesting we
it down it’s really interesting we started building a little I was calling
started building a little I was calling
started building a little I was calling them an emotion map so we had things
them an emotion map so we had things
them an emotion map so we had things like attractors how do you attract
like attractors how do you attract
like attractors how do you attract people into the game we had frustrated
people into the game we had frustrated
people into the game we had frustrated if something’s not annoying enough then
if something’s not annoying enough then
if something’s not annoying enough then you get a bit bored with the game right
you get a bit bored with the game right
you get a bit bored with the game right we had to get to a point that someone
we had to get to a point that someone
we had to get to a point that someone gets to massive elation and winning and
gets to massive elation and winning and
gets to massive elation and winning and then there’s tail off what do they do
then there’s tail off what do they do
then there’s tail off what do they do next how do you make it so that they can
next how do you make it so that they can
next how do you make it so that they can re-experience that feeling get addicted
re-experience that feeling get addicted
re-experience that feeling get addicted back to your game so there are ways to
back to your game so there are ways to
back to your game so there are ways to kind of figure out what are the pieces
kind of figure out what are the pieces
kind of figure out what are the pieces that matter how do we measure them right
that matter how do we measure them right
that matter how do we measure them right a measure can be quantified but there’s
a measure can be quantified but there’s
a measure can be quantified but there’s other qualitative measures as well but
other qualitative measures as well but
other qualitative measures as well but what we’re trying to do and this is the
what we’re trying to do and this is the
what we’re trying to do and this is the thing that I think most people miss
thing that I think most people miss
thing that I think most people miss we’re not measuring to get the most
we’re not measuring to get the most
we’re not measuring to get the most precise number just like we don’t do
precise number just like we don’t do
precise number just like we don’t do things like poker planning to actually
things like poker planning to actually
things like poker planning to actually get the best estimates why do we do
get the best estimates why do we do
get the best estimates why do we do something like poker planning wow you’re
something like poker planning wow you’re
something like poker planning wow you’re really proud you’re there to share
really proud you’re there to share
really proud you’re there to share knowledge right yeah so what we’re
knowledge right yeah so what we’re
knowledge right yeah so what we’re trying to do when we quantify these
trying to do when we quantify these
trying to do when we quantify these things is get down to a level of
things is get down to a level of
things is get down to a level of understanding so that we’ve created
understanding so that we’ve created
understanding so that we’ve created poker your key easy to do the right
poker your key easy to do the right
poker your key easy to do the right thing hard to do the right roll thing
thing hard to do the right roll thing
thing hard to do the right roll thing okay so that you don’t want your product
okay so that you don’t want your product
okay so that you don’t want your product on or your customer who’s got customers
on or your customer who’s got customers
on or your customer who’s got customers that they can’t even get a hold of
that they can’t even get a hold of
that they can’t even get a hold of anyone work with those people right I
anyone work with those people right I
anyone work with those people right I get people where they’re dealing with
get people where they’re dealing with
get people where they’re dealing with generals in the army who are like okay
generals in the army who are like okay
generals in the army who are like okay product development team or war in
product development team or war in
product development team or war in Afghanistan yes he’s going to win right
Afghanistan yes he’s going to win right
Afghanistan yes he’s going to win right when you got those people you’ve got one
when you got those people you’ve got one
when you got those people you’ve got one opportunity to really try and understand
opportunity to really try and understand
opportunity to really try and understand their greater outcomes that make this
their greater outcomes that make this
their greater outcomes that make this work because you can’t sit there going
work because you can’t sit there going
work because you can’t sit there going should it be blue or green what do you
should it be blue or green what do you
should it be blue or green what do you think how many guys like you know those
think how many guys like you know those
think how many guys like you know those aren’t so things I can spend time on so
aren’t so things I can spend time on so
aren’t so things I can spend time on so what we’re trying to do is we set a
what we’re trying to do is we set a
what we’re trying to do is we set a direction once we have a very very clear
direction once we have a very very clear
direction once we have a very very clear direction if you do nothing else then
direction if you do nothing else then
direction if you do nothing else then this will give you a lot of power over
this will give you a lot of power over
this will give you a lot of power over choosing what to do next in your
choosing what to do next in your
choosing what to do next in your products then we want to go into
products then we want to go into
products then we want to go into something called options thinking so
something called options thinking so
something called options thinking so back to agile or whatever methodology
back to agile or whatever methodology
back to agile or whatever methodology a lot of them based on outputs what
a lot of them based on outputs what
a lot of them based on outputs what you’re doing as soon as you start
you’re doing as soon as you start
you’re doing as soon as you start writing up outputs but your product
writing up outputs but your product
writing up outputs but your product backlog your roadmap you’re committing
backlog your roadmap you’re committing
backlog your roadmap you’re committing to a certain future you’re saying this
to a certain future you’re saying this
to a certain future you’re saying this is what we think we’re going to build
is what we think we’re going to build
is what we think we’re going to build this is the danger of current legal
this is the danger of current legal
this is the danger of current legal contracts they force you into defining
contracts they force you into defining
contracts they force you into defining upfront and committing even if you can
upfront and committing even if you can
upfront and committing even if you can change it round it’s harder because once
change it round it’s harder because once
change it round it’s harder because once you give a feature a name or put it in a
you give a feature a name or put it in a
you give a feature a name or put it in a document has a life of its own and
document has a life of its own and
document has a life of its own and people feel like you’re killing it at
people feel like you’re killing it at
people feel like you’re killing it at that point of view take their life away
that point of view take their life away
that point of view take their life away so we’re going to stop pretending that
so we’re going to stop pretending that
so we’re going to stop pretending that you can see the future and again you’ve
you can see the future and again you’ve
you can see the future and again you’ve had this theme pushed at you today about
had this theme pushed at you today about
had this theme pushed at you today about Lean Startup create options so instead
Lean Startup create options so instead
Lean Startup create options so instead of having one thing and to have multiple
of having one thing and to have multiple
of having one thing and to have multiple ideas right we’ve got an outcome to get
ideas right we’ve got an outcome to get
ideas right we’ve got an outcome to get to but there’s many ways to reach that
to but there’s many ways to reach that
to but there’s many ways to reach that outcome so keep your options open what
outcome so keep your options open what
outcome so keep your options open what we want to do is again this is back to
we want to do is again this is back to
we want to do is again this is back to responsible moment thinking you want to
responsible moment thinking you want to
responsible moment thinking you want to make decisions as reversible as possible
make decisions as reversible as possible
make decisions as reversible as possible and irreversible decisions as late as
and irreversible decisions as late as
and irreversible decisions as late as possible options thinking is all about
possible options thinking is all about
possible options thinking is all about setting up potential ideas and then
setting up potential ideas and then
setting up potential ideas and then being able to execute on them rapidly I
being able to execute on them rapidly I
being able to execute on them rapidly I was talking the other day that’s
was talking the other day that’s
was talking the other day that’s interesting but had a different slide up
interesting but had a different slide up
interesting but had a different slide up there we were talking about we were
there we were talking about we were
there we were talking about we were talking about yeah we were talking about
talking about yeah we were talking about
talking about yeah we were talking about Darwin’s theory survival of the fittest
Darwin’s theory survival of the fittest
Darwin’s theory survival of the fittest it’s not about survival survival there’s
it’s not about survival survival there’s
it’s not about survival survival there’s a movie called dudes like a teen movie I
a movie called dudes like a teen movie I
a movie called dudes like a teen movie I think they talk about survivals the
think they talk about survivals the
think they talk about survivals the slovers film was suicide that’s a way of
slovers film was suicide that’s a way of
slovers film was suicide that’s a way of trading order arrival as the of the
trading order arrival as the of the
trading order arrival as the of the fittest is where it’s actually at so
fittest is where it’s actually at so
fittest is where it’s actually at so it’s the people who can get to market
it’s the people who can get to market
it’s the people who can get to market with the right product the quickest not
with the right product the quickest not
with the right product the quickest not the ones who can just survive in that
the ones who can just survive in that
the ones who can just survive in that market we want to be better than
market we want to be better than
market we want to be better than surviving the outcomes options idea is
surviving the outcomes options idea is
surviving the outcomes options idea is fractal so what you can do is have high
fractal so what you can do is have high
fractal so what you can do is have high level outcomes for the company things
level outcomes for the company things
level outcomes for the company things like reach engagement with our customers
like reach engagement with our customers
like reach engagement with our customers monetization they should link through
monetization they should link through
monetization they should link through and align to every area of development
and align to every area of development
and align to every area of development right down to the qualities of your
right down to the qualities of your
right down to the qualities of your system right so every big outcome for a
system right so every big outcome for a
system right so every big outcome for a company breaks down into multiple
company breaks down into multiple
company breaks down into multiple options think of that as your portfolio
options think of that as your portfolio
options think of that as your portfolio your roadmap all of those break down and
your roadmap all of those break down and
your roadmap all of those break down and two different options and streams and so
two different options and streams and so
two different options and streams and so forth okay so it’s a very flexible more
forth okay so it’s a very flexible more
forth okay so it’s a very flexible more of a model that looks like a very
of a model that looks like a very
of a model that looks like a very complex system as opposed to the linear
complex system as opposed to the linear
complex system as opposed to the linear stuff that we’ve got ourselves into an
stuff that we’ve got ourselves into an
stuff that we’ve got ourselves into an agile so if you think about your product
agile so if you think about your product
agile so if you think about your product backlogs they’re very linear they lead
backlogs they’re very linear they lead
backlogs they’re very linear they lead to very tactical thinking that’s why
to very tactical thinking that’s why
to very tactical thinking that’s why originally the talk was called Frank and
originally the talk was called Frank and
originally the talk was called Frank and builds we keep hacking things together
builds we keep hacking things together
builds we keep hacking things together and hoping for a good result so all this
and hoping for a good result so all this
and hoping for a good result so all this does that sounds simple you can go out
does that sounds simple you can go out
does that sounds simple you can go out and do that right how do you know what
and do that right how do you know what
and do that right how do you know what the right thing is we don’t we have no
the right thing is we don’t we have no
the right thing is we don’t we have no inclu right we never know I like the
inclu right we never know I like the
inclu right we never know I like the idea we keep saying there’s no such
idea we keep saying there’s no such
idea we keep saying there’s no such thing as product requirements their
thing as product requirements their
thing as product requirements their product guesses they are options and
product guesses they are options and
product guesses they are options and experiments we should run never think
experiments we should run never think
experiments we should run never think you know in advance because we don’t so
you know in advance because we don’t so
you know in advance because we don’t so stop guessing start testing we’ve talked
stop guessing start testing we’ve talked
stop guessing start testing we’ve talked about the idea running multiple options
about the idea running multiple options
about the idea running multiple options fast the thing is we’re all about Nigel
fast the thing is we’re all about Nigel
fast the thing is we’re all about Nigel making the unknowns known that’s why we
making the unknowns known that’s why we
making the unknowns known that’s why we do vertical slices of our product etc
do vertical slices of our product etc
do vertical slices of our product etc we’re trying to highlight risks we’re
we’re trying to highlight risks we’re
we’re trying to highlight risks we’re trying to get ideas and information back
trying to get ideas and information back
trying to get ideas and information back quickly we talked about set based design
quickly we talked about set based design
quickly we talked about set based design with Josh already so I talked about that
with Josh already so I talked about that
with Josh already so I talked about that that was a slide I thought I’d have so
that was a slide I thought I’d have so
that was a slide I thought I’d have so get an affray of your users early and
get an affray of your users early and
get an affray of your users early and often another big tip I give you I
often another big tip I give you I
often another big tip I give you I noticed a dramatic change we had very
noticed a dramatic change we had very
noticed a dramatic change we had very advanced researchers at Yahoo user
advanced researchers at Yahoo user
advanced researchers at Yahoo user experienced people we had people from
experienced people we had people from
experienced people we had people from Apple’s innovation lands we had the
Apple’s innovation lands we had the
Apple’s innovation lands we had the whole gamut what I found is they would
whole gamut what I found is they would
whole gamut what I found is they would say hey this doesn’t work the people
say hey this doesn’t work the people
say hey this doesn’t work the people don’t use it
don’t use it
don’t use it the developers never quite got it so we
the developers never quite got it so we
the developers never quite got it so we started taking our development teams out
started taking our development teams out
started taking our development teams out on field trips with us
on field trips with us
on field trips with us seeing as believing you need to
seeing as believing you need to
seeing as believing you need to experience it even in our usability labs
experience it even in our usability labs
experience it even in our usability labs I’ll give you an example Ajax had just
I’ll give you an example Ajax had just
I’ll give you an example Ajax had just come out so you could drag and drop and
come out so you could drag and drop and
come out so you could drag and drop and so the team had designed this great
so the team had designed this great
so the team had designed this great photo album feature we could drag and
photo album feature we could drag and
photo album feature we could drag and drop your photos all the execs were like
drop your photos all the execs were like
drop your photos all the execs were like oh that’s the best thing ever you know
oh that’s the best thing ever you know
oh that’s the best thing ever you know it’s going to be the breaking ground and
it’s going to be the breaking ground and
it’s going to be the breaking ground and we got the lab going so we had someone
we got the lab going so we had someone
we got the lab going so we had someone coming in behind that two-way glass and
coming in behind that two-way glass and
coming in behind that two-way glass and they get in and they’re looking at it
they get in and they’re looking at it
they get in and they’re looking at it and someone says okay so how would you
and someone says okay so how would you
and someone says okay so how would you create an album and move your photos
create an album and move your photos
create an album and move your photos into it and that clicking around the
into it and that clicking around the
into it and that clicking around the light
light
light no I give up behind the screen as the
no I give up behind the screen as the
no I give up behind the screen as the developer he said anything I know what
developer he said anything I know what
developer he said anything I know what you do you move you stupid person you
you do you move you stupid person you
you do you move you stupid person you know because of course that user was an
know because of course that user was an
know because of course that user was an idiot because the developers you know
idiot because the developers you know
idiot because the developers you know obviously smart design this very cool
obviously smart design this very cool
obviously smart design this very cool system so uh anyway they walk out and
system so uh anyway they walk out and
system so uh anyway they walk out and Scott just said give me a laptop runs
Scott just said give me a laptop runs
Scott just said give me a laptop runs over do we push it through to the next
over do we push it through to the next
over do we push it through to the next room before the next person got into for
room before the next person got into for
room before the next person got into for the next test problem fixed we’ve been
the next test problem fixed we’ve been
the next test problem fixed we’ve been telling him this for two months Scott
telling him this for two months Scott
telling him this for two months Scott they don’t get it people don’t get this
they don’t get it people don’t get this
they don’t get it people don’t get this drag-and-drop thing it’s not it’s not
drag-and-drop thing it’s not it’s not
drag-and-drop thing it’s not it’s not intuitive so by taking them out we
intuitive so by taking them out we
intuitive so by taking them out we change the whole model I’m ignore your
change the whole model I’m ignore your
change the whole model I’m ignore your mother be fast dirty and cheap hopefully
mother be fast dirty and cheap hopefully
mother be fast dirty and cheap hopefully my mother’s not watching this everyone
my mother’s not watching this everyone
my mother’s not watching this everyone comes to me I got someone the other day
comes to me I got someone the other day
comes to me I got someone the other day we were I’m like they were building
we were I’m like they were building
we were I’m like they were building something and I’m like why are you
something and I’m like why are you
something and I’m like why are you building that way you don’t even know if
building that way you don’t even know if
building that way you don’t even know if it works and they said well in q4 we’re
it works and they said well in q4 we’re
it works and they said well in q4 we’re going to get a user testing group to
going to get a user testing group to
going to get a user testing group to come in and do all this research on it
come in and do all this research on it
come in and do all this research on it what do you mean don’t you have like
what do you mean don’t you have like
what do you mean don’t you have like friends or family now like go and get
friends or family now like go and get
friends or family now like go and get cutlet computers will set up this test
cutlet computers will set up this test
cutlet computers will set up this test so immediately we get feedback to give
so immediately we get feedback to give
so immediately we get feedback to give an example I was in New Zealand giving a
an example I was in New Zealand giving a
an example I was in New Zealand giving a class there and I had someone from the
class there and I had someone from the
class there and I had someone from the police department in the government they
police department in the government they
police department in the government they were building all the software all the
were building all the software all the
were building all the software all the tracking of the police records and she
tracking of the police records and she
tracking of the police records and she used to be a policewoman and she said
used to be a policewoman and she said
used to be a policewoman and she said you know what
you know what
you know what our software is so unusable it makes me
our software is so unusable it makes me
our software is so unusable it makes me sick but I tell the guys and they don’t
sick but I tell the guys and they don’t
sick but I tell the guys and they don’t get it I said you’re a policeman right
get it I said you’re a policeman right
get it I said you’re a policeman right yeah we’re local yep still got friends
yeah we’re local yep still got friends
yeah we’re local yep still got friends there yep okay I said how much will it
there yep okay I said how much will it
there yep okay I said how much will it cost for a little taxi to take everyone
cost for a little taxi to take everyone
cost for a little taxi to take everyone in and no 20 bucks I said what about the
in and no 20 bucks I said what about the
in and no 20 bucks I said what about the police people like doughnuts is that a
police people like doughnuts is that a
police people like doughnuts is that a methyl true and she went actually they
methyl true and she went actually they
methyl true and she went actually they really do and I said okay get a box of
really do and I said okay get a box of
really do and I said okay get a box of doughnuts get your team go and spend the
doughnuts get your team go and spend the
doughnuts get your team go and spend the afternoon with the police people so they
afternoon with the police people so they
afternoon with the police people so they did so for 20 bucks in the cost of a box
did so for 20 bucks in the cost of a box
did so for 20 bucks in the cost of a box of doughnuts they went and sat beside
of doughnuts they went and sat beside
of doughnuts they went and sat beside the police people immediately they saw
the police people immediately they saw
the police people immediately they saw Joe who was 56 with two kids exhausted
Joe who was 56 with two kids exhausted
Joe who was 56 with two kids exhausted after being all down the beat coming and
after being all down the beat coming and
after being all down the beat coming and trying to add his time and it was just
trying to add his time and it was just
trying to add his time and it was just miserable they felt so bad
miserable they felt so bad
miserable they felt so bad they created a little picture of Joe put
they created a little picture of Joe put
they created a little picture of Joe put it on the wall and every time they went
it on the wall and every time they went
it on the wall and every time they went to build something they went yeah
to build something they went yeah
to build something they went yeah but would you really get that probably
but would you really get that probably
but would you really get that probably not right so this is like fast dirty
not right so this is like fast dirty
not right so this is like fast dirty chief a team in Bangalore India every
chief a team in Bangalore India every
chief a team in Bangalore India every lunchtime they go out on a Friday so hey
lunchtime they go out on a Friday so hey
lunchtime they go out on a Friday so hey we’ll give you a free lunch come and try
we’ll give you a free lunch come and try
we’ll give you a free lunch come and try it a yahoo product that’s all it costs
it a yahoo product that’s all it costs
it a yahoo product that’s all it costs to get rapid feedback so don’t think it
to get rapid feedback so don’t think it
to get rapid feedback so don’t think it has to be expensive to do this remember
has to be expensive to do this remember
has to be expensive to do this remember that features are not the experience
that features are not the experience
that features are not the experience again poka-yoke k creates this is the
again poka-yoke k creates this is the
again poka-yoke k creates this is the goal this is the job our customers need
goal this is the job our customers need
goal this is the job our customers need to get done more features are actually a
to get done more features are actually a
to get done more features are actually a problem what we need to do is remove the
problem what we need to do is remove the
problem what we need to do is remove the in essential so the essential can speak
in essential so the essential can speak
in essential so the essential can speak right the more features on the page of
right the more features on the page of
right the more features on the page of the higher it is to use when the iPod
the higher it is to use when the iPod
the higher it is to use when the iPod came out they had about 40% of the
came out they had about 40% of the
came out they had about 40% of the feature set of mostly other mp3 players
feature set of mostly other mp3 players
feature set of mostly other mp3 players because they got what the job people
because they got what the job people
because they got what the job people wanted to get done was to actually get
wanted to get done was to actually get
wanted to get done was to actually get music quickly and listen to it right
music quickly and listen to it right
music quickly and listen to it right which meant you had to be able to
which meant you had to be able to
which meant you had to be able to connect someone to get somewhere to get
connect someone to get somewhere to get
connect someone to get somewhere to get the music and get it and listen to it so
the music and get it and listen to it so
the music and get it and listen to it so again the game is to hit the target with
again the game is to hit the target with
again the game is to hit the target with as few bullets as possible that’s why
as few bullets as possible that’s why
as few bullets as possible that’s why I’m very concerned by current legal
I’m very concerned by current legal
I’m very concerned by current legal contracts they incentivize building more
contracts they incentivize building more
contracts they incentivize building more I’m concerned about things like velocity
I’m concerned about things like velocity
I’m concerned about things like velocity cycle time measures which measure how
cycle time measures which measure how
cycle time measures which measure how much throughput we can get to me that’s
much throughput we can get to me that’s
much throughput we can get to me that’s telling your developers to build as much
telling your developers to build as much
telling your developers to build as much as possible right I want you to do the
as possible right I want you to do the
as possible right I want you to do the opposite so the game we have very smart
opposite so the game we have very smart
opposite so the game we have very smart people we work with we set a game target
people we work with we set a game target
people we work with we set a game target which is we need to get here as quickly
which is we need to get here as quickly
which is we need to get here as quickly and with as few features as possible if
and with as few features as possible if
and with as few features as possible if you do that maybe you can go home early
you do that maybe you can go home early
you do that maybe you can go home early right I don’t want more code so this is
right I don’t want more code so this is
right I don’t want more code so this is about that idea of Lean Startup we’re
about that idea of Lean Startup we’re
about that idea of Lean Startup we’re building minimal viable not maximum
building minimal viable not maximum
building minimal viable not maximum possible your job who are the product
possible your job who are the product
possible your job who are the product owners in the room any yeah your job is
owners in the room any yeah your job is
owners in the room any yeah your job is to try and filter that to get as little
to try and filter that to get as little
to try and filter that to get as little through that pipe as possible measure as
through that pipe as possible measure as
through that pipe as possible measure as you go now everyone’s listened to
you go now everyone’s listened to
you go now everyone’s listened to metrics going yeah I get that we say
metrics going yeah I get that we say
metrics going yeah I get that we say that we need 20 new signups or make X
that we need 20 new signups or make X
that we need 20 new signups or make X amount of revenue what they do is they
amount of revenue what they do is they
amount of revenue what they do is they put in a pitch has anyone seen those
put in a pitch has anyone seen those
put in a pitch has anyone seen those bids for work we want these two hundred
bids for work we want these two hundred
bids for work we want these two hundred resources and a year and we’re going to
resources and a year and we’re going to
resources and a year and we’re going to make fifty million dollars the greatest
make fifty million dollars the greatest
make fifty million dollars the greatest product ever
product ever
product ever right we will see move don’t believe it
right we will see move don’t believe it
right we will see move don’t believe it what we do is we constantly measure that
what we do is we constantly measure that
what we do is we constantly measure that way we get instant feedback are we in
way we get instant feedback are we in
way we get instant feedback are we in the right direction we set up automated
the right direction we set up automated
the right direction we set up automated metrics frameworks so that we can get
metrics frameworks so that we can get
metrics frameworks so that we can get this real-time feedback back at that
this real-time feedback back at that
this real-time feedback back at that beginning step where we talked about go
beginning step where we talked about go
beginning step where we talked about go and gather the data this is when you’ll
and gather the data this is when you’ll
and gather the data this is when you’ll start to see the holes in your
start to see the holes in your
start to see the holes in your organization’s do you have the usage
organization’s do you have the usage
organization’s do you have the usage statistics do you know how people are
statistics do you know how people are
statistics do you know how people are using your products if you don’t start
using your products if you don’t start
using your products if you don’t start building those in as quickly as possible
building those in as quickly as possible
building those in as quickly as possible this is what will drive your company
this is what will drive your company
this is what will drive your company this is why Amazon so good they
this is why Amazon so good they
this is why Amazon so good they constantly look at the numbers they set
constantly look at the numbers they set
constantly look at the numbers they set up very sophisticated frameworks so if
up very sophisticated frameworks so if
up very sophisticated frameworks so if your metrics are at lag your product
your metrics are at lag your product
your metrics are at lag your product will lag you need to get those quickly I
talk too much about continuous
talk too much about continuous deployment this is where the agile group
deployment this is where the agile group
deployment this is where the agile group can come in again that we say our job is
can come in again that we say our job is
can come in again that we say our job is to make it as fast as possible to get
to make it as fast as possible to get
to make it as fast as possible to get that product to market user testing we
that product to market user testing we
that product to market user testing we talked about that I although I said I
talked about that I although I said I
talked about that I although I said I used some lab testing I really like
used some lab testing I really like
used some lab testing I really like sitting up multivariate testing the idea
sitting up multivariate testing the idea
sitting up multivariate testing the idea of split testing with multiple outcomes
of split testing with multiple outcomes
of split testing with multiple outcomes if you do that why test with five people
if you do that why test with five people
if you do that why test with five people in a lab can actually give you
in a lab can actually give you
in a lab can actually give you substandard results we used to be able
substandard results we used to be able
substandard results we used to be able to very quickly at a start-up in San
to very quickly at a start-up in San
to very quickly at a start-up in San Francisco flip the switch and
Francisco flip the switch and
Francisco flip the switch and immediately we could siphon off a couple
immediately we could siphon off a couple
immediately we could siphon off a couple of percent and be testing real-time with
of percent and be testing real-time with
of percent and be testing real-time with thousands of users that gave us much
thousands of users that gave us much
thousands of users that gave us much more contextual data one thing I haven’t
more contextual data one thing I haven’t
more contextual data one thing I haven’t heard a lot about is we talked about
heard a lot about is we talked about
heard a lot about is we talked about metrics and gathering them and a lot of
metrics and gathering them and a lot of
metrics and gathering them and a lot of people think that’s for the managers or
people think that’s for the managers or
people think that’s for the managers or the product owners I always treat my
the product owners I always treat my
the product owners I always treat my developers I think essentially they have
developers I think essentially they have
developers I think essentially they have a gamer mind so they like playing games
a gamer mind so they like playing games
a gamer mind so they like playing games maybe but also they like things like the
maybe but also they like things like the
maybe but also they like things like the thrill of the kill finding defects you
thrill of the kill finding defects you
thrill of the kill finding defects you know they have a game and mindset so
know they have a game and mindset so
know they have a game and mindset so when I set targets transparency is key
when I set targets transparency is key
when I set targets transparency is key give all those numbers back to your
give all those numbers back to your
give all those numbers back to your developers at the startup every month
developers at the startup every month
developers at the startup every month we’d have a pizza meeting we will give
we’d have a pizza meeting we will give
we’d have a pizza meeting we will give them all the spreadsheets we would show
them all the spreadsheets we would show
them all the spreadsheets we would show which stuff we’d launched what feedback
which stuff we’d launched what feedback
which stuff we’d launched what feedback we got what changed in the company so
we got what changed in the company so
we got what changed in the company so again this is not for management this is
again this is not for management this is
again this is not for management this is for everybody to drive by
try to have very visual progress
try to have very visual progress indicators so ways that you can say you
indicators so ways that you can say you
indicators so ways that you can say you know if we want a certain outcome are we
know if we want a certain outcome are we
know if we want a certain outcome are we making incremental progress keep those
making incremental progress keep those
making incremental progress keep those very front of mind you can drop your
very front of mind you can drop your
very front of mind you can drop your burn down charts I don’t need any of
burn down charts I don’t need any of
burn down charts I don’t need any of that stuff I want to see are we actually
that stuff I want to see are we actually
that stuff I want to see are we actually getting valid business results back as
getting valid business results back as
getting valid business results back as we go so how do we know we’ve succeeded
we go so how do we know we’ve succeeded
we go so how do we know we’ve succeeded we get to where we’re going and so we
we get to where we’re going and so we
we get to where we’re going and so we hit the target outcomes and if we don’t
hit the target outcomes and if we don’t
hit the target outcomes and if we don’t we very quickly adapt so we pivot at
we very quickly adapt so we pivot at
we very quickly adapt so we pivot at this point to use Lean Startup language
this point to use Lean Startup language
this point to use Lean Startup language so I get people saying well that’s all
so I get people saying well that’s all
so I get people saying well that’s all well and good but does it really work
well and good but does it really work
well and good but does it really work so which sounds better our velocity is
so which sounds better our velocity is
so which sounds better our velocity is thirty six point forty six points and
thirty six point forty six points and
thirty six point forty six points and we’ve delivered a hundred new features
we’ve delivered a hundred new features
we’ve delivered a hundred new features or we just implemented one feature in
or we just implemented one feature in
or we just implemented one feature in three days which will save us 12 million
three days which will save us 12 million
three days which will save us 12 million pounds over the next year what would you
pounds over the next year what would you
pounds over the next year what would you choose – here we are we did a pilot with
choose – here we are we did a pilot with
choose – here we are we did a pilot with a company recently and trained up one of
a company recently and trained up one of
a company recently and trained up one of our favorite product donors and they had
our favorite product donors and they had
our favorite product donors and they had an All Hands meeting they had big
an All Hands meeting they had big
an All Hands meeting they had big off-site with all the product managers
off-site with all the product managers
off-site with all the product managers all of them put up their powerpoints and
all of them put up their powerpoints and
all of them put up their powerpoints and we’d a little bit of these features on
we’d a little bit of these features on
we’d a little bit of these features on time and on budget of course and so went
time and on budget of course and so went
time and on budget of course and so went and Jamie just got up and went yeah well
and Jamie just got up and went yeah well
and Jamie just got up and went yeah well we made X million and this is what we’re
we made X million and this is what we’re
we made X million and this is what we’re doing and this is how we did it and all
doing and this is how we did it and all
doing and this is how we did it and all the managers just went oh that’s the guy
the managers just went oh that’s the guy
the managers just went oh that’s the guy and he’s you know going off in the
and he’s you know going off in the
and he’s you know going off in the company pretty rapidly what industry can
company pretty rapidly what industry can
company pretty rapidly what industry can he do this with we’re currently building
he do this with we’re currently building
he do this with we’re currently building we’ve been building these huge automated
we’ve been building these huge automated
we’ve been building these huge automated metrics frameworks we not only measure
metrics frameworks we not only measure
metrics frameworks we not only measure products but also transformation areas
products but also transformation areas
products but also transformation areas so I’ve gone to companies where they
so I’ve gone to companies where they
so I’ve gone to companies where they have like an Aikido test are you doing
have like an Aikido test are you doing
have like an Aikido test are you doing daily stand-ups like a maturity model
daily stand-ups like a maturity model
daily stand-ups like a maturity model like oh my god why do you do daily
like oh my god why do you do daily
like oh my god why do you do daily stand-ups to communicate so let’s
stand-ups to communicate so let’s
stand-ups to communicate so let’s measure communication saturation if
measure communication saturation if
measure communication saturation if we’re looking at Cost Reduction let’s
we’re looking at Cost Reduction let’s
we’re looking at Cost Reduction let’s measure total cost of ownership for
measure total cost of ownership for
measure total cost of ownership for systems so we’re putting all these
systems so we’re putting all these
systems so we’re putting all these measures in and not only at a big energy
measures in and not only at a big energy
measures in and not only at a big energy trading company we’re doing metrics
trading company we’re doing metrics
trading company we’re doing metrics video gaming and we’re doing it for a
video gaming and we’re doing it for a
video gaming and we’re doing it for a big voice over IP telco so this stuff
big voice over IP telco so this stuff
big voice over IP telco so this stuff I’m again huge agile person have been
I’m again huge agile person have been
I’m again huge agile person have been doing it for a long time to me this is
doing it for a long time to me this is
doing it for a long time to me this is way way evolved
way way evolved
way way evolved right this is so much better than what I
right this is so much better than what I
right this is so much better than what I was doing before I mean yeah we’re kind
was doing before I mean yeah we’re kind
was doing before I mean yeah we’re kind of doing a startup and then I got
of doing a startup and then I got
of doing a startup and then I got distracted trying to make agile better
distracted trying to make agile better
distracted trying to make agile better but actually this is the game we should
but actually this is the game we should
but actually this is the game we should be playing so I want to talk too much
be playing so I want to talk too much
be playing so I want to talk too much about how this changes everything I
about how this changes everything I
about how this changes everything I mentioned legal contracts but let’s go
mentioned legal contracts but let’s go
mentioned legal contracts but let’s go back so I know we always go to a time
back so I know we always go to a time
back so I know we always go to a time where we started why do you think I said
where we started why do you think I said
where we started why do you think I said add your Lister’s are killing the planet
add your Lister’s are killing the planet
add your Lister’s are killing the planet that’s a pretty strong statement
that’s a pretty strong statement
that’s a pretty strong statement especially for a CST by the way come to
especially for a CST by the way come to
especially for a CST by the way come to my class on Thursday Friday night I when
my class on Thursday Friday night I when
my class on Thursday Friday night I when you use the machine gun approach to
you use the machine gun approach to
you use the machine gun approach to develop your products you’re basically
develop your products you’re basically
develop your products you’re basically building more features faster the more
building more features faster the more
building more features faster the more features you build the more resources
features you build the more resources
features you build the more resources you use so if you think about it every
you use so if you think about it every
you use so if you think about it every feature you build takes a certain amount
feature you build takes a certain amount
feature you build takes a certain amount of machinery power etc to use each time
of machinery power etc to use each time
of machinery power etc to use each time you have to maintain it the more servers
you have to maintain it the more servers
you have to maintain it the more servers that build all your crap and store it up
that build all your crap and store it up
that build all your crap and store it up the more beautiful feature farms that
the more beautiful feature farms that
the more beautiful feature farms that you develop that’s using a lot of power
you develop that’s using a lot of power
you develop that’s using a lot of power right so you’re actually helping to kill
right so you’re actually helping to kill
right so you’re actually helping to kill the planet faster good on you so so the
the planet faster good on you so so the
the planet faster good on you so so the features save the planet um any
features save the planet um any
features save the planet um any questions yes ah great question
questions yes ah great question
questions yes ah great question question was you know I talked about
question was you know I talked about
question was you know I talked about this 12 million we can make how do you
this 12 million we can make how do you
this 12 million we can make how do you measure the money uh well again back to
measure the money uh well again back to
measure the money uh well again back to sort of the thinking of testings we can
sort of the thinking of testings we can
sort of the thinking of testings we can get down to quite sophisticated
get down to quite sophisticated
get down to quite sophisticated financial models because at the end of
financial models because at the end of
financial models because at the end of the day I look at it that we’re asking
the day I look at it that we’re asking
the day I look at it that we’re asking for investments and we’re showing that
for investments and we’re showing that
for investments and we’re showing that we can create value back on that
we can create value back on that
we can create value back on that investment right each bit of money give
investment right each bit of money give
investment right each bit of money give to development’s investment so let’s say
to development’s investment so let’s say
to development’s investment so let’s say usually I do it to give us some plastic
usually I do it to give us some plastic
usually I do it to give us some plastic example I can’t use that one because I
example I can’t use that one because I
example I can’t use that one because I get killed apparently in my India but
get killed apparently in my India but
get killed apparently in my India but Amazon let’s say that I go through
Amazon let’s say that I go through
Amazon let’s say that I go through Amazon it’s taking two weeks to get
Amazon it’s taking two weeks to get
Amazon it’s taking two weeks to get books or maybe actually a very easy one
books or maybe actually a very easy one
books or maybe actually a very easy one is if you look at the customer
is if you look at the customer
is if you look at the customer conversion funnel so from when they land
conversion funnel so from when they land
conversion funnel so from when they land on something like Amazon to when they
on something like Amazon to when they
on something like Amazon to when they actually get through and get the book
actually get through and get the book
actually get through and get the book where do we lose people so often you
where do we lose people so often you
where do we lose people so often you know in the early days people found that
know in the early days people found that
know in the early days people found that they’d get
they’d get
they’d get percent drop off at certain pages you
percent drop off at certain pages you
percent drop off at certain pages you can look at them and say oh weird they
can look at them and say oh weird they
can look at them and say oh weird they get to the shipping page and then they
get to the shipping page and then they
get to the shipping page and then they disappear
disappear
disappear turns out all they wanted was the
turns out all they wanted was the
turns out all they wanted was the shipping information maybe they get
shipping information maybe they get
shipping information maybe they get through into the payment scheme and find
through into the payment scheme and find
through into the payment scheme and find that they can’t actually use a certain
that they can’t actually use a certain
that they can’t actually use a certain payment type this is we would start
payment type this is we would start
payment type this is we would start doing split testing to try the things so
doing split testing to try the things so
doing split testing to try the things so now we know that actually there’s a
now we know that actually there’s a
now we know that actually there’s a lifetime revenue amount that many
lifetime revenue amount that many
lifetime revenue amount that many companies can work out that for each
companies can work out that for each
companies can work out that for each person who converts there’s an immediate
person who converts there’s an immediate
person who converts there’s an immediate money but also you turn them into a
money but also you turn them into a
money but also you turn them into a longer lifetime customer so you might
longer lifetime customer so you might
longer lifetime customer so you might have a revenue model where 50 bucks a
have a revenue model where 50 bucks a
have a revenue model where 50 bucks a year for each person that we have is a
year for each person that we have is a
year for each person that we have is a certain conversion statistic so now I
certain conversion statistic so now I
certain conversion statistic so now I know based on that for each one that
know based on that for each one that
know based on that for each one that completes that conversion that if they
completes that conversion that if they
completes that conversion that if they were worth $50 to me then say if I have
were worth $50 to me then say if I have
were worth $50 to me then say if I have a million people we’re off at 10% okay
a million people we’re off at 10% okay
a million people we’re off at 10% okay then you can go through and say for each
then you can go through and say for each
then you can go through and say for each percentage that’s worth X amount of
percentage that’s worth X amount of
percentage that’s worth X amount of money to the company right so now if I
money to the company right so now if I
money to the company right so now if I know that hey when I’ve got to choose so
know that hey when I’ve got to choose so
know that hey when I’ve got to choose so I’ll give you an example let’s say we
I’ll give you an example let’s say we
I’ll give you an example let’s say we found that there was a big drop-off in
found that there was a big drop-off in
found that there was a big drop-off in conversions when people bounced if the
conversions when people bounced if the
conversions when people bounced if the page loaded slowly right so that causes
page loaded slowly right so that causes
page loaded slowly right so that causes a huge drop-off I could look at three
a huge drop-off I could look at three
a huge drop-off I could look at three options to fix that one could be by
options to fix that one could be by
options to fix that one could be by looking at I say well our hosting is
looking at I say well our hosting is
looking at I say well our hosting is based in England and a lot of the
based in England and a lot of the
based in England and a lot of the drop-off happens for a lot of our users
drop-off happens for a lot of our users
drop-off happens for a lot of our users in Australia because the data center is
in Australia because the data center is
in Australia because the data center is so far away that causes a slowdown it
so far away that causes a slowdown it
so far away that causes a slowdown it could be Network latency maybe we could
could be Network latency maybe we could
could be Network latency maybe we could just catch some page elements there’s a
just catch some page elements there’s a
just catch some page elements there’s a few options right thus we’re getting
few options right thus we’re getting
few options right thus we’re getting into options now I can start costing it
into options now I can start costing it
into options now I can start costing it out I know that if we improve that we
out I know that if we improve that we
out I know that if we improve that we could potentially make X amount of money
could potentially make X amount of money
could potentially make X amount of money which one of these options do I would
which one of these options do I would
which one of these options do I would play out so I probably start with
play out so I probably start with
play out so I probably start with caching the page elements I look at
caching the page elements I look at
caching the page elements I look at maybe some network improvements what is
maybe some network improvements what is
maybe some network improvements what is it but there’s a point where I know that
it but there’s a point where I know that
it but there’s a point where I know that maybe the data center is too expensive
maybe the data center is too expensive
maybe the data center is too expensive for the amount of value I potentially
for the amount of value I potentially
for the amount of value I potentially get back this is where you get into fine
get back this is where you get into fine
get back this is where you get into fine testing they have time to go through all
testing they have time to go through all
testing they have time to go through all that is that okay yeah Wow you look
that is that okay yeah Wow you look
that is that okay yeah Wow you look puzzled still good come to one of my
puzzled still good come to one of my
puzzled still good come to one of my other classes about stuff you
other classes about stuff you
other classes about stuff you and actually it’s a good point so the
and actually it’s a good point so the
and actually it’s a good point so the question was how much is it do you
question was how much is it do you
question was how much is it do you measure too much and again think back to
measure too much and again think back to
measure too much and again think back to investment what’s the value we actually
investment what’s the value we actually
investment what’s the value we actually are very clear about trying to measure
are very clear about trying to measure
are very clear about trying to measure very few things to start with so we
very few things to start with so we
very few things to start with so we might only take three simple measures if
might only take three simple measures if
might only take three simple measures if you measure too much it’s confusing also
you measure too much it’s confusing also
you measure too much it’s confusing also this is like you know if I launch 100
this is like you know if I launch 100
this is like you know if I launch 100 features all at once this is the problem
features all at once this is the problem
features all at once this is the problem with big batch product releases how do I
with big batch product releases how do I
with big batch product releases how do I know which of the killer features within
know which of the killer features within
know which of the killer features within it right by doing that with taking away
it right by doing that with taking away
it right by doing that with taking away our ability to understand what’s going
our ability to understand what’s going
our ability to understand what’s going on
on
on same with metrics you’ve got to be very
same with metrics you’ve got to be very
same with metrics you’ve got to be very careful because it’s a system each thing
careful because it’s a system each thing
careful because it’s a system each thing you change can change five other things
you change can change five other things
you change can change five other things so actually you want to be very specific
so actually you want to be very specific
so actually you want to be very specific about what you measure and when for
about what you measure and when for
about what you measure and when for example if I’m a start-up I’m going to
example if I’m a start-up I’m going to
example if I’m a start-up I’m going to be looking at virality how quickly does
be looking at virality how quickly does
be looking at virality how quickly does my product get taken out their
my product get taken out their
my product get taken out their stickiness to people come back to it I’m
stickiness to people come back to it I’m
stickiness to people come back to it I’m probably not going to be looking at
probably not going to be looking at
probably not going to be looking at revenue so much to begin with because if
revenue so much to begin with because if
revenue so much to begin with because if no one uses it who cares right then I’ll
no one uses it who cares right then I’ll
no one uses it who cares right then I’ll bring in other measures as we go so you
bring in other measures as we go so you
bring in other measures as we go so you do have to know what you’re doing it’s
do have to know what you’re doing it’s
do have to know what you’re doing it’s not a simple thing to get going but
not a simple thing to get going but
not a simple thing to get going but figure out if this is our outcome what
figure out if this is our outcome what
figure out if this is our outcome what are the things to measure at this point
all right
all right so you say about that maybe HB
so you say about that maybe HB
so you say about that maybe HB split-testing and stuff like that can
split-testing and stuff like that can
split-testing and stuff like that can help when your release products etc so
help when your release products etc so
help when your release products etc so what I’m thinking is what about the
what I’m thinking is what about the
what I’m thinking is what about the marketing side they are doing stuff as
marketing side they are doing stuff as
marketing side they are doing stuff as well so how do I make sure that they are
well so how do I make sure that they are
well so how do I make sure that they are not affecting the same stuff I’m
not affecting the same stuff I’m
not affecting the same stuff I’m creating do you understand what I mean
creating do you understand what I mean
creating do you understand what I mean you might have two answers for that
you might have two answers for that
you might have two answers for that about marketing one is involved them I
about marketing one is involved them I
about marketing one is involved them I get them involved in all of the product
get them involved in all of the product
get them involved in all of the product discovery thinking because they can work
discovery thinking because they can work
discovery thinking because they can work at odds for example if the game is to
at odds for example if the game is to
at odds for example if the game is to get someone to buy a book marketing goes
get someone to buy a book marketing goes
get someone to buy a book marketing goes and puts 200 packages up that might get
and puts 200 packages up that might get
and puts 200 packages up that might get the marketing and come up but doesn’t
the marketing and come up but doesn’t
the marketing and come up but doesn’t actually help the user experience in
actually help the user experience in
actually help the user experience in fact it can detract from it that’s a
fact it can detract from it that’s a
fact it can detract from it that’s a problem and then the other part of the
problem and then the other part of the
problem and then the other part of the question was rummy I got back two
question was rummy I got back two
question was rummy I got back two o’clock the morning he can’t remember
o’clock the morning he can’t remember
o’clock the morning he can’t remember either
either
either he can’t remember what was it what was
he can’t remember what was it what was
he can’t remember what was it what was it about marketing those are really
it about marketing those are really
it about marketing those are really important thing that I totally forgot oh
important thing that I totally forgot oh
important thing that I totally forgot oh I’ll causation this is a big thing
I’ll causation this is a big thing
I’ll causation this is a big thing so one of the nerve owners is that we’ll
so one of the nerve owners is that we’ll
so one of the nerve owners is that we’ll have an outcome-based contract we’ve
have an outcome-based contract we’ve
have an outcome-based contract we’ve actually got three one-page contracts
actually got three one-page contracts
actually got three one-page contracts that I’ve developed with the UK lawyer
that I’ve developed with the UK lawyer
that I’ve developed with the UK lawyer called minimal viable contracts how
called minimal viable contracts how
called minimal viable contracts how original and one of them is a pure
original and one of them is a pure
original and one of them is a pure outcome based model that you only get
outcome based model that you only get
outcome based model that you only get paid if you deliver value and if you
paid if you deliver value and if you
paid if you deliver value and if you deliver it early you can get huge upside
deliver it early you can get huge upside
deliver it early you can get huge upside all the consultants can you imagine a
all the consultants can you imagine a
all the consultants can you imagine a world I save that company twelve million
world I save that company twelve million
world I save that company twelve million in three days if I got 10% of that how
in three days if I got 10% of that how
in three days if I got 10% of that how cool would that be right so now you get
cool would that be right so now you get
cool would that be right so now you get into causation ah but we just launched
into causation ah but we just launched
into causation ah but we just launched 200 other things how do we know that
200 other things how do we know that
200 other things how do we know that what you’ve done has impacted the
what you’ve done has impacted the
what you’ve done has impacted the results this is where you need to
results this is where you need to
results this is where you need to measure at different levels so remember
measure at different levels so remember
measure at different levels so remember I talked about the model being fractals
I talked about the model being fractals
I talked about the model being fractals so we have target outcomes and options
so we have target outcomes and options
so we have target outcomes and options underneath it for example you could say
underneath it for example you could say
underneath it for example you could say we want to increase revenue that’s a
we want to increase revenue that’s a
we want to increase revenue that’s a huge outcome for the company like Amazon
huge outcome for the company like Amazon
huge outcome for the company like Amazon I might be at the level where I know if
I might be at the level where I know if
I might be at the level where I know if I’m in the back-end server area if I can
I’m in the back-end server area if I can
I’m in the back-end server area if I can keep the system make it more available
keep the system make it more available
keep the system make it more available we go from 96% availability to 99 then
we go from 96% availability to 99 then
we go from 96% availability to 99 then we actually keep more people on the site
we actually keep more people on the site
we actually keep more people on the site I can show that that has an alignment to
I can show that that has an alignment to
I can show that that has an alignment to the high level outcome right so what
the high level outcome right so what
the high level outcome right so what you’ve got to be able to do is agree
you’ve got to be able to do is agree
you’ve got to be able to do is agree that this is a level that you’re
that this is a level that you’re
that this is a level that you’re comfortable with you want to keep
comfortable with you want to keep
comfortable with you want to keep linking it you should see some variation
linking it you should see some variation
linking it you should see some variation in the revenue but just as we could do
in the revenue but just as we could do
in the revenue but just as we could do something that improves that bottom line
something that improves that bottom line
something that improves that bottom line someone else could do something that
someone else could do something that
someone else could do something that tracks by it so we have to agree that
tracks by it so we have to agree that
tracks by it so we have to agree that our product level this is the right
our product level this is the right
our product level this is the right level to measure it there’s a little bit
level to measure it there’s a little bit
level to measure it there’s a little bit of an act of faith there but if you do
of an act of faith there but if you do
of an act of faith there but if you do enough testing and can get feedback
enough testing and can get feedback
enough testing and can get feedback again this gets into how you do split
again this gets into how you do split
again this gets into how you do split testing etc we can show that there is a
testing etc we can show that there is a
testing etc we can show that there is a link this is the hardest part about
link this is the hardest part about
link this is the hardest part about writing the contracts is to get that
writing the contracts is to get that
writing the contracts is to get that linkage correctly done so that we get
linkage correctly done so that we get
linkage correctly done so that we get the suppliers saying okay we’re going to
the suppliers saying okay we’re going to
the suppliers saying okay we’re going to do this that actually helps the
do this that actually helps the
do this that actually helps the grade-level outcome the other questions
grade-level outcome the other questions
grade-level outcome the other questions other questions okay thank you everybody
”
Be First to Comment