Press "Enter" to skip to content

GOTO 2015 • Why Scaling Agile Doesn’t Work • Jez Humble

00:00:11well what a great introduction I think

00:00:13that’s my my favorite introduction ever

00:00:16Thank You Marcus

00:00:17yeah I wrote this book with Barry and

00:00:20Joann because basically we wrote this

00:00:22book and it was like oh yeah that makes

00:00:24sense and then they try to do it and

00:00:26then they couldn’t because of all this

00:00:27other stuff and so I saw people and

00:00:29they’re like yeah we can’t do it because

00:00:30of this and so I got all the things that

00:00:33they said they couldn’t do it because of

00:00:34and wrote a book to explain why that was

00:00:36all and why they could actually

00:00:38do it so it’s basically it’s a book to

00:00:42arm yourself with when people say oh

00:00:44that’s a great idea but we can’t do it

00:00:46here because of X and you can say well

00:00:48actually so I don’t know if you’ll make

00:00:50many friends by doing this but you know

00:00:53that’s I can’t help with that so if you

00:00:57were here for my workshop this is the

00:00:59too long don’t read version of the

00:01:01workshop so you probably won’t learn

00:01:02anything new so please go and see

00:01:05something else instead unless you’re

00:01:07happy to sit here for 15 minutes and

00:01:09have me retell you bits of the workshop

00:01:11I won’t mind if you leave and even if

00:01:13you didn’t come to the workshop I still

00:01:15won’t mind if you leave that’s fine so

00:01:19why scaling agile doesn’t work and what

00:01:22to do about it I didn’t get straight to

00:01:24the point

00:01:25the reason scaling agile doesn’t work is

00:01:27because agile is just one piece of the

00:01:29puzzle the industry standard process for

00:01:32software development in most companies

00:01:34is still a process that I like to call

00:01:36water scrum ball and the I stole this by

00:01:44the way from Gaia Forrester called Dave

00:01:46West and the problem is this even if

00:01:48you’re doing kind of nice iterations

00:01:50here I’m going to try not to break all

00:01:54the cables will they do this yeah that’s


00:01:58even if you’re doing kind of agile stuff

00:02:00here often it’s the case who works in a

00:02:03company of more than a thousand people

00:02:07ok so he

00:02:10if you work in a big company and even if

00:02:13you work in a small company you probably

00:02:15have this thing where before anyone does

00:02:17any work you spend months you know

00:02:19someone has an idea you have to go and

00:02:21write a budget proposal it has to go

00:02:23through the budgeting cycle you have to

00:02:25go and do big long estimates and work

00:02:27out how much it’s going to cost and how

00:02:28long it’s going to take and then you

00:02:30have to do the design phase and then

00:02:31eventually some enormous document lands

00:02:34on someone’s desk and then then you can

00:02:36start development and we’re doing

00:02:38development in nice little iterations

00:02:40and that’s great but you can’t actually

00:02:41release anything before you do that your

00:02:44work and the work of the other teams has

00:02:46to be integrated and then it goes to

00:02:47central testing and then hopefully you

00:02:50manage to fix some of the bugs you found

00:02:51in testing and then it gets tossed over

00:02:53the wall to IT operations and only then

00:02:55does it actually go out to users and

00:02:57this is the point where we actually

00:02:58derive measurable customer outcomes from

00:03:02the work and so this overall cycle time

00:03:06or lead time depending on how you define

00:03:08the words the time from golf course to

00:03:11measureable customer outcome is usually

00:03:14pretty long and making this bit in the

00:03:16middle a j’l usually doesn’t make a big

00:03:18difference it might make a difference of

00:03:20a few percent but I mean whenever I go

00:03:22and do value stream maps at companies

00:03:24it’s usually like oh yes this bit takes

00:03:27nine months but let’s not talk about

00:03:28that because we can’t do anything about

00:03:30that and I’m like whoa that’s the bit

00:03:33you need to fix because it doesn’t

00:03:36matter how much you mess around with

00:03:37this stuff if it’s taking you nine

00:03:38months to go through this you’re not

00:03:42making a big difference on the overall

00:03:43outcomes so who who has who works at a

00:03:47place where you have this experience or

00:03:48maybe you have a friend who has this

00:03:49experience okay so I most of you alright

00:03:54so good you’re my audience so first of

00:03:57all what tends to happen when you do

00:04:01kind of agile in this environment is you

00:04:04end up creating a lot of teams and the

00:04:07way that that is usually made to work is

00:04:09you still have to do all this stuff and

00:04:11then you take all the work and you break

00:04:14it down into lots of tiny little bits

00:04:16and then you hand the tiny little bits

00:04:19out to all the teams and they go off and

00:04:21do the tiny little bits and then at the


00:04:24they come together and kind of try and

00:04:26make the system as a whole work and it

00:04:28doesn’t and you have to spend weeks or

00:04:31months actually trying to make it work

00:04:33and when it does work you find out that

00:04:34it wasn’t really what you wanted and

00:04:36you’re like oh

00:04:37but now we have to release it because

00:04:39it’s taken us a year or two years to get

00:04:42here and you know it’s on scope it’s on

00:04:44time it’s on budget so yeh so you

00:04:46release it and then maybe like me you

00:04:48had the experience where you rolled off

00:04:50the project and you enter do something

00:04:51instead and then like a year or two

00:04:54later you were talking to one of your

00:04:56colleagues he’s still on the project and

00:04:57they’re like oh yeah that thing that we

00:04:59built no one actually ended up using it

00:05:01it was cancelled or you know went to

00:05:03market wasn’t successful and that for me

00:05:05was the most miserable moment of my

00:05:07career this thing that I built that I

00:05:10spent a year on on this large team where

00:05:13we thought it was a success but then the

00:05:16thing actually never ended up making a

00:05:17lot of money for the company and after a

00:05:19couple of years it got canceled so agile

00:05:24doesn’t fix that problem because

00:05:28fundamentally the problem is the ideas

00:05:30that you had at the beginning wasn’t the

00:05:32right idea but it took you so long to

00:05:33find out that it wasn’t the right idea

00:05:35there you were you know you were screwed

00:05:39and then you look at the decision making

00:05:42process so I love running surveys

00:05:44surveys are a really rich source of

00:05:46confirmation bias and I ran a survey

00:05:50with Forrester to find out about agile

00:05:52practices one of the questions we asked

00:05:55is how the companies make investment

00:05:57decisions for products so 47% of people

00:06:02replied that the committee decides from

00:06:04potential options so decision by

00:06:06committee so any four percent of people

00:06:09said they use some kind of financial

00:06:11modeling they use economics to make

00:06:13their investment decisions whoo that

00:06:16seems like a good idea for a joke

00:06:20we put opinion of the person with the

00:06:22highest salary wins out and you can see

00:06:24I spelled salary wrong here so sorry

00:06:25about that but 13% of people answered

00:06:30that that’s how they made decisions this

00:06:32is called the hippo method the highest

00:06:34paid person’s opinion

00:06:36and that really is the same thing as

00:06:40committee decides from central options

00:06:41so that and that are actually the same

00:06:43thing and then nine percent of people

00:06:45said they use a product portfolio

00:06:46approach which is also the same thing as

00:06:49this and then some sentient people you

00:06:53know which I thought was very admirable

00:06:54said that they take no systematic

00:06:56approach which is basically also the

00:06:59same thing so what we can see here is

00:07:02that 76% of people are not using an

00:07:06economic framework and 24% of people are

00:07:08using an economic framework which is

00:07:10kind of depressing

00:07:11so that’s the state of software

00:07:14development and portfolio management

00:07:16investment today and in most big

00:07:20companies what you find is the output of

00:07:22this process is a few very big projects

00:07:24because somehow the annual budgeting

00:07:26cycle leads to a few big projects being

00:07:30the ones that get funded and so who has

00:07:33like a retirement savings account

00:07:36alright so I’ve got a product that I

00:07:38want to sell you if your retirement

00:07:40savings account and in my products I

00:07:42have four big things that I’m building

00:07:45and it’s going to take me a really long

00:07:48time to build them and at the end we

00:07:50don’t know if they’re going to be

00:07:51successful or not and it’s going to take

00:07:52us a really long time to find out but if

00:07:54they are successful they might make lots

00:07:56of money would you like to invest in my

00:07:58product no because you’re not stupid and

00:08:03yet that’s exactly what we do with the

00:08:06IT budgeting process for building

00:08:08projects in most cases we’d be better

00:08:11off taking our money and pressing it

00:08:13into unit trusts that would provide a

00:08:16better return on investment than our IT

00:08:18budget investments and the things that

00:08:21we do to mitigate the risk of being

00:08:22wrong don’t work so the main activity

00:08:26what’s the main activity we perform to

00:08:28make sure that we should really do the

00:08:29projects the main analysis activity huh

00:08:35business case and what’s the main kind

00:08:38of analysis activity we use to create

00:08:40the business case so return on

00:08:43investment is the main thing we want to

00:08:45know about but what do we spend our

00:08:47analysis time doing to find out if we’ll

00:08:50get rich

00:08:50investment I would love to to know that

00:08:56people actually put effort into finding

00:08:58out the potential business value that’s

00:09:00not where I see most people putting

00:09:02their effort in analysis what I mainly

00:09:05see the effort being is estimation so

00:09:08estimation of cost that’s where I mostly

00:09:10see people putting in a lot of time and

00:09:12effort is working with the developers to

00:09:14estimate the amount of work they’re

00:09:15going to do the problem with that is

00:09:17it’s pointless so there’s a guy called

00:09:20Douglas Hubbard who wrote a book called

00:09:22how to measure anything and he’s been

00:09:23looking at business cases for many many

00:09:25years and he does a kind of analysis

00:09:29called Monte Carlo in Monte Carlo you

00:09:31change the inputs to the business case

00:09:33and you see the impact of changing the

00:09:37values on what you care about which is

00:09:39normally either return on investment or

00:09:42lifecycle profits for the products those

00:09:44are the two variables you care about and

00:09:45what he found is that the cost has a

00:09:50very low information value varying the

00:09:52cost actually doesn’t have a huge impact

00:09:54on return on investment or lifecycle

00:09:57profits which is what you really care

00:09:58about the single most important unknown

00:10:02he found is whether the project will be

00:10:04cancelled the next most important

00:10:06variable is the utilization of the

00:10:08system including how quickly the system

00:10:11rolls out and whether some people will

00:10:13use it at all and those are the two

00:10:15things that you care about is it going

00:10:16to get cancelled and when it rolls out

00:10:19are people going to use it and that’s

00:10:21true for internal IT projects and for

00:10:24external light you know product

00:10:26development is are people really going

00:10:28to use it and that’s the thing we don’t

00:10:29normally spend a lot of time finding out

00:10:31and that’s for me the whole point of the

00:10:35Lean Startup is how we can cheaply run

00:10:37experiments to find out if people will

00:10:40actually use it and pay you money for it

00:10:41and that is way way way more important

00:10:44than any kind of estimation activity and

00:10:46yet it’s much more rare to find people

00:10:49doing that than it is to find them

00:10:50spending you know weeks or months doing

00:10:52in a very detailed estimation and then

00:10:55once you’ve done the estimation you take

00:10:57all the little bits you basically break

00:10:59it down a little bit estimated little

00:11:00bits add it all up and then you give the

00:11:02people all the little bits to do

00:11:03we’re back where we started so the other

00:11:07problem with this approach is that we

00:11:10create these huge projects and when we

00:11:13create huge projects we batch up all

00:11:15these little bits of work together in

00:11:17one big batch which we then push through

00:11:19the process and part of the reason we do

00:11:21that is because the transaction cost of

00:11:24taking work through the entire value

00:11:26stream is so high and so part of the

00:11:28point of continuous delivery is to make

00:11:31it economic to work in small batches so

00:11:33we can get the feedback much more

00:11:34quickly so that’s kind of a quick

00:11:37preview of the continuous delivery thing

00:11:40but when we batch up work what we do is

00:11:43we batch up that very high value things

00:11:45with lots of very low value things and

00:11:47you push them all out together there’s a

00:11:50really good case study called black swan

00:11:53farming using cost of delay this is a

00:11:56case study that was done on masks which

00:11:58was the world’s biggest or second

00:12:00biggest shipping line so a really big

00:12:01company with lots of IT projects and

00:12:04they had about 3,000 different features

00:12:06that were going through the value stream

00:12:09from analysis through development and

00:12:11delivery and one of the first things

00:12:13they did was try and estimate the value

00:12:15of these pieces of work and they did

00:12:18this using a method called cost of delay

00:12:20so if you go to this link here you can

00:12:23download the paper I really highly

00:12:25recommend this paper as a case study of

00:12:27how to implement lean agile at scale

00:12:31what they found that was really

00:12:33interesting is by using cost of delay to

00:12:35measure the value of the features so

00:12:38what cost ablai is briefly is how much

00:12:41it costs you per unit time per week in

00:12:45this case to not deliver the feature so

00:12:48every week that we’re not delivering

00:12:49this feature how much is it costing us

00:12:51in opportunity cost to not deliver it

00:12:53and that’s the way that you can

00:12:54prioritize work in dollar value by

00:12:57considering the opportunity cost I’m not

00:12:59delivering it and comparing it and what

00:13:00they found is of these thousands of

00:13:02requirements that were going through the

00:13:03system there were about three that it

00:13:06was costing the company you know between

00:13:08two and 2.8 million dollars per week to

00:13:11not deliver and then you’ve got this

00:13:13long tail of features where the cost of

00:13:16delay is you know

00:13:17in comparison with this and so it

00:13:20becomes very clear like what should we

00:13:23work on you’ve got this big IT team they

00:13:27should stop doing everything except

00:13:29these three bits of work and deliver

00:13:30these three bits of work as fast as they

00:13:32possibly can and stop doing anything

00:13:35else and when you’re not even exposing

00:13:38the value of what you’re doing and you

00:13:39know they were making assumptions to

00:13:41come to these numbers it can be quite

00:13:42hard to work out the dollar value of

00:13:44these features but when the variability

00:13:47is quite high it doesn’t even matter

00:13:50when there’s like an order of magnitude

00:13:52difference between the value doesn’t

00:13:53matter if I’m wrong by you know a factor

00:13:56of two who cares

00:13:58still going to be really obvious which

00:13:59which the features are and Joshua

00:14:03Arnold’s done a bunch of other projects

00:14:05where he’s analyzed things in this way

00:14:06and he’s found that this this shape this

00:14:08is a power-law curve this kind of power

00:14:11law distribution is very very common

00:14:13most places where they’ve gone and done

00:14:15looked at the the value of the features

00:14:17it’s been this power law curve so this

00:14:19is pretty typical of what you’ll find is

00:14:20that a few small very valuable things

00:14:23get bashed up with a ton of other stuff

00:14:25that’s low value and then when you have

00:14:27these big batches of work going through

00:14:28the system is really hard to prioritize

00:14:30the individual bits so what do we do

00:14:34first of all the whole project paradigm

00:14:38is based on the assumption that the

00:14:40features and requirements that we’ve

00:14:43created are in fact the right thing to

00:14:45do almost always as we’ll see that’s not

00:14:49true they are the wrong thing to do

00:14:51about two-thirds of the time the

00:14:53features that we want to build have zero

00:14:57or negative value and are not the right

00:14:59thing to do I have data to back this

00:15:01statement up which I will give you the

00:15:03reference to in the case of new product

00:15:06development the numbers are much much

00:15:08worse it’s more like 90% of the time the

00:15:11products the wrong idea so for

00:15:13established products with well known

00:15:15business models that work 2/3 of the

00:15:17time the features are 0 negative value

00:15:19with new products where there’s a larger

00:15:22amount of uncertainty over 90% of the

00:15:24time it’s the wrong idea so he shouldn’t

00:15:25optimize for the case way we think we’re

00:15:28right about what we’re building which is

00:15:30what the project

00:15:30paradigm’ does instead of focusing on

00:15:33cost and spending a lot of work on

00:15:34estimation we should focus on value and

00:15:37gathering information to justify the

00:15:40value of what we’re building one of the

00:15:43most important things we can be doing is

00:15:44creating feedback loops throughout our

00:15:46delivery process and making sure that

00:15:49the decisions we make whether that’s the

00:15:52products development decisions the

00:15:53investment decisions the requirement

00:15:55decisions are actually the right thing

00:15:56how can we get the feedback on those

00:15:58decisions as fast as possible and then

00:16:01you know within the development process

00:16:03we want feedback loops did I break the

00:16:06system did I introduce the performance

00:16:08degradation did I introduce a security

00:16:10hole this is part of the point of

00:16:11continuous delivery in the deployment

00:16:12pipeline is to create really effective

00:16:15rapid feedback loops within our delivery

00:16:17process the point of continuous delivery

00:16:20is to make it economic to work in small

00:16:22batches and working in small batches is

00:16:24what us what allows us to get these

00:16:28feedback loops working so that we can

00:16:30get those feedback loops operating at a

00:16:32high frequency by working in small

00:16:34batches we get feedback at high

00:16:35frequency and that allows us to

00:16:38course-correct much more quickly which

00:16:41is the whole point of agile and when

00:16:44we’ve done these things then we can

00:16:45enable an experimental approach to

00:16:47product development based on the

00:16:49scientific method I have a hypothesis

00:16:51about a feature that I want to build how

00:16:53do I test whether my hypothesis is

00:16:55correct what data can I gather to

00:16:57validate my hypothesis and we can apply

00:16:59this experimental approach not just to

00:17:01product development but also to process

00:17:03improvement and continually work to

00:17:06improve the quality of our development

00:17:08and delivery processes so the first idea

00:17:11that I want to attack is this idea of

00:17:14requirements whose requirements are they

00:17:18are they the users requirements users

00:17:24don’t know what they want users know

00:17:27what they don’t want once you’ve built

00:17:28it for them they’re the requirements of

00:17:32the hippo the highly paid person and so

00:17:38I don’t like this word requirements what

00:17:39we have is hypotheses

00:17:42and typically there’s more than one way

00:17:45to achieve the outcome we want to

00:17:46achieve and a lot of the time we’re not

00:17:48even thinking about the outcomes so

00:17:50who’s who’s who uses stories in their

00:17:53projects as a I want them so that yay

00:17:56okay lots of you okay keep your hands up

00:17:59if you’ve written a story where you kind

00:18:03of left out the so that because it was

00:18:05kind of a bit hard to think about it

00:18:06keep your hand up I have yeah most of us

00:18:11have done that at one time or the other

00:18:13and it’s actually pretty common who’s

00:18:15worked on a project where like the so

00:18:17that is missed out most of the time like

00:18:19more than 50% of the time yeah so you

00:18:22know that’s life but I think that’s I

00:18:29mean we all know that that’s the most

00:18:30important bit but in the day to day kind

00:18:32of work process it’s kind of easy to

00:18:34forget that because you know that it’s

00:18:35important and we’ve got to do it so

00:18:37let’s just do it I think we should focus

00:18:38much more on the outcomes so one of my

00:18:40favorite tools is this one by glyco ads

00:18:43they’re called impact mapping it’s a

00:18:45very simple idea but it’s very powerful

00:18:46and his idea is this we should work

00:18:49backwards from the outcome so this is

00:18:52the outcome we want to create this is

00:18:53just an example right reduce transaction

00:18:56costs by 10% for a trading platform so

00:18:58this is a impact map for a trading

00:19:00platform reduce transaction costs by 10

00:19:02percent who are the stakeholders well

00:19:04there’s a German settlement team there’s

00:19:06I didn’t plan by the way that I would

00:19:08deliver this in Berlin just happen that

00:19:10way there’s some traders there’s IT

00:19:12operations here at the second level are

00:19:15the ways in which these stakeholders

00:19:18could help or hinder this outcome and

00:19:21then at the next level are the actual

00:19:23stories these this is the what what you

00:19:26could do to to achieve that outcome and

00:19:29when we do analysis what happens is you

00:19:33pick one and then you throw everything

00:19:35else away and then that one thing is

00:19:38what goes downstream to development and

00:19:40then tidy operations and that’s a real

00:19:43problem a lot at the time the developers

00:19:45never even see the rest of this map they

00:19:47just go improve exception reports so

00:19:50that you know trading costs go down and

00:19:53often the number which is really

00:19:56like how are we going to measure the

00:19:58success of whether we’re done is thrown

00:20:01away so actually having the key thing

00:20:04about the impact map is not creating the

00:20:06impact map the key thing about the

00:20:07impact map as with most agile practices

00:20:09is the shared understanding of the whole

00:20:12team which is created when you build the

00:20:15impact map so we have this thing in

00:20:17agile that’s everywhere where you focus

00:20:19on the artifacts the artifacts not the

00:20:22important thing the shared understanding

00:20:23of the team that created the artifact is

00:20:25actually the important thing so here

00:20:27what’s crucial is having the developers

00:20:29and the operations people and the

00:20:31business people all work together to

00:20:33create the impact map and then pick one

00:20:36like which is the thing that we think

00:20:38will be the smallest possible work for

00:20:40the biggest outcome because as Jeff

00:20:42Patton likes to say what we want to do

00:20:44is minimize output maximize outcomes so

00:20:47we want to maximize the outcome what’s

00:20:49the minimum amount of work we can do to

00:20:51achieve this outcome and you pick one

00:20:52and then you design an experiment to

00:20:54test whether it will in fact achieve

00:20:56this outcome so the other thing I really

00:20:58like that fits with this really nicely

00:21:00is Jeff golf’s template for hypothesis

00:21:04driven delivery so instead of the story

00:21:06format he has this format we believe

00:21:08that building this feature for these

00:21:10people will achieve this outcome wellnot

00:21:13is successful when we see this signal

00:21:16from the market what’s the signal we’re

00:21:18going to measure to prove that the thing

00:21:19we’re building will actually achieve the

00:21:21outcome so we believe that building this

00:21:23feature building this feature for these

00:21:26people for these people will achieve

00:21:29this outcome this outcome wellnot is

00:21:32successful when we see this signal from

00:21:33the market what are we going to measure

00:21:34to demonstrate the feature will actually

00:21:36deliver and there’s many ways we can

00:21:39measure the outcome using an experiment

00:21:44most of this comes from the UX movement

00:21:47and one of the big trends in agile is

00:21:50integrating UX but often what I see

00:21:54maybe you have friends you have this

00:21:55problem is the UX is basically making it

00:21:57look pretty that’s not the point of view

00:22:00X the point of view X is to help us

00:22:04think about delivering things in a way

00:22:07that will satisfy our customers

00:22:09so the whole way in which our customers

00:22:12interact with our products and our

00:22:14company is all within the realm of UX

00:22:16including will the feet features will

00:22:19the products actually make our customers

00:22:21happy and our users happy so this is

00:22:25from Janice Fraser and she divides up

00:22:28user research based on whether it’s

00:22:30quantitative or qualitative and then

00:22:31whether it’s evaluative or generative so

00:22:34generative in design thinking there’s

00:22:37two phases there’s the bit where you

00:22:38generate ideas which is where you can

00:22:41have come up with lots of different

00:22:42options so building an impact map is an

00:22:44example of a generative behavior and

00:22:47then evaluative is when you narrow down

00:22:50the options and pick one so that’s a

00:22:52conversion activity that’s the process

00:22:54of deciding which option is the right

00:22:55one so you can use user research to

00:22:58create the whole bunch of ideas and then

00:23:00you can use different techniques to find

00:23:01out which of those ideas is actually

00:23:03going to produce the required outcome

00:23:04but there’s loads of ways to do this

00:23:06that don’t involve actually building out

00:23:09the whole feature and so you should be

00:23:11thinking very carefully about how we can

00:23:13gather data without building out the

00:23:15whole feature and then finding out

00:23:17whether it was a good idea because

00:23:18usually what happens is even if you find

00:23:20out it wasn’t a good idea

00:23:21there’s the sunk cost fallacy you don’t

00:23:24want to admit you spent all this time

00:23:25building this thing that’s useless who’s

00:23:28actually deleted a big feature from

00:23:30their products in the last year okay

00:23:34that’s pretty good actually I’m pretty

00:23:35impressed did you delete that because

00:23:37you were like this isn’t delivering us

00:23:39valley cool well good for you

00:23:43that should be more common than it is so

00:23:47in terms of an alternative to water

00:23:49scrum fall I like to show this slide

00:23:51from Amazon this is from 2011 they’re

00:23:54about an order of magnitude better right

00:23:55now they are making changes to

00:23:57production on average every 11.6 seconds

00:23:59up to a thousand and 79 deployments per

00:24:02hour up to 10,000 boxes receiving

00:24:04deployment up to 30,000 on average

00:24:0810,000 up to 30 thousand boxes receiving

00:24:10deployment he was certainly calls talk

00:24:12last night okay lots of II so I went

00:24:16spend too much time on this but the

00:24:18reason one of the reasons they did this

00:24:19was to get this which again Nicole


00:24:23so ron iike harvey helped to build

00:24:26amazon’s experimentation platform he was

00:24:28in charge of amazon’s experimentation

00:24:30platform then he went on to work for

00:24:32microsoft and was in charge of the Bing

00:24:34still in charge of being experimentation

00:24:37platform so he has loads of data from a

00:24:40feature development and this is I mean

00:24:44this is the important piece of data we

00:24:46could be spending two-thirds of our time

00:24:49on the beach skiing at home with our

00:24:56kids if only we knew the two-thirds of

00:24:59the features that we builds that have

00:25:00zero or negative value to our company

00:25:03and people don’t think of the negative

00:25:05value case the case where you’re

00:25:07actually making things worse and that

00:25:10kills us in three ways firstly there’s

00:25:13the opportunity cost of not building

00:25:15something the would’ve delivered value

00:25:18second features add complexity to our

00:25:22system they have to be maintained

00:25:23forever so the maintenance cost and

00:25:26third that complexity slows down the

00:25:29rate at which we can add new features so

00:25:31it makes new feature development slower

00:25:33so if you’re not doing experiments to

00:25:37find out if your features are actually

00:25:39valuable you’re killing yourselves in

00:25:42three in these three ways is the biggest

00:25:44source of waste in software delivery is

00:25:45the stuff that we build that delivers

00:25:47zero or negative Valley and so who is

00:25:50actually running experiments to test the

00:25:51value of the features and products

00:25:53they’re building okay there’s a few of

00:25:56you maybe 10 hands went up out the whole

00:25:59audience like this is the biggest thing

00:26:02in my opinion that we need to change our

00:26:04industry is be thinking about how we can

00:26:06actually gather data to validate whether

00:26:08the things we’re doing are actually

00:26:09delivering value for our customers and

00:26:11it doesn’t need to be a be testing it

00:26:13could just be other forms of user

00:26:14research the cool thing for me about

00:26:23software is that software has different

00:26:28characteristics from architecture in to

00:26:31in in terms of buildings right so people

00:26:35often say I wish software could

00:26:37more like buildings meaning that they

00:26:39wish it could not fall down and be

00:26:41reliable and be secure and things like

00:26:42that but actually I think that’s that’s


00:26:45we don’t want software to be like

00:26:47buildings because software has these

00:26:49really unique affordances software is

00:26:51easy to change even at scale compared to

00:26:54buildings if I want to really check this

00:26:56building I have to pull it down and

00:26:58start again from scratch you don’t have

00:27:00to do that with software helps of people

00:27:02do do that with software which is

00:27:04unfortunate but you know you can

00:27:06actually rebuild the plane while it’s in

00:27:08flight with software you can make

00:27:09architectural changes incrementally it’s

00:27:12entirely possible to do that people have

00:27:14done that very successfully software is

00:27:15much cheaper to change you can get value

00:27:18from software before you finish building

00:27:20it like I you couldn’t be in this

00:27:22building if there were no ceilings here

00:27:24but I can develop a piece of software

00:27:26which doesn’t serve all of your needs

00:27:29but might serve the needs of a small

00:27:30segment of you and I can and you might

00:27:32be able to get value from that from

00:27:33really early on so software has these

00:27:35unique affordances that makes it much

00:27:37easier to experiment and try things out

00:27:39and we can’t do that with buildings and

00:27:41with physical architecture and what’s

00:27:45cool about that is we can use the same

00:27:47techniques for experimental product

00:27:51development that we that the lean

00:27:53movement has been using for years for

00:27:56process improvement so to show you what

00:27:59I’m talking about I’m going to give you

00:28:01a case study and the case study is from

00:28:03HP LaserJet firmware division I’ve

00:28:05talked about this before maybe you’ve

00:28:07seen me but I’m going to focus on how

00:28:11they made the transition so this was a

00:28:13team that in 2008 had this really big

00:28:18problem which is that they were on the

00:28:20critical path for software development

00:28:21every new range of printers they wanted

00:28:23to launch they were going to have to

00:28:25build new firmware and that was going to

00:28:28take longer than actually building the

00:28:30chips that went into those printers so

00:28:32that was a really serious problem I

00:28:34actually did some work for a European

00:28:37airline and they were introducing you

00:28:40know those the economy seats with bigger

00:28:42gaps between the seats right like

00:28:45Premium Economy more legroom it was

00:28:47going to take them longer to change the

00:28:49booking system

00:28:50so that you could book the seats the net

00:28:53was going to take them to put the seats

00:28:55into the aeroplanes right which is how

00:28:58you know there’s something really badly

00:29:00wrong with your software delivery

00:29:01process so there’s the same thing here

00:29:04it’s going to take them longer to build

00:29:06the firmware than it was going to take

00:29:07them to fabricate these custom chips

00:29:09which had a one-year lead time to be

00:29:11able to build so they wanted to they

00:29:13tried a whole bunch of different things

00:29:14and eventually they decided to look at

00:29:16their software delivery process what

00:29:18they found is they were spending a whole

00:29:20bunch of time doing non value add things

00:29:22like code integration very detailed

00:29:24planning porting code between branches

00:29:27every time they released a new line of

00:29:28devices they took a branch in version

00:29:31control which meant that when they fixed

00:29:32a bug in one line of devices and they

00:29:34needed support the bug fix they would

00:29:36have to port it across all these code

00:29:38branches same with features they were

00:29:40spending 25% of their time porting

00:29:42features and bug fixes across branches

00:29:4425% of their time on product support

00:29:47what is this tele quality problem 15

00:29:52percent of their time on manual testing

00:29:54if you subtract this from a hundred

00:29:55percent what you find is that only five

00:29:57percent of their investment was actually

00:30:00being spent on building features and by

00:30:03the way all the product management

00:30:04people you know they would say can we

00:30:06get budget for you know adding test

00:30:08automation can we spend some time

00:30:10refactoring and management be like no no

00:30:13you’ve got to build the features who’s

00:30:15had that happen to them right and it’s

00:30:18very clear to see the fallacy in that in

00:30:22that idea because here’s the thing the

00:30:25reason that there’s all this pressure to

00:30:27build the features the reason you’re

00:30:29going so slowly is because you’re

00:30:30spending all this amount of money on

00:30:32things that are not adding value the

00:30:35only way you can get that problem is by

00:30:37fixing this by removing this that will

00:30:39allow you to go faster so you have this

00:30:42vicious cycle you know pressure you have

00:30:45to fix you have to do the features why

00:30:47is the pressure there because we’re

00:30:48doing all this other stuff it’s so

00:30:49painful to deliver anything and if we

00:30:51don’t fix this it’s always going to get

00:30:52worse and worse and worse until we’re

00:30:54just driven into the ground they also

00:30:57looked at their cycle times is taking a

00:30:59week to get changes into trunk they were

00:31:01getting one or two builds a day and it


00:31:04six weeks to do a full manual regression

00:31:05on their software before they could

00:31:07release it so they didn’t want to do

00:31:10continuous deployment they didn’t want

00:31:12to release new firmware ten times a day

00:31:14I mean who upgrate who’s upgraded their

00:31:16printer firmware okay as a few yak

00:31:21shavers in the audience that’s good most

00:31:24the time you don’t want to upgrade your

00:31:26firmware ten times a day but what they

00:31:29found is by implementing continuous

00:31:31delivery by working in small batches it

00:31:35changed the economics of the software

00:31:37delivery process because it created

00:31:40feedback loops that enable them to build

00:31:43quality in rather than try and test

00:31:45quality and at the end and over the

00:31:48course of two years and well first of

00:31:50all they re architected in a big bang’

00:31:52way which is always very risky that I

00:31:54don’t recommend it but in this case it

00:31:55works and I’ll explain why later but

00:31:58they really tected such that they didn’t

00:32:02have branches for different ranges of

00:32:05devices anymore instead they had one

00:32:07firmware build that would work on any

00:32:09set of devices so the firmware boots

00:32:11looks at the hardware profile says oh

00:32:14I’m a printer I’m going to turn these

00:32:17features off I’m going to keep these

00:32:18features on or boots looks around says

00:32:20oh I’m a scanner so I’m going to turn

00:32:22these features off and turn these

00:32:23features on so it’s basically feature

00:32:24toggles for architectural differences so

00:32:29a guy I know called danbo dot has a

00:32:31saying feature branching is a poor

00:32:33person’s modular architecture so this is

00:32:38a case where they re architected so they

00:32:39wouldn’t have to use branches and that

00:32:42allowed them to work on trunk and do

00:32:44continuous integration and build a very

00:32:46sophisticated test automation suite

00:32:48where they had over 30,000 hours of

00:32:51tests that would actually send signals

00:32:52to the logic boards and then get signals

00:32:55back as part of their test automation so

00:32:57anyone who complains about test

00:32:59automation for websites I actually I

00:33:02don’t have it today but I often carry

00:33:04around a copy of this book to spank

00:33:06people who whine about test automation

00:33:08because now here’s people who did it

00:33:10with logic boards that’s super cool so

00:33:14they built a very sophisticated set of

00:33:15automated tests anytime anyone

00:33:18act in they used get so every developer

00:33:20so 400 developers by the way so at scale

00:33:23400 developers on three different

00:33:25countries Brazil USA India or pushing

00:33:30changes in to get they each had their

00:33:33own little repo and they built a CI tool

00:33:35that looked at those repos anytime so

00:33:39and pushed at range it would take that

00:33:40change and run two hours worth of

00:33:42automated tests against that change in a

00:33:44simulator if that works

00:33:46it gets promoted to the stage two and

00:33:50then what happens in stage two is all

00:33:52the changes in the last time period get

00:33:55merged and then they run automated tests

00:33:57in the simulator against that merged set

00:33:59of changes if that fails the developers

00:34:01get an email here the test that broke

00:34:03here’s a button to be able to reproduce

00:34:06those test failures on your development

00:34:07workstation that’s super important if

00:34:10developers can’t reproduce the

00:34:11acceptance test failures that’s a very

00:34:13serious problem it’s an architectural

00:34:15problem that you need to fix if they


00:34:20that’s the only way you can get into

00:34:21trunk so to get into trunk anything that

00:34:25passes stage 2 gets into trunk and

00:34:27that’s the only way the developers can

00:34:29get into trunk so this basically fixed

00:34:33the problem where the bills always read

00:34:34because the developers don’t care about

00:34:36it because guess what the only way you

00:34:38get into trunk is if the the bill is

00:34:39green out of level 2 this runs on a big

00:34:45rack of servers in a simulator another 2

00:34:48hours worth of automated tests then you

00:34:50get promoted to level 3 which runs

00:34:51actually on physical logic boards with

00:34:54emulators and then if that works then it

00:34:57gets promoted to the overnight level 4

00:34:59and that’s the entire 30,000 hours worth

00:35:01of test run in parallel so you get the

00:35:03results overnight and so you can find

00:35:06out if the firmware is releasable within

00:35:0824 hours of making the check-in so they

00:35:11completely removed the six-week

00:35:13stabilization phase that just goes away

00:35:15and the whole point of continuous

00:35:18delivery by the way is to get rid of

00:35:22this whole thing like this whole thing

00:35:24should all go away even if you’re not

00:35:27releasing 10 times a day you should be

00:35:30able to release yourself

00:35:31I should always be provably reducible on

00:35:34a daily basis that’s the point of

00:35:37continuous delivery and so even if

00:35:39you’re not doing continuous deployment

00:35:40that produces huge benefits they

00:35:43massively reduce the amount of time

00:35:44they’re spending on code integration and

00:35:46on planning and on porting K between

00:35:48branches product support activities goes

00:35:51down from 25% of costs to 10% of course

00:35:53what does that tell you higher quality

00:35:58so this is how they measured the quality

00:36:00improvement manual testing goes down for

00:36:0315 percent to 5 percent costs invested

00:36:07in innovation go from 5 percent to 40

00:36:10percent so 8x improvement in

00:36:12productivity measured in terms of the

00:36:14amount of investment going towards

00:36:16innovation what do the arithmetic in the

00:36:19audience notice about these numbers less

00:36:23than 100% because there is a new

00:36:26activity creating maintaining evolving

00:36:31the Suites of automated tests if you

00:36:34enter your manager right now and said

00:36:35please can we invest 23 percent of our

00:36:38budget in test automation

00:36:39what would your manager say that’s a

00:36:45rhetorical question and yet here we have

00:36:48the business case for doing that the

00:36:50business case is quite clear despite the

00:36:52fact you’ve got 23 percent of your

00:36:53budget spent on test automation that has

00:36:55enabled an 8x improvement in cost spent

00:36:59on innovation because it’s massively

00:37:01reduce the amount of non value I’d work

00:37:02we’re doing and this is how lien works

00:37:04lien works by investing in removing

00:37:07waste so that you can increase

00:37:11throughput and the reason I like to talk

00:37:17about this case study is because they

00:37:18wrote it into a book and you can get the

00:37:19book and here are the numbers to take to

00:37:23your CFO to explain why they should

00:37:26investing its newest delivery what’s

00:37:31interesting what I’m going to spend the

00:37:32last few minutes on out of 10 minutes

00:37:35right yep is how they did it because how

00:37:39they did it was really interesting they

00:37:42did not create a three year plan

00:37:45detail of how they were going to do this

00:37:48what they did is they had a goal their

00:37:52goal was to fold one get firmware off

00:37:56the critical path goal to 10x

00:38:00productivity increase so those that was

00:38:03it those that’s all they wanted to do

00:38:05they had no idea how they would do it

00:38:06but they set these two very measurable

00:38:10easy-to-understand goals like everyone

00:38:12in this room can understand those goals

00:38:14it’s really obvious and it’s really

00:38:16obvious when you’ve achieved them and

00:38:17when you haven’t achieved them that’s

00:38:19super important anytime you’re

00:38:20implementing agile or DevOps or any

00:38:23methodology step zero is to understand

00:38:26your goals in measurable terms what goal

00:38:29are we trying to achieve how will we

00:38:32know when we’ve achieved it there’s

00:38:34acceptance criteria for organizational

00:38:36transformation what’s our goal if you if

00:38:40you implement agile because agile is

00:38:41cool or because it will make things

00:38:43better you will fail you’ve got to

00:38:46understand what the goals you’re trying

00:38:48to achieve are for your organization in

00:38:49measurable terms and then we want to

00:38:52take an experimental approach to

00:38:53achieving those goals so what some

00:38:56intermediate goal which will get us some

00:38:58of the way there let’s try a bunch of

00:39:00things out to find out if we can achieve

00:39:01that intermediate goal if we don’t

00:39:03succeed we’ve learned something but we

00:39:05haven’t spent too much time learning it

00:39:07which is important and if we succeed

00:39:10that’s great let’s try the next step so

00:39:12this is crucial we’re taking an

00:39:15incremental approach to process

00:39:17improvement and to changing

00:39:19organizational behavior in the same way

00:39:22that we take an incremental approach to

00:39:24product development and that for me was

00:39:26like the big aha moment I had writing

00:39:28the book is well guess what the

00:39:30incremental iterative approach doesn’t

00:39:31just apply to how we do product

00:39:32development it also applies to how we do

00:39:34process improvement and how we improve

00:39:36our companies and our processes and it’s

00:39:38something we should be doing habitually

00:39:40as part of our daily work and what was

00:39:43cool is that this was a case study for

00:39:45something that only got written about

00:39:47subsequently which is true I mean they

00:39:49were doing it in its livery

00:39:50before I wrote the continuous delivery

00:39:51bug and they didn’t call it continuous

00:39:53delivery they called it getting better

00:39:55at engineering right didn’t need a name

00:39:58is he was

00:39:59like oh yeah we’re just trying to get

00:40:00better at what we’re doing and and in

00:40:03the same way they also simultaneously

00:40:05invented this thing the improvement

00:40:07cutter so a cutter comes from Japanese

00:40:10martial arts it’s a habit

00:40:12it’s sorry it’s a base practice that you

00:40:14practice over and over again until it

00:40:16becomes habitual and it’s the same idea

00:40:18whereas music you practice scales over

00:40:21and over again that’s the first thing

00:40:22you do until you get really good at it

00:40:23and then you move to higher order

00:40:25practices that combine the things you


00:40:27same thing with sports you learn the

00:40:29basic moves for tennis and then you’re

00:40:31able to combine those any human creative

00:40:33activities the same thing you learn the

00:40:35basic moves first and then you until

00:40:37they become habitual and then you

00:40:39combine them into higher-order creative

00:40:41processes so his point is with this book

00:40:45improvement cutter based on the study of

00:40:47Toyotas management method is that

00:40:49improvement work getting better or what

00:40:52we do should be a habitual process and

00:40:54here’s the basic practice that you have

00:40:56to practice over and over again in order

00:40:59to get better first understand the

00:41:01directional challenge so HP LaserJet 10x

00:41:04productivity increase get firmer off the

00:41:06critical path number to grasp the

00:41:08current condition current condition step


00:41:18establish the next target condition so

00:41:21I’m going to show you what a target

00:41:22condition looks like but it’s basically

00:41:24an intermediate measurable set of goals

00:41:27for the program as a whole

00:41:29and then you don’t plan how you’re going

00:41:31to do it instead you specify the

00:41:34outcomes and then you allow the people

00:41:36to experiment with ideas to achieve

00:41:38those outcomes in the same way as we do

00:41:41with product development and then every

00:41:43day everyone’s asking themselves what’s

00:41:45the challenge we’re trying to achieve

00:41:46right now what’s the actual condition

00:41:49what obstacles are in our way what are

00:41:52we going to try next when can we see go

00:41:55and see what we learned from taking that

00:41:56step and so the difficult bit of this is

00:41:59working out the targa condition so I’m

00:42:02going to give you an example from HP

00:42:04LaserJet firmware this is 30 months in

00:42:08but these were their entire goals for

00:42:11the whole program of work for

00:42:13100 people on three continents this was

00:42:16their plan for the month it fits on one

00:42:19piece of paper and this is quite long

00:42:22when they started their monthly plans

00:42:24for the program of work were three or

00:42:27four bullet points but crucially they

00:42:30had measurable out outcomes so for month

00:42:3530 it was Priority One issues open less

00:42:38than one week level two test failures 24

00:42:40hour response final priority one change

00:42:43request fixed reliability error rate at

00:42:45release criteria and these are ordered

00:42:46by priority so you you want to get these

00:42:50ones done before you start on these ones

00:42:52these ones are the most important so

00:42:54these are all measurable and here’s the

00:42:57thing because this is the only thing

00:43:00that’s on the program plan and it’s just

00:43:02the outcomes it’s up to the teams to

00:43:04experiment with ideas to achieve those

00:43:06things in most of the scaled agile

00:43:08frameworks where you break up the work

00:43:10into little bits you hand it out to all

00:43:12the people and then you come together at

00:43:14the end and it doesn’t work properly

00:43:15everyone says well I did my bit it was

00:43:18those guys now usually that’s not true

00:43:23usually actually everyone did do that

00:43:25bit the problem wasn’t that people

00:43:27didn’t do their bit the problem was that

00:43:29the bits didn’t fit together or weren’t

00:43:32the right things to do it was an

00:43:33analysis problem in this model nobody

00:43:37succeeds unless everybody succeeds

00:43:39there’s no I did my bit either we all

00:43:43succeed as a team or we fail as a team

00:43:45and because it’s only one month the

00:43:49impact of a failure isn’t too bad the

00:43:51point of failure is to expose problems

00:43:53so we can fix them not so that we can

00:43:57blame people for being done well no

00:43:58guess what the reason we can do it why

00:44:00can we do is because of this Oh our

00:44:01assumption about this was wrong so let’s

00:44:03make different objectives for next month

00:44:06that address the underlying problems

00:44:09so I guess my conclusion for this is we

00:44:17want to take an experimental approach

00:44:18not just to product development but also

00:44:20to process improvement the processes

00:44:23that you have are the wrong processes

00:44:26and you’ll know

00:44:27I have the right ones but the important

00:44:28thing is to constantly think about

00:44:30improving your processes with the

00:44:32particular goal in mind and we should

00:44:34all be doing that if you leave your

00:44:36development process is the same they

00:44:38don’t stay the same what happens is they

00:44:40gradually degrade over time and become

00:44:42worse and worse if you’re not constantly

00:44:43working to improve then by default

00:44:45you’re gradually getting worse and so I

00:44:48want to end by just what I started with

00:44:52don’t autumn eyes for the case where

00:44:53you’re right assume that what you’re

00:44:54doing is not the best thing you could be

00:44:56doing focus not on cost not on

00:45:01estimation but on how we can best

00:45:03provide value in our products and in the

00:45:05processes that we’re running build

00:45:08feedback loops and grow and water and

00:45:10tend to your feedback loops to validate

00:45:13your assumptions and expose your

00:45:14assumptions work to make it so that you

00:45:17can get feedback in small batches both

00:45:19for your product development and for

00:45:21your process improvement work so that

00:45:23you can take an experimental approach

00:45:24not just a product development but also

00:45:26to process improvement so hopefully you

00:45:29have a couple of minutes for questions

00:45:30in the meantime please remember to rate

00:45:32this session I value your feedback also

00:45:35if you want a bunch of my free stuff

00:45:37email Jess humble at send your sliced

00:45:39calm with the subject DevOps and you’ll

00:45:41get a bunch of free stuff including

00:45:43these slides so that’s Joe Sample at

00:45:44send your sliced calm DevOps questions

00:45:46Marcus has a microphone questions

00:45:54who’s first god I have questions from

00:45:59the tool okay which are about one guy

00:46:03said you know it’s easy to do the split

00:46:05testing and so on yeah it’s not as easy

00:46:08is analyzing and in a similar vein it’s

00:46:11like great that we should evaluate cost

00:46:14of delay or value but how do we do it

00:46:17how do you do there’s loads of different

00:46:19ways to do it I could and have spent a

00:46:22whole day talking about mechanisms to do

00:46:24that cost of delay I really recommends

00:46:27looking at this case study from

00:46:31mask-like they talk a lot about cost of

00:46:33delay in this case study and how to

00:46:35evaluate it but the key thing is this

00:46:37anytime you have an idea you’re going to

00:46:40make assumptions

00:46:41and people tend to find fight about the

00:46:42ideas this idea is good know this idea

00:46:44is good what you want to expose is what

00:46:47assumptions are you making and let’s

00:46:49test the assumptions and find ways to

00:46:51test the assumptions and there’s loads

00:46:54of way to do that like one of my

00:46:55favorite stories is from Zappos so

00:46:57Zappos sells shoes online the first

00:47:00version of the product

00:47:01there was no supply chain what happened

00:47:04was any time someone ordered some shoes

00:47:06the guy would go to the shoe shop buy

00:47:09the shoes and then post them so this is

00:47:13the art of experimentation is finding

00:47:15cheap ways to achieve to run experiments

00:47:19without actually building the thing and

00:47:20I agree that it’s difficult but you know

00:47:22we’re all software developers we didn’t

00:47:24get into software development because it

00:47:26was easy right so you know when people

00:47:28like oh that’s difficult I’m like yes of

00:47:30course it’s difficult I think if you

00:47:32want an easy job you know you came into

00:47:34the wrong field and but I what I do

00:47:36agree is that it’s not a very well known

00:47:38not very well established field so what

00:47:43I urge you to do is come up with

00:47:45ingenious ways to experiment test your

00:47:47assumptions and then blog about them and

00:47:49share them because as a community we all

00:47:51need to get much better at sharing our

00:47:52ideas for doing these things increasing

00:47:54a body of knowledge around it one more


00:47:58ok I hope over

00:48:10hi hi I would like to ask how how do you

00:48:14start with this because I think most of

00:48:15the guys here are at the like not the

00:48:19top of the food chain in a company and

00:48:21and so do you do you recommend rather

00:48:26buying the books and getting it as a

00:48:28present Christmas present to your

00:48:30classes or or

00:48:33or starting at your position to

00:48:36experiment and ensure the good results

00:48:39so I’m certainly not going to complain

00:48:40if you buy my book for your boss but I

00:48:42think you need to do both right there’s

00:48:44a part of this which is guerrilla

00:48:46experimentation like try and find smart

00:48:49ways to to do this in your own backyards

00:48:52and experiment with it and gain

00:48:53confidence they can actually work and

00:48:55find other people in your company you

00:48:57have who want to try this stuff like you

00:49:00know the best hack of all is finding

00:49:02someone who you don’t normally talk to

00:49:05in your company and taking them out for

00:49:07lunch like find your database

00:49:08administrator take them out for lunch

00:49:10find out why they hate you find out one

00:49:13thing you can do to make their life a

00:49:14little bit better and do that and you

00:49:16know do that once a week just randomly

00:49:18pick someone in your company you don’t

00:49:20normally get to hang out with

00:49:21try and pick the person who you’re like

00:49:24cursing under your breath when something

00:49:25doesn’t work right you’re like that

00:49:27asshole in UX again they’ve sent me this

00:49:30terrible screenshot and I can’t like

00:49:32there are idiots go and take them out

00:49:34for lunch go and find out what’s

00:49:36bothering them there’s a great quote

00:49:38from Jesse Robbins he dirt he says don’t

00:49:41fight stupid make more awesome and I

00:49:45think that’s very important try and find

00:49:47ways to make things more awesome and

00:49:48find the people who you think are being

00:49:50stupid find out why it’s normally

00:49:51because they’re really frustrated about

00:49:53something or they just don’t know what’s

00:49:55going on because we don’t have feedback

00:49:56loops and try and find ways to help make

00:49:59that person more awesome

00:50:00and if we all did that everyday that

00:50:02would be really cool what I will say is

00:50:04it’s hard to create lasting change

00:50:05without executive support and the reason

00:50:07that executives change their minds is

00:50:09usually because there’s a disaster so as

00:50:13a book called by John Kotter called

00:50:16leading change and he says the most

00:50:17important thing for a organizational

00:50:20change to be successful is a sense of


00:50:22so find people

00:50:24all who are feeling the pressure those

00:50:25are normally the people who are willing

00:50:27to try out different things and if you

00:50:29can get those people on board at the

00:50:30senior level and the people in the

00:50:32middle feeding urgency and the people on

00:50:34the ground feeling urgency that’s

00:50:36normally a good place to start people

00:50:38with the level of urgency and and the

00:50:41capability to do something about it

00:50:42that’s your sweet spot for this kind of

00:50:44thing so again not an easy problem but

00:50:47those are my two ideas for helping get

00:50:50started with that stuff thank you very