00:00:11well what a great introduction I think
00:00:13that’s my my favorite introduction ever
00:00:16Thank You Marcus
00:00:17yeah I wrote this book with Barry and
00:00:20Joann because basically we wrote this
00:00:22book and it was like oh yeah that makes
00:00:24sense and then they try to do it and
00:00:26then they couldn’t because of all this
00:00:27other stuff and so I saw people and
00:00:29they’re like yeah we can’t do it because
00:00:30of this and so I got all the things that
00:00:33they said they couldn’t do it because of
00:00:34and wrote a book to explain why that was
00:00:36all and why they could actually
00:00:38do it so it’s basically it’s a book to
00:00:42arm yourself with when people say oh
00:00:44that’s a great idea but we can’t do it
00:00:46here because of X and you can say well
00:00:48actually so I don’t know if you’ll make
00:00:50many friends by doing this but you know
00:00:53that’s I can’t help with that so if you
00:00:57were here for my workshop this is the
00:00:59too long don’t read version of the
00:01:01workshop so you probably won’t learn
00:01:02anything new so please go and see
00:01:05something else instead unless you’re
00:01:07happy to sit here for 15 minutes and
00:01:09have me retell you bits of the workshop
00:01:11I won’t mind if you leave and even if
00:01:13you didn’t come to the workshop I still
00:01:15won’t mind if you leave that’s fine so
00:01:19why scaling agile doesn’t work and what
00:01:22to do about it I didn’t get straight to
00:01:24the point
00:01:25the reason scaling agile doesn’t work is
00:01:27because agile is just one piece of the
00:01:29puzzle the industry standard process for
00:01:32software development in most companies
00:01:34is still a process that I like to call
00:01:36water scrum ball and the I stole this by
00:01:44the way from Gaia Forrester called Dave
00:01:46West and the problem is this even if
00:01:48you’re doing kind of nice iterations
00:01:50here I’m going to try not to break all
00:01:54the cables will they do this yeah that’s
00:01:58better
00:01:58even if you’re doing kind of agile stuff
00:02:00here often it’s the case who works in a
00:02:03company of more than a thousand people
00:02:07ok so he
00:02:10if you work in a big company and even if
00:02:13you work in a small company you probably
00:02:15have this thing where before anyone does
00:02:17any work you spend months you know
00:02:19someone has an idea you have to go and
00:02:21write a budget proposal it has to go
00:02:23through the budgeting cycle you have to
00:02:25go and do big long estimates and work
00:02:27out how much it’s going to cost and how
00:02:28long it’s going to take and then you
00:02:30have to do the design phase and then
00:02:31eventually some enormous document lands
00:02:34on someone’s desk and then then you can
00:02:36start development and we’re doing
00:02:38development in nice little iterations
00:02:40and that’s great but you can’t actually
00:02:41release anything before you do that your
00:02:44work and the work of the other teams has
00:02:46to be integrated and then it goes to
00:02:47central testing and then hopefully you
00:02:50manage to fix some of the bugs you found
00:02:51in testing and then it gets tossed over
00:02:53the wall to IT operations and only then
00:02:55does it actually go out to users and
00:02:57this is the point where we actually
00:02:58derive measurable customer outcomes from
00:03:02the work and so this overall cycle time
00:03:06or lead time depending on how you define
00:03:08the words the time from golf course to
00:03:11measureable customer outcome is usually
00:03:14pretty long and making this bit in the
00:03:16middle a j’l usually doesn’t make a big
00:03:18difference it might make a difference of
00:03:20a few percent but I mean whenever I go
00:03:22and do value stream maps at companies
00:03:24it’s usually like oh yes this bit takes
00:03:27nine months but let’s not talk about
00:03:28that because we can’t do anything about
00:03:30that and I’m like whoa that’s the bit
00:03:33you need to fix because it doesn’t
00:03:36matter how much you mess around with
00:03:37this stuff if it’s taking you nine
00:03:38months to go through this you’re not
00:03:42making a big difference on the overall
00:03:43outcomes so who who has who works at a
00:03:47place where you have this experience or
00:03:48maybe you have a friend who has this
00:03:49experience okay so I most of you alright
00:03:54so good you’re my audience so first of
00:03:57all what tends to happen when you do
00:04:01kind of agile in this environment is you
00:04:04end up creating a lot of teams and the
00:04:07way that that is usually made to work is
00:04:09you still have to do all this stuff and
00:04:11then you take all the work and you break
00:04:14it down into lots of tiny little bits
00:04:16and then you hand the tiny little bits
00:04:19out to all the teams and they go off and
00:04:21do the tiny little bits and then at the
00:04:23end
00:04:24they come together and kind of try and
00:04:26make the system as a whole work and it
00:04:28doesn’t and you have to spend weeks or
00:04:31months actually trying to make it work
00:04:33and when it does work you find out that
00:04:34it wasn’t really what you wanted and
00:04:36you’re like oh
00:04:37but now we have to release it because
00:04:39it’s taken us a year or two years to get
00:04:42here and you know it’s on scope it’s on
00:04:44time it’s on budget so yeh so you
00:04:46release it and then maybe like me you
00:04:48had the experience where you rolled off
00:04:50the project and you enter do something
00:04:51instead and then like a year or two
00:04:54later you were talking to one of your
00:04:56colleagues he’s still on the project and
00:04:57they’re like oh yeah that thing that we
00:04:59built no one actually ended up using it
00:05:01it was cancelled or you know went to
00:05:03market wasn’t successful and that for me
00:05:05was the most miserable moment of my
00:05:07career this thing that I built that I
00:05:10spent a year on on this large team where
00:05:13we thought it was a success but then the
00:05:16thing actually never ended up making a
00:05:17lot of money for the company and after a
00:05:19couple of years it got canceled so agile
00:05:24doesn’t fix that problem because
00:05:28fundamentally the problem is the ideas
00:05:30that you had at the beginning wasn’t the
00:05:32right idea but it took you so long to
00:05:33find out that it wasn’t the right idea
00:05:35there you were you know you were screwed
00:05:39and then you look at the decision making
00:05:42process so I love running surveys
00:05:44surveys are a really rich source of
00:05:46confirmation bias and I ran a survey
00:05:50with Forrester to find out about agile
00:05:52practices one of the questions we asked
00:05:55is how the companies make investment
00:05:57decisions for products so 47% of people
00:06:02replied that the committee decides from
00:06:04potential options so decision by
00:06:06committee so any four percent of people
00:06:09said they use some kind of financial
00:06:11modeling they use economics to make
00:06:13their investment decisions whoo that
00:06:16seems like a good idea for a joke
00:06:20we put opinion of the person with the
00:06:22highest salary wins out and you can see
00:06:24I spelled salary wrong here so sorry
00:06:25about that but 13% of people answered
00:06:30that that’s how they made decisions this
00:06:32is called the hippo method the highest
00:06:34paid person’s opinion
00:06:36and that really is the same thing as
00:06:40committee decides from central options
00:06:41so that and that are actually the same
00:06:43thing and then nine percent of people
00:06:45said they use a product portfolio
00:06:46approach which is also the same thing as
00:06:49this and then some sentient people you
00:06:53know which I thought was very admirable
00:06:54said that they take no systematic
00:06:56approach which is basically also the
00:06:59same thing so what we can see here is
00:07:02that 76% of people are not using an
00:07:06economic framework and 24% of people are
00:07:08using an economic framework which is
00:07:10kind of depressing
00:07:11so that’s the state of software
00:07:14development and portfolio management
00:07:16investment today and in most big
00:07:20companies what you find is the output of
00:07:22this process is a few very big projects
00:07:24because somehow the annual budgeting
00:07:26cycle leads to a few big projects being
00:07:30the ones that get funded and so who has
00:07:33like a retirement savings account
00:07:36alright so I’ve got a product that I
00:07:38want to sell you if your retirement
00:07:40savings account and in my products I
00:07:42have four big things that I’m building
00:07:45and it’s going to take me a really long
00:07:48time to build them and at the end we
00:07:50don’t know if they’re going to be
00:07:51successful or not and it’s going to take
00:07:52us a really long time to find out but if
00:07:54they are successful they might make lots
00:07:56of money would you like to invest in my
00:07:58product no because you’re not stupid and
00:08:03yet that’s exactly what we do with the
00:08:06IT budgeting process for building
00:08:08projects in most cases we’d be better
00:08:11off taking our money and pressing it
00:08:13into unit trusts that would provide a
00:08:16better return on investment than our IT
00:08:18budget investments and the things that
00:08:21we do to mitigate the risk of being
00:08:22wrong don’t work so the main activity
00:08:26what’s the main activity we perform to
00:08:28make sure that we should really do the
00:08:29projects the main analysis activity huh
00:08:35business case and what’s the main kind
00:08:38of analysis activity we use to create
00:08:40the business case so return on
00:08:43investment is the main thing we want to
00:08:45know about but what do we spend our
00:08:47analysis time doing to find out if we’ll
00:08:50get rich
00:08:50investment I would love to to know that
00:08:56people actually put effort into finding
00:08:58out the potential business value that’s
00:09:00not where I see most people putting
00:09:02their effort in analysis what I mainly
00:09:05see the effort being is estimation so
00:09:08estimation of cost that’s where I mostly
00:09:10see people putting in a lot of time and
00:09:12effort is working with the developers to
00:09:14estimate the amount of work they’re
00:09:15going to do the problem with that is
00:09:17it’s pointless so there’s a guy called
00:09:20Douglas Hubbard who wrote a book called
00:09:22how to measure anything and he’s been
00:09:23looking at business cases for many many
00:09:25years and he does a kind of analysis
00:09:29called Monte Carlo in Monte Carlo you
00:09:31change the inputs to the business case
00:09:33and you see the impact of changing the
00:09:37values on what you care about which is
00:09:39normally either return on investment or
00:09:42lifecycle profits for the products those
00:09:44are the two variables you care about and
00:09:45what he found is that the cost has a
00:09:50very low information value varying the
00:09:52cost actually doesn’t have a huge impact
00:09:54on return on investment or lifecycle
00:09:57profits which is what you really care
00:09:58about the single most important unknown
00:10:02he found is whether the project will be
00:10:04cancelled the next most important
00:10:06variable is the utilization of the
00:10:08system including how quickly the system
00:10:11rolls out and whether some people will
00:10:13use it at all and those are the two
00:10:15things that you care about is it going
00:10:16to get cancelled and when it rolls out
00:10:19are people going to use it and that’s
00:10:21true for internal IT projects and for
00:10:24external light you know product
00:10:26development is are people really going
00:10:28to use it and that’s the thing we don’t
00:10:29normally spend a lot of time finding out
00:10:31and that’s for me the whole point of the
00:10:35Lean Startup is how we can cheaply run
00:10:37experiments to find out if people will
00:10:40actually use it and pay you money for it
00:10:41and that is way way way more important
00:10:44than any kind of estimation activity and
00:10:46yet it’s much more rare to find people
00:10:49doing that than it is to find them
00:10:50spending you know weeks or months doing
00:10:52in a very detailed estimation and then
00:10:55once you’ve done the estimation you take
00:10:57all the little bits you basically break
00:10:59it down a little bit estimated little
00:11:00bits add it all up and then you give the
00:11:02people all the little bits to do
00:11:03we’re back where we started so the other
00:11:07problem with this approach is that we
00:11:10create these huge projects and when we
00:11:13create huge projects we batch up all
00:11:15these little bits of work together in
00:11:17one big batch which we then push through
00:11:19the process and part of the reason we do
00:11:21that is because the transaction cost of
00:11:24taking work through the entire value
00:11:26stream is so high and so part of the
00:11:28point of continuous delivery is to make
00:11:31it economic to work in small batches so
00:11:33we can get the feedback much more
00:11:34quickly so that’s kind of a quick
00:11:37preview of the continuous delivery thing
00:11:40but when we batch up work what we do is
00:11:43we batch up that very high value things
00:11:45with lots of very low value things and
00:11:47you push them all out together there’s a
00:11:50really good case study called black swan
00:11:53farming using cost of delay this is a
00:11:56case study that was done on masks which
00:11:58was the world’s biggest or second
00:12:00biggest shipping line so a really big
00:12:01company with lots of IT projects and
00:12:04they had about 3,000 different features
00:12:06that were going through the value stream
00:12:09from analysis through development and
00:12:11delivery and one of the first things
00:12:13they did was try and estimate the value
00:12:15of these pieces of work and they did
00:12:18this using a method called cost of delay
00:12:20so if you go to this link here you can
00:12:23download the paper I really highly
00:12:25recommend this paper as a case study of
00:12:27how to implement lean agile at scale
00:12:31what they found that was really
00:12:33interesting is by using cost of delay to
00:12:35measure the value of the features so
00:12:38what cost ablai is briefly is how much
00:12:41it costs you per unit time per week in
00:12:45this case to not deliver the feature so
00:12:48every week that we’re not delivering
00:12:49this feature how much is it costing us
00:12:51in opportunity cost to not deliver it
00:12:53and that’s the way that you can
00:12:54prioritize work in dollar value by
00:12:57considering the opportunity cost I’m not
00:12:59delivering it and comparing it and what
00:13:00they found is of these thousands of
00:13:02requirements that were going through the
00:13:03system there were about three that it
00:13:06was costing the company you know between
00:13:08two and 2.8 million dollars per week to
00:13:11not deliver and then you’ve got this
00:13:13long tail of features where the cost of
00:13:16delay is you know
00:13:17in comparison with this and so it
00:13:20becomes very clear like what should we
00:13:23work on you’ve got this big IT team they
00:13:27should stop doing everything except
00:13:29these three bits of work and deliver
00:13:30these three bits of work as fast as they
00:13:32possibly can and stop doing anything
00:13:35else and when you’re not even exposing
00:13:38the value of what you’re doing and you
00:13:39know they were making assumptions to
00:13:41come to these numbers it can be quite
00:13:42hard to work out the dollar value of
00:13:44these features but when the variability
00:13:47is quite high it doesn’t even matter
00:13:50when there’s like an order of magnitude
00:13:52difference between the value doesn’t
00:13:53matter if I’m wrong by you know a factor
00:13:56of two who cares
00:13:58still going to be really obvious which
00:13:59which the features are and Joshua
00:14:03Arnold’s done a bunch of other projects
00:14:05where he’s analyzed things in this way
00:14:06and he’s found that this this shape this
00:14:08is a power-law curve this kind of power
00:14:11law distribution is very very common
00:14:13most places where they’ve gone and done
00:14:15looked at the the value of the features
00:14:17it’s been this power law curve so this
00:14:19is pretty typical of what you’ll find is
00:14:20that a few small very valuable things
00:14:23get bashed up with a ton of other stuff
00:14:25that’s low value and then when you have
00:14:27these big batches of work going through
00:14:28the system is really hard to prioritize
00:14:30the individual bits so what do we do
00:14:34first of all the whole project paradigm
00:14:38is based on the assumption that the
00:14:40features and requirements that we’ve
00:14:43created are in fact the right thing to
00:14:45do almost always as we’ll see that’s not
00:14:49true they are the wrong thing to do
00:14:51about two-thirds of the time the
00:14:53features that we want to build have zero
00:14:57or negative value and are not the right
00:14:59thing to do I have data to back this
00:15:01statement up which I will give you the
00:15:03reference to in the case of new product
00:15:06development the numbers are much much
00:15:08worse it’s more like 90% of the time the
00:15:11products the wrong idea so for
00:15:13established products with well known
00:15:15business models that work 2/3 of the
00:15:17time the features are 0 negative value
00:15:19with new products where there’s a larger
00:15:22amount of uncertainty over 90% of the
00:15:24time it’s the wrong idea so he shouldn’t
00:15:25optimize for the case way we think we’re
00:15:28right about what we’re building which is
00:15:30what the project
00:15:30paradigm’ does instead of focusing on
00:15:33cost and spending a lot of work on
00:15:34estimation we should focus on value and
00:15:37gathering information to justify the
00:15:40value of what we’re building one of the
00:15:43most important things we can be doing is
00:15:44creating feedback loops throughout our
00:15:46delivery process and making sure that
00:15:49the decisions we make whether that’s the
00:15:52products development decisions the
00:15:53investment decisions the requirement
00:15:55decisions are actually the right thing
00:15:56how can we get the feedback on those
00:15:58decisions as fast as possible and then
00:16:01you know within the development process
00:16:03we want feedback loops did I break the
00:16:06system did I introduce the performance
00:16:08degradation did I introduce a security
00:16:10hole this is part of the point of
00:16:11continuous delivery in the deployment
00:16:12pipeline is to create really effective
00:16:15rapid feedback loops within our delivery
00:16:17process the point of continuous delivery
00:16:20is to make it economic to work in small
00:16:22batches and working in small batches is
00:16:24what us what allows us to get these
00:16:28feedback loops working so that we can
00:16:30get those feedback loops operating at a
00:16:32high frequency by working in small
00:16:34batches we get feedback at high
00:16:35frequency and that allows us to
00:16:38course-correct much more quickly which
00:16:41is the whole point of agile and when
00:16:44we’ve done these things then we can
00:16:45enable an experimental approach to
00:16:47product development based on the
00:16:49scientific method I have a hypothesis
00:16:51about a feature that I want to build how
00:16:53do I test whether my hypothesis is
00:16:55correct what data can I gather to
00:16:57validate my hypothesis and we can apply
00:16:59this experimental approach not just to
00:17:01product development but also to process
00:17:03improvement and continually work to
00:17:06improve the quality of our development
00:17:08and delivery processes so the first idea
00:17:11that I want to attack is this idea of
00:17:14requirements whose requirements are they
00:17:18are they the users requirements users
00:17:24don’t know what they want users know
00:17:27what they don’t want once you’ve built
00:17:28it for them they’re the requirements of
00:17:32the hippo the highly paid person and so
00:17:38I don’t like this word requirements what
00:17:39we have is hypotheses
00:17:42and typically there’s more than one way
00:17:45to achieve the outcome we want to
00:17:46achieve and a lot of the time we’re not
00:17:48even thinking about the outcomes so
00:17:50who’s who’s who uses stories in their
00:17:53projects as a I want them so that yay
00:17:56okay lots of you okay keep your hands up
00:17:59if you’ve written a story where you kind
00:18:03of left out the so that because it was
00:18:05kind of a bit hard to think about it
00:18:06keep your hand up I have yeah most of us
00:18:11have done that at one time or the other
00:18:13and it’s actually pretty common who’s
00:18:15worked on a project where like the so
00:18:17that is missed out most of the time like
00:18:19more than 50% of the time yeah so you
00:18:22know that’s life but I think that’s I
00:18:29mean we all know that that’s the most
00:18:30important bit but in the day to day kind
00:18:32of work process it’s kind of easy to
00:18:34forget that because you know that it’s
00:18:35important and we’ve got to do it so
00:18:37let’s just do it I think we should focus
00:18:38much more on the outcomes so one of my
00:18:40favorite tools is this one by glyco ads
00:18:43they’re called impact mapping it’s a
00:18:45very simple idea but it’s very powerful
00:18:46and his idea is this we should work
00:18:49backwards from the outcome so this is
00:18:52the outcome we want to create this is
00:18:53just an example right reduce transaction
00:18:56costs by 10% for a trading platform so
00:18:58this is a impact map for a trading
00:19:00platform reduce transaction costs by 10
00:19:02percent who are the stakeholders well
00:19:04there’s a German settlement team there’s
00:19:06I didn’t plan by the way that I would
00:19:08deliver this in Berlin just happen that
00:19:10way there’s some traders there’s IT
00:19:12operations here at the second level are
00:19:15the ways in which these stakeholders
00:19:18could help or hinder this outcome and
00:19:21then at the next level are the actual
00:19:23stories these this is the what what you
00:19:26could do to to achieve that outcome and
00:19:29when we do analysis what happens is you
00:19:33pick one and then you throw everything
00:19:35else away and then that one thing is
00:19:38what goes downstream to development and
00:19:40then tidy operations and that’s a real
00:19:43problem a lot at the time the developers
00:19:45never even see the rest of this map they
00:19:47just go improve exception reports so
00:19:50that you know trading costs go down and
00:19:53often the number which is really
00:19:56like how are we going to measure the
00:19:58success of whether we’re done is thrown
00:20:01away so actually having the key thing
00:20:04about the impact map is not creating the
00:20:06impact map the key thing about the
00:20:07impact map as with most agile practices
00:20:09is the shared understanding of the whole
00:20:12team which is created when you build the
00:20:15impact map so we have this thing in
00:20:17agile that’s everywhere where you focus
00:20:19on the artifacts the artifacts not the
00:20:22important thing the shared understanding
00:20:23of the team that created the artifact is
00:20:25actually the important thing so here
00:20:27what’s crucial is having the developers
00:20:29and the operations people and the
00:20:31business people all work together to
00:20:33create the impact map and then pick one
00:20:36like which is the thing that we think
00:20:38will be the smallest possible work for
00:20:40the biggest outcome because as Jeff
00:20:42Patton likes to say what we want to do
00:20:44is minimize output maximize outcomes so
00:20:47we want to maximize the outcome what’s
00:20:49the minimum amount of work we can do to
00:20:51achieve this outcome and you pick one
00:20:52and then you design an experiment to
00:20:54test whether it will in fact achieve
00:20:56this outcome so the other thing I really
00:20:58like that fits with this really nicely
00:21:00is Jeff golf’s template for hypothesis
00:21:04driven delivery so instead of the story
00:21:06format he has this format we believe
00:21:08that building this feature for these
00:21:10people will achieve this outcome wellnot
00:21:13is successful when we see this signal
00:21:16from the market what’s the signal we’re
00:21:18going to measure to prove that the thing
00:21:19we’re building will actually achieve the
00:21:21outcome so we believe that building this
00:21:23feature building this feature for these
00:21:26people for these people will achieve
00:21:29this outcome this outcome wellnot is
00:21:32successful when we see this signal from
00:21:33the market what are we going to measure
00:21:34to demonstrate the feature will actually
00:21:36deliver and there’s many ways we can
00:21:39measure the outcome using an experiment
00:21:44most of this comes from the UX movement
00:21:47and one of the big trends in agile is
00:21:50integrating UX but often what I see
00:21:54maybe you have friends you have this
00:21:55problem is the UX is basically making it
00:21:57look pretty that’s not the point of view
00:22:00X the point of view X is to help us
00:22:04think about delivering things in a way
00:22:07that will satisfy our customers
00:22:09so the whole way in which our customers
00:22:12interact with our products and our
00:22:14company is all within the realm of UX
00:22:16including will the feet features will
00:22:19the products actually make our customers
00:22:21happy and our users happy so this is
00:22:25from Janice Fraser and she divides up
00:22:28user research based on whether it’s
00:22:30quantitative or qualitative and then
00:22:31whether it’s evaluative or generative so
00:22:34generative in design thinking there’s
00:22:37two phases there’s the bit where you
00:22:38generate ideas which is where you can
00:22:41have come up with lots of different
00:22:42options so building an impact map is an
00:22:44example of a generative behavior and
00:22:47then evaluative is when you narrow down
00:22:50the options and pick one so that’s a
00:22:52conversion activity that’s the process
00:22:54of deciding which option is the right
00:22:55one so you can use user research to
00:22:58create the whole bunch of ideas and then
00:23:00you can use different techniques to find
00:23:01out which of those ideas is actually
00:23:03going to produce the required outcome
00:23:04but there’s loads of ways to do this
00:23:06that don’t involve actually building out
00:23:09the whole feature and so you should be
00:23:11thinking very carefully about how we can
00:23:13gather data without building out the
00:23:15whole feature and then finding out
00:23:17whether it was a good idea because
00:23:18usually what happens is even if you find
00:23:20out it wasn’t a good idea
00:23:21there’s the sunk cost fallacy you don’t
00:23:24want to admit you spent all this time
00:23:25building this thing that’s useless who’s
00:23:28actually deleted a big feature from
00:23:30their products in the last year okay
00:23:34that’s pretty good actually I’m pretty
00:23:35impressed did you delete that because
00:23:37you were like this isn’t delivering us
00:23:39valley cool well good for you
00:23:43that should be more common than it is so
00:23:47in terms of an alternative to water
00:23:49scrum fall I like to show this slide
00:23:51from Amazon this is from 2011 they’re
00:23:54about an order of magnitude better right
00:23:55now they are making changes to
00:23:57production on average every 11.6 seconds
00:23:59up to a thousand and 79 deployments per
00:24:02hour up to 10,000 boxes receiving
00:24:04deployment up to 30,000 on average
00:24:0810,000 up to 30 thousand boxes receiving
00:24:10deployment he was certainly calls talk
00:24:12last night okay lots of II so I went
00:24:16spend too much time on this but the
00:24:18reason one of the reasons they did this
00:24:19was to get this which again Nicole
00:24:22showed
00:24:23so ron iike harvey helped to build
00:24:26amazon’s experimentation platform he was
00:24:28in charge of amazon’s experimentation
00:24:30platform then he went on to work for
00:24:32microsoft and was in charge of the Bing
00:24:34still in charge of being experimentation
00:24:37platform so he has loads of data from a
00:24:40feature development and this is I mean
00:24:44this is the important piece of data we
00:24:46could be spending two-thirds of our time
00:24:49on the beach skiing at home with our
00:24:56kids if only we knew the two-thirds of
00:24:59the features that we builds that have
00:25:00zero or negative value to our company
00:25:03and people don’t think of the negative
00:25:05value case the case where you’re
00:25:07actually making things worse and that
00:25:10kills us in three ways firstly there’s
00:25:13the opportunity cost of not building
00:25:15something the would’ve delivered value
00:25:18second features add complexity to our
00:25:22system they have to be maintained
00:25:23forever so the maintenance cost and
00:25:26third that complexity slows down the
00:25:29rate at which we can add new features so
00:25:31it makes new feature development slower
00:25:33so if you’re not doing experiments to
00:25:37find out if your features are actually
00:25:39valuable you’re killing yourselves in
00:25:42three in these three ways is the biggest
00:25:44source of waste in software delivery is
00:25:45the stuff that we build that delivers
00:25:47zero or negative Valley and so who is
00:25:50actually running experiments to test the
00:25:51value of the features and products
00:25:53they’re building okay there’s a few of
00:25:56you maybe 10 hands went up out the whole
00:25:59audience like this is the biggest thing
00:26:02in my opinion that we need to change our
00:26:04industry is be thinking about how we can
00:26:06actually gather data to validate whether
00:26:08the things we’re doing are actually
00:26:09delivering value for our customers and
00:26:11it doesn’t need to be a be testing it
00:26:13could just be other forms of user
00:26:14research the cool thing for me about
00:26:23software is that software has different
00:26:28characteristics from architecture in to
00:26:31in in terms of buildings right so people
00:26:35often say I wish software could
00:26:37more like buildings meaning that they
00:26:39wish it could not fall down and be
00:26:41reliable and be secure and things like
00:26:42that but actually I think that’s that’s
00:26:45false
00:26:45we don’t want software to be like
00:26:47buildings because software has these
00:26:49really unique affordances software is
00:26:51easy to change even at scale compared to
00:26:54buildings if I want to really check this
00:26:56building I have to pull it down and
00:26:58start again from scratch you don’t have
00:27:00to do that with software helps of people
00:27:02do do that with software which is
00:27:04unfortunate but you know you can
00:27:06actually rebuild the plane while it’s in
00:27:08flight with software you can make
00:27:09architectural changes incrementally it’s
00:27:12entirely possible to do that people have
00:27:14done that very successfully software is
00:27:15much cheaper to change you can get value
00:27:18from software before you finish building
00:27:20it like I you couldn’t be in this
00:27:22building if there were no ceilings here
00:27:24but I can develop a piece of software
00:27:26which doesn’t serve all of your needs
00:27:29but might serve the needs of a small
00:27:30segment of you and I can and you might
00:27:32be able to get value from that from
00:27:33really early on so software has these
00:27:35unique affordances that makes it much
00:27:37easier to experiment and try things out
00:27:39and we can’t do that with buildings and
00:27:41with physical architecture and what’s
00:27:45cool about that is we can use the same
00:27:47techniques for experimental product
00:27:51development that we that the lean
00:27:53movement has been using for years for
00:27:56process improvement so to show you what
00:27:59I’m talking about I’m going to give you
00:28:01a case study and the case study is from
00:28:03HP LaserJet firmware division I’ve
00:28:05talked about this before maybe you’ve
00:28:07seen me but I’m going to focus on how
00:28:11they made the transition so this was a
00:28:13team that in 2008 had this really big
00:28:18problem which is that they were on the
00:28:20critical path for software development
00:28:21every new range of printers they wanted
00:28:23to launch they were going to have to
00:28:25build new firmware and that was going to
00:28:28take longer than actually building the
00:28:30chips that went into those printers so
00:28:32that was a really serious problem I
00:28:34actually did some work for a European
00:28:37airline and they were introducing you
00:28:40know those the economy seats with bigger
00:28:42gaps between the seats right like
00:28:45Premium Economy more legroom it was
00:28:47going to take them longer to change the
00:28:49booking system
00:28:50so that you could book the seats the net
00:28:53was going to take them to put the seats
00:28:55into the aeroplanes right which is how
00:28:58you know there’s something really badly
00:29:00wrong with your software delivery
00:29:01process so there’s the same thing here
00:29:04it’s going to take them longer to build
00:29:06the firmware than it was going to take
00:29:07them to fabricate these custom chips
00:29:09which had a one-year lead time to be
00:29:11able to build so they wanted to they
00:29:13tried a whole bunch of different things
00:29:14and eventually they decided to look at
00:29:16their software delivery process what
00:29:18they found is they were spending a whole
00:29:20bunch of time doing non value add things
00:29:22like code integration very detailed
00:29:24planning porting code between branches
00:29:27every time they released a new line of
00:29:28devices they took a branch in version
00:29:31control which meant that when they fixed
00:29:32a bug in one line of devices and they
00:29:34needed support the bug fix they would
00:29:36have to port it across all these code
00:29:38branches same with features they were
00:29:40spending 25% of their time porting
00:29:42features and bug fixes across branches
00:29:4425% of their time on product support
00:29:47what is this tele quality problem 15
00:29:52percent of their time on manual testing
00:29:54if you subtract this from a hundred
00:29:55percent what you find is that only five
00:29:57percent of their investment was actually
00:30:00being spent on building features and by
00:30:03the way all the product management
00:30:04people you know they would say can we
00:30:06get budget for you know adding test
00:30:08automation can we spend some time
00:30:10refactoring and management be like no no
00:30:13you’ve got to build the features who’s
00:30:15had that happen to them right and it’s
00:30:18very clear to see the fallacy in that in
00:30:22that idea because here’s the thing the
00:30:25reason that there’s all this pressure to
00:30:27build the features the reason you’re
00:30:29going so slowly is because you’re
00:30:30spending all this amount of money on
00:30:32things that are not adding value the
00:30:35only way you can get that problem is by
00:30:37fixing this by removing this that will
00:30:39allow you to go faster so you have this
00:30:42vicious cycle you know pressure you have
00:30:45to fix you have to do the features why
00:30:47is the pressure there because we’re
00:30:48doing all this other stuff it’s so
00:30:49painful to deliver anything and if we
00:30:51don’t fix this it’s always going to get
00:30:52worse and worse and worse until we’re
00:30:54just driven into the ground they also
00:30:57looked at their cycle times is taking a
00:30:59week to get changes into trunk they were
00:31:01getting one or two builds a day and it
00:31:03six
00:31:04six weeks to do a full manual regression
00:31:05on their software before they could
00:31:07release it so they didn’t want to do
00:31:10continuous deployment they didn’t want
00:31:12to release new firmware ten times a day
00:31:14I mean who upgrate who’s upgraded their
00:31:16printer firmware okay as a few yak
00:31:21shavers in the audience that’s good most
00:31:24the time you don’t want to upgrade your
00:31:26firmware ten times a day but what they
00:31:29found is by implementing continuous
00:31:31delivery by working in small batches it
00:31:35changed the economics of the software
00:31:37delivery process because it created
00:31:40feedback loops that enable them to build
00:31:43quality in rather than try and test
00:31:45quality and at the end and over the
00:31:48course of two years and well first of
00:31:50all they re architected in a big bang’
00:31:52way which is always very risky that I
00:31:54don’t recommend it but in this case it
00:31:55works and I’ll explain why later but
00:31:58they really tected such that they didn’t
00:32:02have branches for different ranges of
00:32:05devices anymore instead they had one
00:32:07firmware build that would work on any
00:32:09set of devices so the firmware boots
00:32:11looks at the hardware profile says oh
00:32:14I’m a printer I’m going to turn these
00:32:17features off I’m going to keep these
00:32:18features on or boots looks around says
00:32:20oh I’m a scanner so I’m going to turn
00:32:22these features off and turn these
00:32:23features on so it’s basically feature
00:32:24toggles for architectural differences so
00:32:29a guy I know called danbo dot has a
00:32:31saying feature branching is a poor
00:32:33person’s modular architecture so this is
00:32:38a case where they re architected so they
00:32:39wouldn’t have to use branches and that
00:32:42allowed them to work on trunk and do
00:32:44continuous integration and build a very
00:32:46sophisticated test automation suite
00:32:48where they had over 30,000 hours of
00:32:51tests that would actually send signals
00:32:52to the logic boards and then get signals
00:32:55back as part of their test automation so
00:32:57anyone who complains about test
00:32:59automation for websites I actually I
00:33:02don’t have it today but I often carry
00:33:04around a copy of this book to spank
00:33:06people who whine about test automation
00:33:08because now here’s people who did it
00:33:10with logic boards that’s super cool so
00:33:14they built a very sophisticated set of
00:33:15automated tests anytime anyone
00:33:18act in they used get so every developer
00:33:20so 400 developers by the way so at scale
00:33:23400 developers on three different
00:33:25countries Brazil USA India or pushing
00:33:30changes in to get they each had their
00:33:33own little repo and they built a CI tool
00:33:35that looked at those repos anytime so
00:33:39and pushed at range it would take that
00:33:40change and run two hours worth of
00:33:42automated tests against that change in a
00:33:44simulator if that works
00:33:46it gets promoted to the stage two and
00:33:50then what happens in stage two is all
00:33:52the changes in the last time period get
00:33:55merged and then they run automated tests
00:33:57in the simulator against that merged set
00:33:59of changes if that fails the developers
00:34:01get an email here the test that broke
00:34:03here’s a button to be able to reproduce
00:34:06those test failures on your development
00:34:07workstation that’s super important if
00:34:10developers can’t reproduce the
00:34:11acceptance test failures that’s a very
00:34:13serious problem it’s an architectural
00:34:15problem that you need to fix if they
00:34:19succeed
00:34:20that’s the only way you can get into
00:34:21trunk so to get into trunk anything that
00:34:25passes stage 2 gets into trunk and
00:34:27that’s the only way the developers can
00:34:29get into trunk so this basically fixed
00:34:33the problem where the bills always read
00:34:34because the developers don’t care about
00:34:36it because guess what the only way you
00:34:38get into trunk is if the the bill is
00:34:39green out of level 2 this runs on a big
00:34:45rack of servers in a simulator another 2
00:34:48hours worth of automated tests then you
00:34:50get promoted to level 3 which runs
00:34:51actually on physical logic boards with
00:34:54emulators and then if that works then it
00:34:57gets promoted to the overnight level 4
00:34:59and that’s the entire 30,000 hours worth
00:35:01of test run in parallel so you get the
00:35:03results overnight and so you can find
00:35:06out if the firmware is releasable within
00:35:0824 hours of making the check-in so they
00:35:11completely removed the six-week
00:35:13stabilization phase that just goes away
00:35:15and the whole point of continuous
00:35:18delivery by the way is to get rid of
00:35:22this whole thing like this whole thing
00:35:24should all go away even if you’re not
00:35:27releasing 10 times a day you should be
00:35:30able to release yourself
00:35:31I should always be provably reducible on
00:35:34a daily basis that’s the point of
00:35:37continuous delivery and so even if
00:35:39you’re not doing continuous deployment
00:35:40that produces huge benefits they
00:35:43massively reduce the amount of time
00:35:44they’re spending on code integration and
00:35:46on planning and on porting K between
00:35:48branches product support activities goes
00:35:51down from 25% of costs to 10% of course
00:35:53what does that tell you higher quality
00:35:58so this is how they measured the quality
00:36:00improvement manual testing goes down for
00:36:0315 percent to 5 percent costs invested
00:36:07in innovation go from 5 percent to 40
00:36:10percent so 8x improvement in
00:36:12productivity measured in terms of the
00:36:14amount of investment going towards
00:36:16innovation what do the arithmetic in the
00:36:19audience notice about these numbers less
00:36:23than 100% because there is a new
00:36:26activity creating maintaining evolving
00:36:31the Suites of automated tests if you
00:36:34enter your manager right now and said
00:36:35please can we invest 23 percent of our
00:36:38budget in test automation
00:36:39what would your manager say that’s a
00:36:45rhetorical question and yet here we have
00:36:48the business case for doing that the
00:36:50business case is quite clear despite the
00:36:52fact you’ve got 23 percent of your
00:36:53budget spent on test automation that has
00:36:55enabled an 8x improvement in cost spent
00:36:59on innovation because it’s massively
00:37:01reduce the amount of non value I’d work
00:37:02we’re doing and this is how lien works
00:37:04lien works by investing in removing
00:37:07waste so that you can increase
00:37:11throughput and the reason I like to talk
00:37:17about this case study is because they
00:37:18wrote it into a book and you can get the
00:37:19book and here are the numbers to take to
00:37:23your CFO to explain why they should
00:37:26investing its newest delivery what’s
00:37:31interesting what I’m going to spend the
00:37:32last few minutes on out of 10 minutes
00:37:35right yep is how they did it because how
00:37:39they did it was really interesting they
00:37:42did not create a three year plan
00:37:45detail of how they were going to do this
00:37:48what they did is they had a goal their
00:37:52goal was to fold one get firmware off
00:37:56the critical path goal to 10x
00:38:00productivity increase so those that was
00:38:03it those that’s all they wanted to do
00:38:05they had no idea how they would do it
00:38:06but they set these two very measurable
00:38:10easy-to-understand goals like everyone
00:38:12in this room can understand those goals
00:38:14it’s really obvious and it’s really
00:38:16obvious when you’ve achieved them and
00:38:17when you haven’t achieved them that’s
00:38:19super important anytime you’re
00:38:20implementing agile or DevOps or any
00:38:23methodology step zero is to understand
00:38:26your goals in measurable terms what goal
00:38:29are we trying to achieve how will we
00:38:32know when we’ve achieved it there’s
00:38:34acceptance criteria for organizational
00:38:36transformation what’s our goal if you if
00:38:40you implement agile because agile is
00:38:41cool or because it will make things
00:38:43better you will fail you’ve got to
00:38:46understand what the goals you’re trying
00:38:48to achieve are for your organization in
00:38:49measurable terms and then we want to
00:38:52take an experimental approach to
00:38:53achieving those goals so what some
00:38:56intermediate goal which will get us some
00:38:58of the way there let’s try a bunch of
00:39:00things out to find out if we can achieve
00:39:01that intermediate goal if we don’t
00:39:03succeed we’ve learned something but we
00:39:05haven’t spent too much time learning it
00:39:07which is important and if we succeed
00:39:10that’s great let’s try the next step so
00:39:12this is crucial we’re taking an
00:39:15incremental approach to process
00:39:17improvement and to changing
00:39:19organizational behavior in the same way
00:39:22that we take an incremental approach to
00:39:24product development and that for me was
00:39:26like the big aha moment I had writing
00:39:28the book is well guess what the
00:39:30incremental iterative approach doesn’t
00:39:31just apply to how we do product
00:39:32development it also applies to how we do
00:39:34process improvement and how we improve
00:39:36our companies and our processes and it’s
00:39:38something we should be doing habitually
00:39:40as part of our daily work and what was
00:39:43cool is that this was a case study for
00:39:45something that only got written about
00:39:47subsequently which is true I mean they
00:39:49were doing it in its livery
00:39:50before I wrote the continuous delivery
00:39:51bug and they didn’t call it continuous
00:39:53delivery they called it getting better
00:39:55at engineering right didn’t need a name
00:39:58is he was
00:39:59like oh yeah we’re just trying to get
00:40:00better at what we’re doing and and in
00:40:03the same way they also simultaneously
00:40:05invented this thing the improvement
00:40:07cutter so a cutter comes from Japanese
00:40:10martial arts it’s a habit
00:40:12it’s sorry it’s a base practice that you
00:40:14practice over and over again until it
00:40:16becomes habitual and it’s the same idea
00:40:18whereas music you practice scales over
00:40:21and over again that’s the first thing
00:40:22you do until you get really good at it
00:40:23and then you move to higher order
00:40:25practices that combine the things you
00:40:27learn
00:40:27same thing with sports you learn the
00:40:29basic moves for tennis and then you’re
00:40:31able to combine those any human creative
00:40:33activities the same thing you learn the
00:40:35basic moves first and then you until
00:40:37they become habitual and then you
00:40:39combine them into higher-order creative
00:40:41processes so his point is with this book
00:40:45improvement cutter based on the study of
00:40:47Toyotas management method is that
00:40:49improvement work getting better or what
00:40:52we do should be a habitual process and
00:40:54here’s the basic practice that you have
00:40:56to practice over and over again in order
00:40:59to get better first understand the
00:41:01directional challenge so HP LaserJet 10x
00:41:04productivity increase get firmer off the
00:41:06critical path number to grasp the
00:41:08current condition current condition step
00:41:173
00:41:18establish the next target condition so
00:41:21I’m going to show you what a target
00:41:22condition looks like but it’s basically
00:41:24an intermediate measurable set of goals
00:41:27for the program as a whole
00:41:29and then you don’t plan how you’re going
00:41:31to do it instead you specify the
00:41:34outcomes and then you allow the people
00:41:36to experiment with ideas to achieve
00:41:38those outcomes in the same way as we do
00:41:41with product development and then every
00:41:43day everyone’s asking themselves what’s
00:41:45the challenge we’re trying to achieve
00:41:46right now what’s the actual condition
00:41:49what obstacles are in our way what are
00:41:52we going to try next when can we see go
00:41:55and see what we learned from taking that
00:41:56step and so the difficult bit of this is
00:41:59working out the targa condition so I’m
00:42:02going to give you an example from HP
00:42:04LaserJet firmware this is 30 months in
00:42:08but these were their entire goals for
00:42:11the whole program of work for
00:42:13100 people on three continents this was
00:42:16their plan for the month it fits on one
00:42:19piece of paper and this is quite long
00:42:22when they started their monthly plans
00:42:24for the program of work were three or
00:42:27four bullet points but crucially they
00:42:30had measurable out outcomes so for month
00:42:3530 it was Priority One issues open less
00:42:38than one week level two test failures 24
00:42:40hour response final priority one change
00:42:43request fixed reliability error rate at
00:42:45release criteria and these are ordered
00:42:46by priority so you you want to get these
00:42:50ones done before you start on these ones
00:42:52these ones are the most important so
00:42:54these are all measurable and here’s the
00:42:57thing because this is the only thing
00:43:00that’s on the program plan and it’s just
00:43:02the outcomes it’s up to the teams to
00:43:04experiment with ideas to achieve those
00:43:06things in most of the scaled agile
00:43:08frameworks where you break up the work
00:43:10into little bits you hand it out to all
00:43:12the people and then you come together at
00:43:14the end and it doesn’t work properly
00:43:15everyone says well I did my bit it was
00:43:18those guys now usually that’s not true
00:43:23usually actually everyone did do that
00:43:25bit the problem wasn’t that people
00:43:27didn’t do their bit the problem was that
00:43:29the bits didn’t fit together or weren’t
00:43:32the right things to do it was an
00:43:33analysis problem in this model nobody
00:43:37succeeds unless everybody succeeds
00:43:39there’s no I did my bit either we all
00:43:43succeed as a team or we fail as a team
00:43:45and because it’s only one month the
00:43:49impact of a failure isn’t too bad the
00:43:51point of failure is to expose problems
00:43:53so we can fix them not so that we can
00:43:57blame people for being done well no
00:43:58guess what the reason we can do it why
00:44:00can we do is because of this Oh our
00:44:01assumption about this was wrong so let’s
00:44:03make different objectives for next month
00:44:06that address the underlying problems
00:44:09so I guess my conclusion for this is we
00:44:17want to take an experimental approach
00:44:18not just to product development but also
00:44:20to process improvement the processes
00:44:23that you have are the wrong processes
00:44:26and you’ll know
00:44:27I have the right ones but the important
00:44:28thing is to constantly think about
00:44:30improving your processes with the
00:44:32particular goal in mind and we should
00:44:34all be doing that if you leave your
00:44:36development process is the same they
00:44:38don’t stay the same what happens is they
00:44:40gradually degrade over time and become
00:44:42worse and worse if you’re not constantly
00:44:43working to improve then by default
00:44:45you’re gradually getting worse and so I
00:44:48want to end by just what I started with
00:44:52don’t autumn eyes for the case where
00:44:53you’re right assume that what you’re
00:44:54doing is not the best thing you could be
00:44:56doing focus not on cost not on
00:45:01estimation but on how we can best
00:45:03provide value in our products and in the
00:45:05processes that we’re running build
00:45:08feedback loops and grow and water and
00:45:10tend to your feedback loops to validate
00:45:13your assumptions and expose your
00:45:14assumptions work to make it so that you
00:45:17can get feedback in small batches both
00:45:19for your product development and for
00:45:21your process improvement work so that
00:45:23you can take an experimental approach
00:45:24not just a product development but also
00:45:26to process improvement so hopefully you
00:45:29have a couple of minutes for questions
00:45:30in the meantime please remember to rate
00:45:32this session I value your feedback also
00:45:35if you want a bunch of my free stuff
00:45:37email Jess humble at send your sliced
00:45:39calm with the subject DevOps and you’ll
00:45:41get a bunch of free stuff including
00:45:43these slides so that’s Joe Sample at
00:45:44send your sliced calm DevOps questions
00:45:46Marcus has a microphone questions
00:45:54who’s first god I have questions from
00:45:59the tool okay which are about one guy
00:46:03said you know it’s easy to do the split
00:46:05testing and so on yeah it’s not as easy
00:46:08is analyzing and in a similar vein it’s
00:46:11like great that we should evaluate cost
00:46:14of delay or value but how do we do it
00:46:17how do you do there’s loads of different
00:46:19ways to do it I could and have spent a
00:46:22whole day talking about mechanisms to do
00:46:24that cost of delay I really recommends
00:46:27looking at this case study from
00:46:31mask-like they talk a lot about cost of
00:46:33delay in this case study and how to
00:46:35evaluate it but the key thing is this
00:46:37anytime you have an idea you’re going to
00:46:40make assumptions
00:46:41and people tend to find fight about the
00:46:42ideas this idea is good know this idea
00:46:44is good what you want to expose is what
00:46:47assumptions are you making and let’s
00:46:49test the assumptions and find ways to
00:46:51test the assumptions and there’s loads
00:46:54of way to do that like one of my
00:46:55favorite stories is from Zappos so
00:46:57Zappos sells shoes online the first
00:47:00version of the product
00:47:01there was no supply chain what happened
00:47:04was any time someone ordered some shoes
00:47:06the guy would go to the shoe shop buy
00:47:09the shoes and then post them so this is
00:47:13the art of experimentation is finding
00:47:15cheap ways to achieve to run experiments
00:47:19without actually building the thing and
00:47:20I agree that it’s difficult but you know
00:47:22we’re all software developers we didn’t
00:47:24get into software development because it
00:47:26was easy right so you know when people
00:47:28like oh that’s difficult I’m like yes of
00:47:30course it’s difficult I think if you
00:47:32want an easy job you know you came into
00:47:34the wrong field and but I what I do
00:47:36agree is that it’s not a very well known
00:47:38not very well established field so what
00:47:43I urge you to do is come up with
00:47:45ingenious ways to experiment test your
00:47:47assumptions and then blog about them and
00:47:49share them because as a community we all
00:47:51need to get much better at sharing our
00:47:52ideas for doing these things increasing
00:47:54a body of knowledge around it one more
00:47:58question
00:47:58ok I hope over
00:48:10hi hi I would like to ask how how do you
00:48:14start with this because I think most of
00:48:15the guys here are at the like not the
00:48:19top of the food chain in a company and
00:48:21and so do you do you recommend rather
00:48:26buying the books and getting it as a
00:48:28present Christmas present to your
00:48:30classes or or
00:48:33or starting at your position to
00:48:36experiment and ensure the good results
00:48:39so I’m certainly not going to complain
00:48:40if you buy my book for your boss but I
00:48:42think you need to do both right there’s
00:48:44a part of this which is guerrilla
00:48:46experimentation like try and find smart
00:48:49ways to to do this in your own backyards
00:48:52and experiment with it and gain
00:48:53confidence they can actually work and
00:48:55find other people in your company you
00:48:57have who want to try this stuff like you
00:49:00know the best hack of all is finding
00:49:02someone who you don’t normally talk to
00:49:05in your company and taking them out for
00:49:07lunch like find your database
00:49:08administrator take them out for lunch
00:49:10find out why they hate you find out one
00:49:13thing you can do to make their life a
00:49:14little bit better and do that and you
00:49:16know do that once a week just randomly
00:49:18pick someone in your company you don’t
00:49:20normally get to hang out with
00:49:21try and pick the person who you’re like
00:49:24cursing under your breath when something
00:49:25doesn’t work right you’re like that
00:49:27asshole in UX again they’ve sent me this
00:49:30terrible screenshot and I can’t like
00:49:32there are idiots go and take them out
00:49:34for lunch go and find out what’s
00:49:36bothering them there’s a great quote
00:49:38from Jesse Robbins he dirt he says don’t
00:49:41fight stupid make more awesome and I
00:49:45think that’s very important try and find
00:49:47ways to make things more awesome and
00:49:48find the people who you think are being
00:49:50stupid find out why it’s normally
00:49:51because they’re really frustrated about
00:49:53something or they just don’t know what’s
00:49:55going on because we don’t have feedback
00:49:56loops and try and find ways to help make
00:49:59that person more awesome
00:50:00and if we all did that everyday that
00:50:02would be really cool what I will say is
00:50:04it’s hard to create lasting change
00:50:05without executive support and the reason
00:50:07that executives change their minds is
00:50:09usually because there’s a disaster so as
00:50:13a book called by John Kotter called
00:50:16leading change and he says the most
00:50:17important thing for a organizational
00:50:20change to be successful is a sense of
00:50:21urgency
00:50:22so find people
00:50:24all who are feeling the pressure those
00:50:25are normally the people who are willing
00:50:27to try out different things and if you
00:50:29can get those people on board at the
00:50:30senior level and the people in the
00:50:32middle feeding urgency and the people on
00:50:34the ground feeling urgency that’s
00:50:36normally a good place to start people
00:50:38with the level of urgency and and the
00:50:41capability to do something about it
00:50:42that’s your sweet spot for this kind of
00:50:44thing so again not an easy problem but
00:50:47those are my two ideas for helping get
00:50:50started with that stuff thank you very
00:50:53much