[Music] so my name is Vijay I’m a machine
so my name is Vijay I’m a machine
so my name is Vijay I’m a machine learning specialist I work for Google
learning specialist I work for Google
learning specialist I work for Google hope you’re all having a good day today
hope you’re all having a good day today
hope you’re all having a good day today so far how many people saw the last talk
so far how many people saw the last talk
so far how many people saw the last talk in this room the meaning of life talk
in this room the meaning of life talk
in this room the meaning of life talk okay good and that’s actually a good
okay good and that’s actually a good
okay good and that’s actually a good natural segue into this talk because
natural segue into this talk because
natural segue into this talk because that talk was kind of the grand the
that talk was kind of the grand the
that talk was kind of the grand the grand vision of of artificial
grand vision of of artificial
grand vision of of artificial intelligence some of the big milestones
intelligence some of the big milestones
intelligence some of the big milestones and this talk is going to be very much
and this talk is going to be very much
and this talk is going to be very much practical knowledge if you and the
practical knowledge if you and the
practical knowledge if you and the audience want to build a machine
audience want to build a machine
audience want to build a machine learning model how do you do that so
learning model how do you do that so
learning model how do you do that so credit to the conference organizers for
credit to the conference organizers for
credit to the conference organizers for planning it like that how many people
planning it like that how many people
planning it like that how many people have built a machine learning model
have built a machine learning model
have built a machine learning model before okay and how many people have
before okay and how many people have
before okay and how many people have specifically used or at least played
specifically used or at least played
specifically used or at least played around with tensorflow before okay good
around with tensorflow before okay good
around with tensorflow before okay good so the flow of this talk is going to be
so the flow of this talk is going to be
so the flow of this talk is going to be the first 15 minutes or so are gonna be
the first 15 minutes or so are gonna be
the first 15 minutes or so are gonna be slides and then we’re gonna spend the
slides and then we’re gonna spend the
slides and then we’re gonna spend the bulk of the time actually building a
bulk of the time actually building a
bulk of the time actually building a machine learning model end to end I’ll
machine learning model end to end I’ll
machine learning model end to end I’ll actually show you the code execute the
actually show you the code execute the
actually show you the code execute the code this will be the largest screen
code this will be the largest screen
code this will be the largest screen that I’ve ever run live code on so
that I’ve ever run live code on so
that I’ve ever run live code on so hopefully it doesn’t go too badly so
hopefully it doesn’t go too badly so
hopefully it doesn’t go too badly so when I say end-to-end machine learning
when I say end-to-end machine learning
when I say end-to-end machine learning pipeline this is this is what I mean so
pipeline this is this is what I mean so
pipeline this is this is what I mean so you start with the raw data set you do
you start with the raw data set you do
you start with the raw data set you do some pre-processing on that data set
some pre-processing on that data set
some pre-processing on that data set then you train your model you deploy
then you train your model you deploy
then you train your model you deploy that model to some place in this case
that model to some place in this case
that model to some place in this case we’re going to deploy it to the cloud
we’re going to deploy it to the cloud
we’re going to deploy it to the cloud and you run some predictions against
and you run some predictions against
and you run some predictions against that model and specifically the problem
that model and specifically the problem
that model and specifically the problem that we’re going to work on is
that we’re going to work on is
that we’re going to work on is predicting housing prices so we’ll
predicting housing prices so we’ll
predicting housing prices so we’ll actually start with the raw CSV file and
actually start with the raw CSV file and
actually start with the raw CSV file and then go through the whole set of
then go through the whole set of
then go through the whole set of instructions there so before we get into
instructions there so before we get into
instructions there so before we get into that just a couple slides about machine
that just a couple slides about machine
that just a couple slides about machine learning within Google so most of you
learning within Google so most of you
learning within Google so most of you probably know that machine learning is
probably know that machine learning is
probably know that machine learning is pretty central to what we do as a
pretty central to what we do as a
pretty central to what we do as a company we are ultimately a data company
company we are ultimately a data company
company we are ultimately a data company and so this is a chart that shows the
and so this is a chart that shows the
and so this is a chart that shows the growth of machine learning models in
growth of machine learning models in
growth of machine learning models in production in Google over time so you
production in Google over time so you
production in Google over time so you can see even at Google back in 2012 this
can see even at Google back in 2012 this
can see even at Google back in 2012 this was very new for us and fast forward to
was very new for us and fast forward to
was very new for us and fast forward to today it’s we have over 4,000 machine
today it’s we have over 4,000 machine
today it’s we have over 4,000 machine learning models in production and it’s
learning models in production and it’s
learning models in production and it’s gotten to the point where now within
gotten to the point where now within
gotten to the point where now within Google machine learning has become an
Google machine learning has become an
Google machine learning has become an expected skill set for every developer
expected skill set for every developer
expected skill set for every developer to have so you don’t have a special set
to have so you don’t have a special set
to have so you don’t have a special set of data scientists over here and then
of data scientists over here and then
of data scientists over here and then your web developers over here machine
your web developers over here machine
your web developers over here machine learning knowledge is becoming as common
learning knowledge is becoming as common
learning knowledge is becoming as common is knowing Java or Python within Google
is knowing Java or Python within Google
is knowing Java or Python within Google and we offer training courses and
and we offer training courses and
and we offer training courses and programs for every software engineer
programs for every software engineer
programs for every software engineer within Google to get competency within
within Google to get competency within
within Google to get competency within within this field so these are some of
within this field so these are some of
within this field so these are some of the products that we use machine
the products that we use machine
the products that we use machine learning in it’s it’s in almost every
learning in it’s it’s in almost every
learning in it’s it’s in almost every one of our consumer facing products so
one of our consumer facing products so
one of our consumer facing products so it’s a big part of the ranking in our
it’s a big part of the ranking in our
it’s a big part of the ranking in our search algorithm using an algorithm
search algorithm using an algorithm
search algorithm using an algorithm called rankbrain Google image search
called rankbrain Google image search
called rankbrain Google image search relies on machine learning Gmail spam
relies on machine learning Gmail spam
relies on machine learning Gmail spam classification there’s a new feature in
classification there’s a new feature in
classification there’s a new feature in Gmail that some of you may have seen
Gmail that some of you may have seen
Gmail that some of you may have seen called Smart reply where it actually
called Smart reply where it actually
called Smart reply where it actually tries to predict an appropriate response
tries to predict an appropriate response
tries to predict an appropriate response to your email so you don’t have to type
to your email so you don’t have to type
to your email so you don’t have to type it manually which is quite useful so
it manually which is quite useful so
it manually which is quite useful so then when we talk about Google cloud
then when we talk about Google cloud
then when we talk about Google cloud platform that represents the
platform that represents the
platform that represents the externalization of all of this internal
externalization of all of this internal
externalization of all of this internal work we’ve done on our own products so
work we’ve done on our own products so
work we’ve done on our own products so we have very large research teams in
we have very large research teams in
we have very large research teams in last year alone we published over 300
last year alone we published over 300
last year alone we published over 300 research papers in top tier academic
research papers in top tier academic
research papers in top tier academic journals about machine learning and what
journals about machine learning and what
journals about machine learning and what we’ve done is we’ve taken some of these
we’ve done is we’ve taken some of these
we’ve done is we’ve taken some of these models and made them available to the
models and made them available to the
models and made them available to the public so all of that forms falls into
public so all of that forms falls into
public so all of that forms falls into the Google cloud platform umbrella and
the Google cloud platform umbrella and
the Google cloud platform umbrella and you can form a division within that on
you can form a division within that on
you can form a division within that on the left side you have your custom
the left side you have your custom
the left side you have your custom machine learning models and on the right
machine learning models and on the right
machine learning models and on the right side you have your pre trained machine
side you have your pre trained machine
side you have your pre trained machine learning models so most of this talk is
learning models so most of this talk is
learning models so most of this talk is going to be about how do you build a
going to be about how do you build a
going to be about how do you build a custom machine learning model
custom machine learning model
custom machine learning model but I do want to touch just a little bit
but I do want to touch just a little bit
but I do want to touch just a little bit on the pre-training machine learning
on the pre-training machine learning
on the pre-training machine learning model so you know what they are and how
model so you know what they are and how
model so you know what they are and how to use them and when you would use them
to use them and when you would use them
to use them and when you would use them so these machine learning models
so these machine learning models
so these machine learning models typically as a general principle you
typically as a general principle you
typically as a general principle you always want to start with the simplest
always want to start with the simplest
always want to start with the simplest solution and add complexity as necessary
solution and add complexity as necessary
solution and add complexity as necessary so there are some problems that you can
so there are some problems that you can
so there are some problems that you can solve without building a model at all
solve without building a model at all
solve without building a model at all and if you can do that then that’s what
and if you can do that then that’s what
and if you can do that then that’s what you would want to do because there’s
you would want to do because there’s
you would want to do because there’s much less engineering effort then you
much less engineering effort then you
much less engineering effort then you don’t have to reinvent the wheel and
don’t have to reinvent the wheel and
don’t have to reinvent the wheel and these are all very low barrier to entry
these are all very low barrier to entry
these are all very low barrier to entry is you can actually trial these you
is you can actually trial these you
is you can actually trial these you don’t even need a Google cloud account
don’t even need a Google cloud account
don’t even need a Google cloud account you just go to your browser go to the
you just go to your browser go to the
you just go to your browser go to the respective page so this example here is
respective page so this example here is
respective page so this example here is our vision API and you can just drag in
our vision API and you can just drag in
our vision API and you can just drag in an image and it’ll show you exactly what
an image and it’ll show you exactly what
an image and it’ll show you exactly what it’ll it will return so in this case
it’ll it will return so in this case
it’ll it will return so in this case it’s giving me a bunch of labels about
it’s giving me a bunch of labels about
it’s giving me a bunch of labels about this image with the confidence score for
this image with the confidence score for
this image with the confidence score for each one of these so for example it’s 85
each one of these so for example it’s 85
each one of these so for example it’s 85 percent confidence that this is a
percent confidence that this is a
percent confidence that this is a suspension bridge there’s some other
suspension bridge there’s some other
suspension bridge there’s some other features here there’s a landmarks
features here there’s a landmarks
features here there’s a landmarks feature that actually recognizes certain
feature that actually recognizes certain
feature that actually recognizes certain landmarks so it knows that this is
landmarks so it knows that this is
landmarks so it knows that this is specifically the Golden Gate Bridge in
specifically the Golden Gate Bridge in
specifically the Golden Gate Bridge in San Francisco and it gives me the
San Francisco and it gives me the
San Francisco and it gives me the lat/long coordinates and when you’re
lat/long coordinates and when you’re
lat/long coordinates and when you’re calling it over the rest api you’ll get
calling it over the rest api you’ll get
calling it over the rest api you’ll get your results in this JSON format and you
your results in this JSON format and you
your results in this JSON format and you can integrate it into your application
can integrate it into your application
can integrate it into your application as you see fit
so the best way to think about using
so the best way to think about using these pre-trained api’s is as lego
these pre-trained api’s is as lego
these pre-trained api’s is as lego blocks it’s very unlikely that a single
blocks it’s very unlikely that a single
blocks it’s very unlikely that a single pre-trained model is gonna solve your
pre-trained model is gonna solve your
pre-trained model is gonna solve your specific problem entirely by itself
specific problem entirely by itself
specific problem entirely by itself because by the nature of a pre-trained
because by the nature of a pre-trained
because by the nature of a pre-trained model they’re trained on google’s data
model they’re trained on google’s data
model they’re trained on google’s data it’s trying to be a one-size-fits-all
it’s trying to be a one-size-fits-all
it’s trying to be a one-size-fits-all solution so we train on massive data
solution so we train on massive data
solution so we train on massive data sets but it’s very it’s a very
sets but it’s very it’s a very
sets but it’s very it’s a very generalized data set and in your
generalized data set and in your
generalized data set and in your applications you’re working on very
applications you’re working on very
applications you’re working on very specific problems so
specific problems so
specific problems so here’s an example of a customer out of
here’s an example of a customer out of
here’s an example of a customer out of Japan that built a platform to auction
Japan that built a platform to auction
Japan that built a platform to auction off cars and they wanted to add a
off cars and they wanted to add a
off cars and they wanted to add a functionality where if they uploaded an
functionality where if they uploaded an
functionality where if they uploaded an image of a car they would have an API
image of a car they would have an API
image of a car they would have an API that would return certain information
that would return certain information
that would return certain information about that car so it would tell you what
about that car so it would tell you what
about that car so it would tell you what is the make of the car what is the model
is the make of the car what is the model
is the make of the car what is the model of the car if it’s a specific part of
of the car if it’s a specific part of
of the car if it’s a specific part of the car what part of it is is it the
the car what part of it is is it the
the car what part of it is is it the tires the steering wheel and also what
tires the steering wheel and also what
tires the steering wheel and also what side of the car are you looking at are
side of the car are you looking at are
side of the car are you looking at are you looking at the front or the left or
you looking at the front or the left or
you looking at the front or the left or the right of the car and the reason that
the right of the car and the reason that
the right of the car and the reason that this was important is because that’s how
this was important is because that’s how
this was important is because that’s how they categorized these images for their
they categorized these images for their
they categorized these images for their consumers to look at on the website so
consumers to look at on the website so
consumers to look at on the website so before machine learning they had actual
before machine learning they had actual
before machine learning they had actual human to go in and tag each of these
human to go in and tag each of these
human to go in and tag each of these images and this is the left side of a
images and this is the left side of a
images and this is the left side of a Ford Focus etc very time-consuming and
Ford Focus etc very time-consuming and
Ford Focus etc very time-consuming and they wanted to try to use machine
they wanted to try to use machine
they wanted to try to use machine learning to automate that and what they
learning to automate that and what they
learning to automate that and what they did at first was a common mistake which
did at first was a common mistake which
did at first was a common mistake which is try to build a single model that
is try to build a single model that
is try to build a single model that solves that entire problem that spits
solves that entire problem that spits
solves that entire problem that spits out all of the information that they
out all of the information that they
out all of the information that they care about from a single model and they
care about from a single model and they
care about from a single model and they invested some significant engineering
invested some significant engineering
invested some significant engineering effort and eventually gave up because
effort and eventually gave up because
effort and eventually gave up because the model just wasn’t giving the results
the model just wasn’t giving the results
the model just wasn’t giving the results that they needed then they took a step
that they needed then they took a step
that they needed then they took a step back and said what if we break this into
back and said what if we break this into
back and said what if we break this into multiple pieces so they they said first
multiple pieces so they they said first
multiple pieces so they they said first let’s use Google’s pre train models and
let’s use Google’s pre train models and
let’s use Google’s pre train models and see how far that gives us gets us so
see how far that gives us gets us so
see how far that gives us gets us so they ran their images through our vision
they ran their images through our vision
they ran their images through our vision API which is zero engineering work on
API which is zero engineering work on
API which is zero engineering work on their part because we’ve already done
their part because we’ve already done
their part because we’ve already done the model and exposed it and they’ve
the model and exposed it and they’ve
the model and exposed it and they’ve noticed that’s all part of the problem
noticed that’s all part of the problem
noticed that’s all part of the problem it tells you what is the make of a car
it tells you what is the make of a car
it tells you what is the make of a car but not what is a specific model and it
but not what is a specific model and it
but not what is a specific model and it also can identify what part of a car it
also can identify what part of a car it
also can identify what part of a car it is is this a tire or is this a steering
is is this a tire or is this a steering
is is this a tire or is this a steering wheel but what it could not do is for
wheel but what it could not do is for
wheel but what it could not do is for example tell you is the left side or the
example tell you is the left side or the
example tell you is the left side or the right side of a car because it just
right side of a car because it just
right side of a car because it just wasn’t trained to do that so now when
wasn’t trained to do that so now when
wasn’t trained to do that so now when you narrow the scope of the problem to
you narrow the scope of the problem to
you narrow the scope of the problem to just that it’s actually quite easy to
just that it’s actually quite easy to
just that it’s actually quite easy to build a custom model that does that
build a custom model that does that
build a custom model that does that using a framework like tensorflow
using a framework like tensorflow
using a framework like tensorflow you get a thousand images of left sides
you get a thousand images of left sides
you get a thousand images of left sides of cars and a thousand images of right
of cars and a thousand images of right
of cars and a thousand images of right sides of cars different
sides of cars different
sides of cars different types of cars and you put them through a
types of cars and you put them through a
types of cars and you put them through a neural network and you get a pretty
neural network and you get a pretty
neural network and you get a pretty robust algorithm that can now tell the
robust algorithm that can now tell the
robust algorithm that can now tell the difference between the left and right
difference between the left and right
difference between the left and right side of a car and that solves part of
side of a car and that solves part of
side of a car and that solves part of the problem so they did this in a
the problem so they did this in a
the problem so they did this in a piecemeal approach and then aggregated
piecemeal approach and then aggregated
piecemeal approach and then aggregated the information from each of these
the information from each of these
the information from each of these individual models these small models to
individual models these small models to
individual models these small models to solve the total business problem and
solve the total business problem and
solve the total business problem and they ended up getting where they want it
they ended up getting where they want it
they ended up getting where they want it to be but in a much faster and cleaner
to be but in a much faster and cleaner
to be but in a much faster and cleaner way so that’s really how you should
way so that’s really how you should
way so that’s really how you should think about these problems there’s a lot
think about these problems there’s a lot
think about these problems there’s a lot of times you can make them modular and a
of times you can make them modular and a
of times you can make them modular and a series of smaller simpler models can
series of smaller simpler models can
series of smaller simpler models can aggregate to solve what you’re trying to
aggregate to solve what you’re trying to
aggregate to solve what you’re trying to solve okay so that’s it for pre-trained
solve okay so that’s it for pre-trained
solve okay so that’s it for pre-trained models now let’s say you do need to
models now let’s say you do need to
models now let’s say you do need to build a custom machine learning model
build a custom machine learning model
build a custom machine learning model you have narrow the scope of it as much
you have narrow the scope of it as much
you have narrow the scope of it as much as possible to use things that were
as possible to use things that were
as possible to use things that were already built so you’re not reinventing
already built so you’re not reinventing
already built so you’re not reinventing the wheel but usually there’s at least
the wheel but usually there’s at least
the wheel but usually there’s at least some part that you have to build custom
some part that you have to build custom
some part that you have to build custom and when I say custom that means you’re
and when I say custom that means you’re
and when I say custom that means you’re training on your own data you’re writing
training on your own data you’re writing
training on your own data you’re writing your own code and the framework that we
your own code and the framework that we
your own code and the framework that we at Google internally used for this type
at Google internally used for this type
at Google internally used for this type of thing is called tension flow so
of thing is called tension flow so
of thing is called tension flow so tensor flow is something that probably
tensor flow is something that probably
tensor flow is something that probably most of you have heard of it seemed like
most of you have heard of it seemed like
most of you have heard of it seemed like about a quarter to a third of the room
about a quarter to a third of the room
about a quarter to a third of the room has actually already used so first of
has actually already used so first of
has actually already used so first of all what is tensor flow tensor flow is a
all what is tensor flow tensor flow is a
all what is tensor flow tensor flow is a machine learning learning framework it
machine learning learning framework it
machine learning learning framework it was open source by Google in November of
was open source by Google in November of
was open source by Google in November of 2015 and since it has been open source
2015 and since it has been open source
2015 and since it has been open source it’s become the most popular machine
it’s become the most popular machine
it’s become the most popular machine learning framework and I’ll talk about
learning framework and I’ll talk about
learning framework and I’ll talk about some reasons to why it may have become
some reasons to why it may have become
some reasons to why it may have become so popular so one is it’s worth
so popular so one is it’s worth
so popular so one is it’s worth mentioning that the popularity of
mentioning that the popularity of
mentioning that the popularity of tensorflow itself is a big reason to use
tensorflow itself is a big reason to use
tensorflow itself is a big reason to use it because with popularity becomes
it because with popularity becomes
it because with popularity becomes developer ecosystem so if you’re working
developer ecosystem so if you’re working
developer ecosystem so if you’re working on a problem chances are someone has
on a problem chances are someone has
on a problem chances are someone has already solved a problem similar to it
already solved a problem similar to it
already solved a problem similar to it using tensor flow and you could do a
using tensor flow and you could do a
using tensor flow and you could do a search and you’ll find a blog post if
search and you’ll find a blog post if
search and you’ll find a blog post if not a official tensor flow tutorial
not a official tensor flow tutorial
not a official tensor flow tutorial showing you a code template and you can
showing you a code template and you can
showing you a code template and you can use that to bootstrap your own project
use that to bootstrap your own project
use that to bootstrap your own project from so that’s a very valuable and the
from so that’s a very valuable and the
from so that’s a very valuable and the second thing there is
second thing there is
second thing there is if you get stuck or you run into a bug
if you get stuck or you run into a bug
if you get stuck or you run into a bug when you post it on to something like
when you post it on to something like
when you post it on to something like Stack Overflow because there’s such a
Stack Overflow because there’s such a
Stack Overflow because there’s such a large community around it you’re
large community around it you’re
large community around it you’re typically going to get a response faster
typically going to get a response faster
typically going to get a response faster than if you’re using a lesser used
than if you’re using a lesser used
than if you’re using a lesser used framework so the popularity itself is a
framework so the popularity itself is a
framework so the popularity itself is a big asset at this point another reason
big asset at this point another reason
big asset at this point another reason that tensorflow is popular is this idea
that tensorflow is popular is this idea
that tensorflow is popular is this idea that you can program it at various
that you can program it at various
that you can program it at various levels of abstraction so if you don’t
levels of abstraction so if you don’t
levels of abstraction so if you don’t need a lot of control over your
need a lot of control over your
need a lot of control over your algorithm if you’re not a researcher
algorithm if you’re not a researcher
algorithm if you’re not a researcher that’s that’s tweaking your algorithm
that’s that’s tweaking your algorithm
that’s that’s tweaking your algorithm down to a very fine level of granularity
down to a very fine level of granularity
down to a very fine level of granularity then you can abstract yourself away from
then you can abstract yourself away from
then you can abstract yourself away from that using a higher-level API like
that using a higher-level API like
that using a higher-level API like Kerris which some of you may have heard
Kerris which some of you may have heard
Kerris which some of you may have heard of or like the estimator API which is
of or like the estimator API which is
of or like the estimator API which is actually the code that I’ll be showing
actually the code that I’ll be showing
actually the code that I’ll be showing later on in the talk and this allows you
later on in the talk and this allows you
later on in the talk and this allows you to do things like just use a quote
to do things like just use a quote
to do things like just use a quote unquote canned model that’s pre
unquote canned model that’s pre
unquote canned model that’s pre implemented for you it already has the
implemented for you it already has the
implemented for you it already has the training loop built in already has the
training loop built in already has the
training loop built in already has the evaluation loop built in and with just a
evaluation loop built in and with just a
evaluation loop built in and with just a couple lines of code you call that and
couple lines of code you call that and
couple lines of code you call that and it saves you a lot of work but if you do
it saves you a lot of work but if you do
it saves you a lot of work but if you do need additional flexibility you can drop
need additional flexibility you can drop
need additional flexibility you can drop down a level in the hierarchy to these
down a level in the hierarchy to these
down a level in the hierarchy to these lower levels ultimately down to the the
lower levels ultimately down to the the
lower levels ultimately down to the the base vanilla tensorflow level where
base vanilla tensorflow level where
base vanilla tensorflow level where you’re explicitly defining your matrices
you’re explicitly defining your matrices
you’re explicitly defining your matrices and doing your matrix multiplications at
and doing your matrix multiplications at
and doing your matrix multiplications at that point another reason to enter flow
that point another reason to enter flow
that point another reason to enter flow is popular is because it’s
is popular is because it’s
is popular is because it’s production-ready
production-ready
production-ready so the reason Google created tensorflow
so the reason Google created tensorflow
so the reason Google created tensorflow in the first place is because
in the first place is because
in the first place is because historically the deep learning
historically the deep learning
historically the deep learning frameworks that existed at the time all
frameworks that existed at the time all
frameworks that existed at the time all came out of academic institutions so MX
came out of academic institutions so MX
came out of academic institutions so MX net came out of Carnegie Mellon dno came
net came out of Carnegie Mellon dno came
net came out of Carnegie Mellon dno came out of the University of Montreal Cafe
out of the University of Montreal Cafe
out of the University of Montreal Cafe came out of the University of Berkeley
came out of the University of Berkeley
came out of the University of Berkeley and if you think of the needs of an
and if you think of the needs of an
and if you think of the needs of an academic researcher is very different
academic researcher is very different
academic researcher is very different from the needs of someone who’s in
from the needs of someone who’s in
from the needs of someone who’s in industry
industry
industry so a researcher their motivation is
so a researcher their motivation is
so a researcher their motivation is essentially to prototype something so
essentially to prototype something so
essentially to prototype something so that they can prove out a theory publish
that they can prove out a theory publish
that they can prove out a theory publish that theory in a academic journal and
that theory in a academic journal and
that theory in a academic journal and then move on to something else so
then move on to something else so
then move on to something else so there’s no real thought about how do i
there’s no real thought about how do i
there’s no real thought about how do i scale this model how many users are
scale this model how many users are
scale this model how many users are going to hit it what are my security
going to hit it what are my security
going to hit it what are my security considerations and things like that so
considerations and things like that so
considerations and things like that so when Google start got into this machine
when Google start got into this machine
when Google start got into this machine learning space knowing that everything
learning space knowing that everything
learning space knowing that everything they built was going to be used by
they built was going to be used by
they built was going to be used by millions of users they had to create a
millions of users they had to create a
millions of users they had to create a tool to solve this problem that didn’t
tool to solve this problem that didn’t
tool to solve this problem that didn’t exist before so all of these things like
exist before so all of these things like
exist before so all of these things like serving in latency are first class
serving in latency are first class
serving in latency are first class citizens and tensorflow
citizens and tensorflow
citizens and tensorflow and they’re built into the tool so it’s
and they’re built into the tool so it’s
and they’re built into the tool so it’s very performant and you can also deploy
very performant and you can also deploy
very performant and you can also deploy it across different types of hardware so
it across different types of hardware so
it across different types of hardware so you write your code once and then you
you write your code once and then you
you write your code once and then you can deploy it on a CPU or GPU or on a
can deploy it on a CPU or GPU or on a
can deploy it on a CPU or GPU or on a mobile phone Android or iOS or even on
mobile phone Android or iOS or even on
mobile phone Android or iOS or even on embedded devices like Raspberry Pi and
embedded devices like Raspberry Pi and
embedded devices like Raspberry Pi and of course algorithms so the main reason
of course algorithms so the main reason
of course algorithms so the main reason that you use a machine learning
that you use a machine learning
that you use a machine learning framework is so that you don’t have to
framework is so that you don’t have to
framework is so that you don’t have to re-implement algorithms that are already
re-implement algorithms that are already
re-implement algorithms that are already well established and well proven in
well established and well proven in
well established and well proven in industry so this is an example of the
industry so this is an example of the
industry so this is an example of the tensor flow object detection API
tensor flow object detection API
tensor flow object detection API it is a algorithm that identifies labels
it is a algorithm that identifies labels
it is a algorithm that identifies labels in an image and not only does that but
in an image and not only does that but
in an image and not only does that but it draws bounding boxes around those
it draws bounding boxes around those
it draws bounding boxes around those labels so this is a much harder problem
labels so this is a much harder problem
labels so this is a much harder problem than the the original type of computer
than the the original type of computer
than the the original type of computer vision problem which was image
vision problem which was image
vision problem which was image classification that would tell me that
classification that would tell me that
classification that would tell me that there’s a kite in the image somewhere
there’s a kite in the image somewhere
there’s a kite in the image somewhere but it wouldn’t tell me where within the
but it wouldn’t tell me where within the
but it wouldn’t tell me where within the image that kite was or it wouldn’t tell
image that kite was or it wouldn’t tell
image that kite was or it wouldn’t tell me how many kites are in that image so
me how many kites are in that image so
me how many kites are in that image so this is a algorithm that came out of
this is a algorithm that came out of
this is a algorithm that came out of Google research and then we open sourced
Google research and then we open sourced
Google research and then we open sourced the results of this via tensor flow so
the results of this via tensor flow so
the results of this via tensor flow so now you can actually leverage this
now you can actually leverage this
now you can actually leverage this algorithm with a few lines of code and
algorithm with a few lines of code and
algorithm with a few lines of code and if you just do a Google search for ten
if you just do a Google search for ten
if you just do a Google search for ten to flow object detection API you’ll see
to flow object detection API you’ll see
to flow object detection API you’ll see a blog post that tells you how to do
a blog post that tells you how to do
a blog post that tells you how to do that
and here is another example this this
and here is another example this this idea of a wide and deep algorithm is no
idea of a wide and deep algorithm is no
idea of a wide and deep algorithm is no longer really considered
longer really considered
longer really considered state-of-the-art but when it came out
state-of-the-art but when it came out
state-of-the-art but when it came out tensorflow was the phrase first popular
tensorflow was the phrase first popular
tensorflow was the phrase first popular framework to kind of distribute this or
framework to kind of distribute this or
framework to kind of distribute this or democratize this type of modeling to the
democratize this type of modeling to the
democratize this type of modeling to the quote unquote common man so the idea of
quote unquote common man so the idea of
quote unquote common man so the idea of widened deep models is you have this
widened deep models is you have this
widened deep models is you have this thing called deep learning which works
thing called deep learning which works
thing called deep learning which works very well for dense datasets and you
very well for dense datasets and you
very well for dense datasets and you have linear models which work very well
have linear models which work very well
have linear models which work very well for sparse data sets and the idea is
for sparse data sets and the idea is
for sparse data sets and the idea is quite simple which is what if we combine
quite simple which is what if we combine
quite simple which is what if we combine the best of both worlds and have the the
the best of both worlds and have the the
the best of both worlds and have the the deep model work on the dense features
deep model work on the dense features
deep model work on the dense features and the wide model work on the sparse
and the wide model work on the sparse
and the wide model work on the sparse features and then combine the results at
features and then combine the results at
features and then combine the results at the end that’s called widened deep
the end that’s called widened deep
the end that’s called widened deep learning and that was another example of
learning and that was another example of
learning and that was another example of an algorithm that tensorflow really
an algorithm that tensorflow really
an algorithm that tensorflow really brought to the general public
speed so of course when you’re thinking
speed so of course when you’re thinking about deploying something for production
about deploying something for production
about deploying something for production especially if it’s going to power a
especially if it’s going to power a
especially if it’s going to power a website or you you have some real time
website or you you have some real time
website or you you have some real time need for it then performance becomes
need for it then performance becomes
need for it then performance becomes very important so tints flow is very
very important so tints flow is very
very important so tints flow is very transparent about its benchmarks on
transparent about its benchmarks on
transparent about its benchmarks on different problems how it scales
different problems how it scales
different problems how it scales horizontally and you can go to
horizontally and you can go to
horizontally and you can go to tensorflow dot org slash performance
tensorflow dot org slash performance
tensorflow dot org slash performance benchmarks and just see the tests we’ve
benchmarks and just see the tests we’ve
benchmarks and just see the tests we’ve run and see exactly what our
run and see exactly what our
run and see exactly what our performances and how it compares and
performances and how it compares and
performances and how it compares and then the last thing I’ll say about
then the last thing I’ll say about
then the last thing I’ll say about tension flow is the reason why it can
tension flow is the reason why it can
tension flow is the reason why it can you have this trade-off of it both being
you have this trade-off of it both being
you have this trade-off of it both being developer friendly written in a language
developer friendly written in a language
developer friendly written in a language like Python which is considered
like Python which is considered
like Python which is considered developer friendly but traditionally
developer friendly but traditionally
developer friendly but traditionally considered slow but also very performant
considered slow but also very performant
considered slow but also very performant is because the output of your tensorflow
is because the output of your tensorflow
is because the output of your tensorflow Python code is a what we call a data
Python code is a what we call a data
Python code is a what we call a data flow graph which is looks like this and
flow graph which is looks like this and
flow graph which is looks like this and this is actually visualized using a tool
this is actually visualized using a tool
this is actually visualized using a tool called tension board which I’ll also
called tension board which I’ll also
called tension board which I’ll also demo and that graph is just
demo and that graph is just
demo and that graph is just series of instructions that is then
series of instructions that is then
series of instructions that is then interpreted by a C++ back end and the
interpreted by a C++ back end and the
interpreted by a C++ back end and the C++ back and takes care of how do I
C++ back and takes care of how do I
C++ back and takes care of how do I compile this code in a very fast way and
compile this code in a very fast way and
compile this code in a very fast way and specific to the type of hardware that
specific to the type of hardware that
specific to the type of hardware that I’m instructed to run on whether that’s
I’m instructed to run on whether that’s
I’m instructed to run on whether that’s a CPU or GPU for example
so that’s tensorflow is a framework why
so that’s tensorflow is a framework why you would want to use that for your
you would want to use that for your
you would want to use that for your custom machine learning modeling and
custom machine learning modeling and
custom machine learning modeling and then why Google Cloud machine learning
then why Google Cloud machine learning
then why Google Cloud machine learning engine so basically why would you do
engine so basically why would you do
engine so basically why would you do this in the cloud so the first thing to
this in the cloud so the first thing to
this in the cloud so the first thing to state here is a lot of times you don’t
state here is a lot of times you don’t
state here is a lot of times you don’t need to do it in the cloud as a rule of
need to do it in the cloud as a rule of
need to do it in the cloud as a rule of thumb if you can train your model in
thumb if you can train your model in
thumb if you can train your model in under an hour on your local laptop just
under an hour on your local laptop just
under an hour on your local laptop just stick on your local laptop but at some
stick on your local laptop but at some
stick on your local laptop but at some point if you’re using large enough data
point if you’re using large enough data
point if you’re using large enough data sets or you’re using complex enough
sets or you’re using complex enough
sets or you’re using complex enough algorithms you’re going to hit that
algorithms you’re going to hit that
algorithms you’re going to hit that bottleneck where the training time on
bottleneck where the training time on
bottleneck where the training time on your computer your computer hour
your computer your computer hour
your computer your computer hour essentially becomes your bottleneck to
essentially becomes your bottleneck to
essentially becomes your bottleneck to creativity and development and it’ll
creativity and development and it’ll
creativity and development and it’ll stop you from trying different ideas or
stop you from trying different ideas or
stop you from trying different ideas or it’ll stop a problem from being
it’ll stop a problem from being
it’ll stop a problem from being tractable altogether and that’s when you
tractable altogether and that’s when you
tractable altogether and that’s when you want to make the jump to cloud computing
want to make the jump to cloud computing
want to make the jump to cloud computing because essentially then you have access
because essentially then you have access
because essentially then you have access to a supercomputer over over an internet
to a supercomputer over over an internet
to a supercomputer over over an internet connection and the tool I’ll actually be
connection and the tool I’ll actually be
connection and the tool I’ll actually be using today is called Google Cloud
using today is called Google Cloud
using today is called Google Cloud machine learning engine which is a
machine learning engine which is a
machine learning engine which is a managed tensorflow service and it gives
managed tensorflow service and it gives
managed tensorflow service and it gives you some nice things for example it
you some nice things for example it
you some nice things for example it makes horizontal scaling very easy you
makes horizontal scaling very easy you
makes horizontal scaling very easy you just specify I want to run this across
just specify I want to run this across
just specify I want to run this across ten GPUs and Google will take care of
ten GPUs and Google will take care of
ten GPUs and Google will take care of spinning up the infrastructure for you
spinning up the infrastructure for you
spinning up the infrastructure for you provisioning the virtual machines and
provisioning the virtual machines and
provisioning the virtual machines and then spinning them down when your job is
then spinning them down when your job is
then spinning them down when your job is done it has an automatic hyper parameter
done it has an automatic hyper parameter
done it has an automatic hyper parameter tuning function automatic monitoring
tuning function automatic monitoring
tuning function automatic monitoring logging versioning built-in auto scaling
logging versioning built-in auto scaling
logging versioning built-in auto scaling prediction service and coming soon
prediction service and coming soon
prediction service and coming soon you’ll have access to tensor processing
you’ll have access to tensor processing
you’ll have access to tensor processing units which I’ll talk about on the next
units which I’ll talk about on the next
units which I’ll talk about on the next slide and there’s no lock-in
slide and there’s no lock-in
slide and there’s no lock-in so because tensorflow is an open source
so because tensorflow is an open source
so because tensorflow is an open source framework once you train your model even
framework once you train your model even
framework once you train your model even if you use our product as your
if you use our product as your
if you use our product as your supercomputer to give you the horsepower
supercomputer to give you the horsepower
supercomputer to give you the horsepower to complete that training in a
to complete that training in a
to complete that training in a reasonable amount of time
reasonable amount of time
reasonable amount of time the result of that is an open source
the result of that is an open source
the result of that is an open source so you can download that model you can
so you can download that model you can
so you can download that model you can deploy it on a phone you can deploy it
deploy it on a phone you can deploy it
deploy it on a phone you can deploy it on premise you can deploy it anywhere
on premise you can deploy it anywhere
on premise you can deploy it anywhere you want to and the reverse is true as
you want to and the reverse is true as
you want to and the reverse is true as well you can train your tension flow
well you can train your tension flow
well you can train your tension flow model on your local laptop but you can
model on your local laptop but you can
model on your local laptop but you can deploy it to our cloud just for the
deploy it to our cloud just for the
deploy it to our cloud just for the prediction service because maybe you’re
prediction service because maybe you’re
prediction service because maybe you’re expecting hundreds of thousands
expecting hundreds of thousands
expecting hundreds of thousands prediction predictions per second and
prediction predictions per second and
prediction predictions per second and you don’t want to build it up your own
you don’t want to build it up your own
you don’t want to build it up your own infrastructure to handle that so you can
infrastructure to handle that so you can
infrastructure to handle that so you can train on premise and deploy for
train on premise and deploy for
train on premise and deploy for prediction in the cloud as well
prediction in the cloud as well
prediction in the cloud as well tensor processing unit for those of you
tensor processing unit for those of you
tensor processing unit for those of you who haven’t heard of this this was the
who haven’t heard of this this was the
who haven’t heard of this this was the hardware that powered the the artificial
hardware that powered the the artificial
hardware that powered the the artificial intelligence that defeated the go
intelligence that defeated the go
intelligence that defeated the go champion if you heard of that it also
champion if you heard of that it also
champion if you heard of that it also powers Google Translate and a bunch of
powers Google Translate and a bunch of
powers Google Translate and a bunch of other internal applications and we’ve
other internal applications and we’ve
other internal applications and we’ve now made this available to the to the
now made this available to the to the
now made this available to the to the public in alpha at this point and just a
public in alpha at this point and just a
public in alpha at this point and just a hardware comparison to give you kind of
hardware comparison to give you kind of
hardware comparison to give you kind of a scale of how powerful these things are
a scale of how powerful these things are
a scale of how powerful these things are a tensor processing unit can process 180
a tensor processing unit can process 180
a tensor processing unit can process 180 trillion floating-point operations per
trillion floating-point operations per
trillion floating-point operations per second an Nvidia K 80 Tesla kad GPU
second an Nvidia K 80 Tesla kad GPU
second an Nvidia K 80 Tesla kad GPU which is kind of the de facto cloud GPU
which is kind of the de facto cloud GPU
which is kind of the de facto cloud GPU at this point is nine teraflops so this
at this point is nine teraflops so this
at this point is nine teraflops so this becomes actually more important for
becomes actually more important for
becomes actually more important for inference than training because in
inference than training because in
inference than training because in training can always just scale
training can always just scale
training can always just scale horizontally so you could use 20 Nvidia
horizontally so you could use 20 Nvidia
horizontally so you could use 20 Nvidia K ATS and you’d get a similar speed to
K ATS and you’d get a similar speed to
K ATS and you’d get a similar speed to using a single TP you you just have to
using a single TP you you just have to
using a single TP you you just have to do the price performance calculation but
do the price performance calculation but
do the price performance calculation but when it comes to inference also in those
when it comes to inference also in those
when it comes to inference also in those prediction time and you need to get that
prediction time and you need to get that
prediction time and you need to get that result back within milliseconds the
result back within milliseconds the
result back within milliseconds the overhead of horizontal computing
overhead of horizontal computing
overhead of horizontal computing actually it becomes more will take you
actually it becomes more will take you
actually it becomes more will take you out of your real time requirement so at
out of your real time requirement so at
out of your real time requirement so at that point vertical scaling on a single
that point vertical scaling on a single
that point vertical scaling on a single chip becomes important and that’s when
chip becomes important and that’s when
chip becomes important and that’s when these advanced chips like TPS becomes
these advanced chips like TPS becomes
these advanced chips like TPS becomes more important okay so on to the actual
more important okay so on to the actual
more important okay so on to the actual problem that we’re gonna build an
problem that we’re gonna build an
problem that we’re gonna build an antenna solution for so what we’re gonna
antenna solution for so what we’re gonna
antenna solution for so what we’re gonna do is predict housing sale prices and so
do is predict housing sale prices and so
do is predict housing sale prices and so I’m gonna start by
I’m gonna start by
I’m gonna start by asking you at the audience to make a
asking you at the audience to make a
asking you at the audience to make a prediction on a simple data set so my
prediction on a simple data set so my
prediction on a simple data set so my dataset is a 1,000 square foot house
dataset is a 1,000 square foot house
dataset is a 1,000 square foot house sells for $100,000 a 3,000 square-foot
sells for $100,000 a 3,000 square-foot
sells for $100,000 a 3,000 square-foot house sells for $300,000 and my question
house sells for $300,000 and my question
house sells for $300,000 and my question for you and you can just yell it out is
for you and you can just yell it out is
for you and you can just yell it out is how much would a 2000 square-foot house
how much would a 2000 square-foot house
how much would a 2000 square-foot house all for two hundred thousand but it’s
all for two hundred thousand but it’s
all for two hundred thousand but it’s pretty easy right so let’s make it a
pretty easy right so let’s make it a
pretty easy right so let’s make it a little bit more complicated let’s add
little bit more complicated let’s add
little bit more complicated let’s add two more features to my data set so in
two more features to my data set so in
two more features to my data set so in addition to just square footage of a
addition to just square footage of a
addition to just square footage of a house I’m gonna predict the housing
house I’m gonna predict the housing
house I’m gonna predict the housing price based off of what is the crime
price based off of what is the crime
price based off of what is the crime rate in the area and also how good are
rate in the area and also how good are
rate in the area and also how good are the schools in the area so now I’m just
the schools in the area so now I’m just
the schools in the area so now I’m just gonna let you read this slide and the
gonna let you read this slide and the
gonna let you read this slide and the question is how much would a three
question is how much would a three
question is how much would a three thousand square foot house with a crime
thousand square foot house with a crime
thousand square foot house with a crime rate of 30 and a school rating of two
rate of 30 and a school rating of two
rate of 30 and a school rating of two cost any brave souls won’t answer yeah
cost any brave souls won’t answer yeah
cost any brave souls won’t answer yeah it’s not so easy now and that’s just
it’s not so easy now and that’s just
it’s not so easy now and that’s just with three features so imagine if you
with three features so imagine if you
with three features so imagine if you had a hundred features and imagine if
had a hundred features and imagine if
had a hundred features and imagine if you had millions of rows this is where
you had millions of rows this is where
you had millions of rows this is where you now need to use a machine learning
you now need to use a machine learning
you now need to use a machine learning to make this type of prediction so this
to make this type of prediction so this
to make this type of prediction so this is the problem that we’re going to be
is the problem that we’re going to be
is the problem that we’re going to be working on so in order to solve this and
working on so in order to solve this and
working on so in order to solve this and actually by the way if you want to
actually by the way if you want to
actually by the way if you want to recreate this lab after the after this
recreate this lab after the after this
recreate this lab after the after this talk this is the short link to some
talk this is the short link to some
talk this is the short link to some documentation on how to do that so you
documentation on how to do that so you
documentation on how to do that so you can snap a picture of this also you’ll
can snap a picture of this also you’ll
can snap a picture of this also you’ll have access to the slides
have access to the slides
have access to the slides after the through the conference website
okay so to do to actually do our
okay so to do to actually do our development we’re going to use a tool
development we’re going to use a tool
development we’re going to use a tool called data lab so data lab is basically
called data lab so data lab is basically
called data lab so data lab is basically a white labeled version of Jupiter or
a white labeled version of Jupiter or
a white labeled version of Jupiter or ipython notebooks how many people have
ipython notebooks how many people have
ipython notebooks how many people have used Jupiter or ipython notebooks okay
used Jupiter or ipython notebooks okay
used Jupiter or ipython notebooks okay so – it’s a good way to combine your
so – it’s a good way to combine your
so – it’s a good way to combine your code with documentation in a way that’s
code with documentation in a way that’s
code with documentation in a way that’s kind of digestible and easy to share and
kind of digestible and easy to share and
kind of digestible and easy to share and collaborate so within this notebook I’m
collaborate so within this notebook I’m
collaborate so within this notebook I’m starting by just importing the
starting by just importing the
starting by just importing the frameworks that I need to use so in this
frameworks that I need to use so in this
frameworks that I need to use so in this case I’m using pandas which is a
case I’m using pandas which is a
case I’m using pandas which is a framework that enables me to deal with
framework that enables me to deal with
framework that enables me to deal with the CSV data that I’m gonna use easily
the CSV data that I’m gonna use easily
the CSV data that I’m gonna use easily and parse it and then of course
and parse it and then of course
and parse it and then of course tensorflow so I’ll just execute those
tensorflow so I’ll just execute those
tensorflow so I’ll just execute those two cells and it’s always a good idea to
two cells and it’s always a good idea to
two cells and it’s always a good idea to know what version of tension flow that
know what version of tension flow that
know what version of tension flow that you’re dealing with so you can debug any
you’re dealing with so you can debug any
you’re dealing with so you can debug any version conflict issues so I’m using
version conflict issues so I’m using
version conflict issues so I’m using test foal 1.2 and we talked a little bit
test foal 1.2 and we talked a little bit
test foal 1.2 and we talked a little bit about this idea of tons of flow api’s
about this idea of tons of flow api’s
about this idea of tons of flow api’s and the hierarchy so in this case I’m
and the hierarchy so in this case I’m
and the hierarchy so in this case I’m gonna be programming at these top two
gonna be programming at these top two
gonna be programming at these top two levels the experiment and estimators API
levels the experiment and estimators API
levels the experiment and estimators API and the main reason I’m gonna do this is
and the main reason I’m gonna do this is
and the main reason I’m gonna do this is one it makes for less lines of code but
one it makes for less lines of code but
one it makes for less lines of code but the other reason is it gives me
the other reason is it gives me
the other reason is it gives me distributed computing for free so as
distributed computing for free so as
distributed computing for free so as long as I stick to these higher-level
long as I stick to these higher-level
long as I stick to these higher-level api’s then the then it doesn’t tend to
api’s then the then it doesn’t tend to
api’s then the then it doesn’t tend to float just understands how to run on ten
float just understands how to run on ten
float just understands how to run on ten GPUs versus one GPU and I don’t have to
GPUs versus one GPU and I don’t have to
GPUs versus one GPU and I don’t have to make any code changes which if you’ve
make any code changes which if you’ve
make any code changes which if you’ve tried to do distributed computing across
tried to do distributed computing across
tried to do distributed computing across multiple CPUs or GPUs manually before
multiple CPUs or GPUs manually before
multiple CPUs or GPUs manually before you’ll know this is a big time saver it
you’ll know this is a big time saver it
you’ll know this is a big time saver it saves you a lot of headache and so the
saves you a lot of headache and so the
saves you a lot of headache and so the actual steps that we’re going to go
actual steps that we’re going to go
actual steps that we’re going to go through in this notebook are one we’re
through in this notebook are one we’re
through in this notebook are one we’re going to load in our raw data we’re
going to load in our raw data we’re
going to load in our raw data we’re gonna write our tensorflow code and
gonna write our tensorflow code and
gonna write our tensorflow code and that’s gonna have some steps broken down
that’s gonna have some steps broken down
that’s gonna have some steps broken down which I’ll explain then we’re going to
which I’ll explain then we’re going to
which I’ll explain then we’re going to package that code up so that we can
package that code up so that we can
package that code up so that we can train it in the cloud we’re going to
train it in the cloud we’re going to
train it in the cloud we’re going to inspect the results of that training
inspect the results of that training
inspect the results of that training using tensorflow to make sure that we’re
using tensorflow to make sure that we’re
using tensorflow to make sure that we’re happy with our
happy with our
happy with our with our accuracy once we’re happy with
with our accuracy once we’re happy with
with our accuracy once we’re happy with their accuracy we’re going to deploy
their accuracy we’re going to deploy
their accuracy we’re going to deploy that model to the cloud so that we can
that model to the cloud so that we can
that model to the cloud so that we can have a third party make predictions
have a third party make predictions
have a third party make predictions against that model over the internet so
we’ll start here by loading our raw data
we’ll start here by loading our raw data and I’m using just a publicly available
and I’m using just a publicly available
and I’m using just a publicly available data set about Boston area suburbs circa
data set about Boston area suburbs circa
data set about Boston area suburbs circa 1978 and it’s hosted in a Google Cloud
1978 and it’s hosted in a Google Cloud
1978 and it’s hosted in a Google Cloud Storage bucket so this code here is just
Storage bucket so this code here is just
Storage bucket so this code here is just downloading that CSV file from the
downloading that CSV file from the
downloading that CSV file from the Google Cloud storage into a couple of
Google Cloud storage into a couple of
Google Cloud storage into a couple of Python variables and memory and here I’m
Python variables and memory and here I’m
Python variables and memory and here I’m just going to print out the first few
just going to print out the first few
just going to print out the first few rows of that data so you can see what it
rows of that data so you can see what it
rows of that data so you can see what it what it actually looks like and this may
what it actually looks like and this may
what it actually looks like and this may be the first live coding stall here so
be the first live coding stall here so
be the first live coding stall here so bear with me
so this is running off of a virtual
so this is running off of a virtual machine in the cloud and that virtual
machine in the cloud and that virtual
machine in the cloud and that virtual machine lost lost his connection
machine lost lost his connection
machine lost lost his connection probably because I was doing the slides
probably because I was doing the slides
probably because I was doing the slides for too long without refreshing this so
for too long without refreshing this so
for too long without refreshing this so I’m just reconnecting here
and while this is coming up what I can
and while this is coming up what I can do is skip to the the second step here
do is skip to the the second step here
do is skip to the the second step here is once you have your raw data loaded so
is once you have your raw data loaded so
is once you have your raw data loaded so in this case a CSV file the next step
in this case a CSV file the next step
in this case a CSV file the next step that you want to do is a data scientist
that you want to do is a data scientist
that you want to do is a data scientist just explore that data understand first
just explore that data understand first
just explore that data understand first of all is my data clean
of all is my data clean
of all is my data clean do I have outliers just do a sanity
do I have outliers just do a sanity
do I have outliers just do a sanity check that the data makes sense and then
check that the data makes sense and then
check that the data makes sense and then second of all do any transformations
second of all do any transformations
second of all do any transformations that I need to do so so this is a tool
that I need to do so so this is a tool
that I need to do so so this is a tool called Google Cloud Data prep and it’s
called Google Cloud Data prep and it’s
called Google Cloud Data prep and it’s nice because right off the bat when you
nice because right off the bat when you
nice because right off the bat when you upload a CSV file it already builds
upload a CSV file it already builds
upload a CSV file it already builds these histograms of all your fields for
these histograms of all your fields for
these histograms of all your fields for you so for example I can see that the
you so for example I can see that the
you so for example I can see that the field here the age of the house the
field here the age of the house the
field here the age of the house the range is between 3 and 100 years old
range is between 3 and 100 years old
range is between 3 and 100 years old that’s how old these houses are and it’s
that’s how old these houses are and it’s
that’s how old these houses are and it’s more concentrated towards the upper end
more concentrated towards the upper end
more concentrated towards the upper end so I have more houses that are old
so I have more houses that are old
so I have more houses that are old closer to the hundred years old then
closer to the hundred years old then
closer to the hundred years old then that are brand new houses and then if I
that are brand new houses and then if I
that are brand new houses and then if I click Edit and then column details I can
click Edit and then column details I can
click Edit and then column details I can get my basic statistics about that
get my basic statistics about that
get my basic statistics about that particular column my main median
particular column my main median
particular column my main median standard deviation and my quartiles and
standard deviation and my quartiles and
standard deviation and my quartiles and things like that and let’s say that I
things like that and let’s say that I
things like that and let’s say that I want to do some manipulation on this
want to do some manipulation on this
want to do some manipulation on this data then I can just choose a
data then I can just choose a
data then I can just choose a transformation here and there’s
transformation here and there’s
transformation here and there’s different things I can do I can do
different things I can do I can do
different things I can do I can do aggregation
aggregation
aggregation I can flatten I can join I can pivot I
I can flatten I can join I can pivot I
I can flatten I can join I can pivot I can do windowing so a lot of your basic
can do windowing so a lot of your basic
can do windowing so a lot of your basic data manipulation here you can do
data manipulation here you can do
data manipulation here you can do through this graphical tool and the nice
through this graphical tool and the nice
through this graphical tool and the nice thing about this is the output of this
thing about this is the output of this
thing about this is the output of this tool is a data flow pipeline and data
tool is a data flow pipeline and data
tool is a data flow pipeline and data flow is an implementation of Apache beam
flow is an implementation of Apache beam
flow is an implementation of Apache beam which is designed to work on large out
which is designed to work on large out
which is designed to work on large out of memory data sets
so let’s say that you know I’ve looked
so let’s say that you know I’ve looked at my data I’m happy with the
at my data I’m happy with the
at my data I’m happy with the transformations then at this point what
transformations then at this point what
transformations then at this point what I would do is I would just export this
I would do is I would just export this
I would do is I would just export this data to a data flow job that would run
data to a data flow job that would run
data to a data flow job that would run the transformations and then I could
the transformations and then I could
the transformations and then I could then pick back up with that data back
then pick back up with that data back
then pick back up with that data back get back in tensor flow and of course
get back in tensor flow and of course
get back in tensor flow and of course this had to happen as I was doing the
this had to happen as I was doing the
this had to happen as I was doing the live demo
so it’s provisioning the Google cloud
so it’s provisioning the Google cloud shell machine which is how I connect to
shell machine which is how I connect to
shell machine which is how I connect to the data lab instance Google cloud shell
the data lab instance Google cloud shell
the data lab instance Google cloud shell is actually a free service so sometimes
is actually a free service so sometimes
is actually a free service so sometimes if there’s a lot of demand in a
if there’s a lot of demand in a
if there’s a lot of demand in a particular region you have to wait a
particular region you have to wait a
particular region you have to wait a little bit to get a machine
so what I can do is hopefully that will
so what I can do is hopefully that will come up or actually it may just be that
come up or actually it may just be that
come up or actually it may just be that my internet connection in general is not
my internet connection in general is not
my internet connection in general is not working right now yeah I think that’s
working right now yeah I think that’s
working right now yeah I think that’s what it is
okay there we go so one of the things
okay there we go so one of the things about working in the cloud is you
about working in the cloud is you
about working in the cloud is you actually need an internet connection for
actually need an internet connection for
actually need an internet connection for it to work okay so let me spin this back
it to work okay so let me spin this back
it to work okay so let me spin this back up now there we go
up now there we go
up now there we go and I’m going to connect back into that
and I’m going to connect back into that
and I’m going to connect back into that instance okay we’ll give that a second
instance okay we’ll give that a second
instance okay we’ll give that a second and while that’s loading up I’m going to
and while that’s loading up I’m going to
and while that’s loading up I’m going to continue on in the github repository
continue on in the github repository
continue on in the github repository here so now that I’ve done I’ve loaded
here so now that I’ve done I’ve loaded
here so now that I’ve done I’ve loaded my raw data I’ve explored it I’m happy
my raw data I’ve explored it I’m happy
my raw data I’ve explored it I’m happy with what it looks like now I can
with what it looks like now I can
with what it looks like now I can actually start writing my tensorflow
actually start writing my tensorflow
actually start writing my tensorflow code so to write my attention flow code
code so to write my attention flow code
code so to write my attention flow code there’s about four or five steps I need
there’s about four or five steps I need
there’s about four or five steps I need to do so the first thing I need to do is
to do so the first thing I need to do is
to do so the first thing I need to do is define my data interface I need to tell
define my data interface I need to tell
define my data interface I need to tell tensorflow how to interpret the
tensorflow how to interpret the
tensorflow how to interpret the different columns within my data set so
different columns within my data set so
different columns within my data set so there’s certain there’s different column
there’s certain there’s different column
there’s certain there’s different column types you have continuous data you have
types you have continuous data you have
types you have continuous data you have categorical data so price for example is
categorical data so price for example is
categorical data so price for example is an example of a continuous variable it
an example of a continuous variable it
an example of a continuous variable it can take any real value department in a
can take any real value department in a
can take any real value department in a university for example would be an
university for example would be an
university for example would be an example of a categorical value so it
example of a categorical value so it
example of a categorical value so it could be one of a discrete set of things
could be one of a discrete set of things
could be one of a discrete set of things math or English or psychology etc and
math or English or psychology etc and
math or English or psychology etc and each of these features are optimized in
each of these features are optimized in
each of these features are optimized in a different way by machine learning
a different way by machine learning
a different way by machine learning algorithms so you have to inform the
algorithms so you have to inform the
algorithms so you have to inform the algorithm what type of what type of data
algorithm what type of what type of data
algorithm what type of what type of data this is and in my particular data set
this is and in my particular data set
this is and in my particular data set it’s pretty easy because all of my all
it’s pretty easy because all of my all
it’s pretty easy because all of my all of my code is all my features are
of my code is all my features are
of my code is all my features are numerical real value numbers so they’re
numerical real value numbers so they’re
numerical real value numbers so they’re all going to be categorical features
so here basically I just have a for loop
so here basically I just have a for loop going through the features that I’m
going through the features that I’m
going through the features that I’m going to use for this and telling you to
going to use for this and telling you to
going to use for this and telling you to treat them all as real valued columns so
treat them all as real valued columns so
treat them all as real valued columns so that’s my data interface now the next
that’s my data interface now the next
that’s my data interface now the next thing that I need to do is pick out my
thing that I need to do is pick out my
thing that I need to do is pick out my actual estimator so my estimator is what
actual estimator so my estimator is what
actual estimator so my estimator is what you could think of is your canned model
and in this case I’m going to use a
and in this case I’m going to use a neural network deep neural network and
neural network deep neural network and
neural network deep neural network and the reason that I’m using that is
the reason that I’m using that is
the reason that I’m using that is because the relationship between my
because the relationship between my
because the relationship between my input features and the actual selling
input features and the actual selling
input features and the actual selling price of a house is nonlinear so I’m not
price of a house is nonlinear so I’m not
price of a house is nonlinear so I’m not going to have a lot of success if I use
going to have a lot of success if I use
going to have a lot of success if I use something like logistic regression where
something like logistic regression where
something like logistic regression where as deep learning and deep learning the
as deep learning and deep learning the
as deep learning and deep learning the most common definition of deep learning
most common definition of deep learning
most common definition of deep learning is just a neural network with two or
is just a neural network with two or
is just a neural network with two or more hidden layers it’s good for
more hidden layers it’s good for
more hidden layers it’s good for modeling that type of relationship so my
modeling that type of relationship so my
modeling that type of relationship so my code for for defining a neural network
code for for defining a neural network
code for for defining a neural network is just this cell here and I’m
is just this cell here and I’m
is just this cell here and I’m specifying that I want two hidden layers
specifying that I want two hidden layers
specifying that I want two hidden layers each with 210 neurons in each layer so
each with 210 neurons in each layer so
each with 210 neurons in each layer so 10 neurons in the first layer and 10
10 neurons in the first layer and 10
10 neurons in the first layer and 10 euros in the second layer and that’s all
euros in the second layer and that’s all
euros in the second layer and that’s all I have to do to define that and I’m
I have to do to define that and I’m
I have to do to define that and I’m passing in my feature definition from
passing in my feature definition from
passing in my feature definition from the previous cell so it knows how to
the previous cell so it knows how to
the previous cell so it knows how to interpret that data so just take a
interpret that data so just take a
interpret that data so just take a second to appreciate that that I could
second to appreciate that that I could
second to appreciate that that I could do that with just a few lines of code
do that with just a few lines of code
do that with just a few lines of code defining a neural network is typically
defining a neural network is typically
defining a neural network is typically something that’s pretty involved to do
something that’s pretty involved to do
something that’s pretty involved to do so that’s the advantage of dealing at
so that’s the advantage of dealing at
so that’s the advantage of dealing at this higher level of abstraction so then
this higher level of abstraction so then
this higher level of abstraction so then I’m going to define my input function so
I’m going to define my input function so
I’m going to define my input function so at this point I’ve just specified a data
at this point I’ve just specified a data
at this point I’ve just specified a data API if you will but I haven’t actually
API if you will but I haven’t actually
API if you will but I haven’t actually passed the data in so this data is
passed the data in so this data is
passed the data in so this data is actually reading from that that pandas
actually reading from that that pandas
actually reading from that that pandas variable that I created earlier which if
variable that I created earlier which if
variable that I created earlier which if you remember just pull that CSV down
you remember just pull that CSV down
you remember just pull that CSV down from Google Cloud Storage and stored
from Google Cloud Storage and stored
from Google Cloud Storage and stored in memory and I’m saying that that’s
in memory and I’m saying that that’s
in memory and I’m saying that that’s where my features exist and my label
where my features exist and my label
where my features exist and my label which is what I’m trying to predict is
which is what I’m trying to predict is
which is what I’m trying to predict is going to be the the feature that I’ve
going to be the the feature that I’ve
going to be the the feature that I’ve identified as label which I did earlier
identified as label which I did earlier
identified as label which I did earlier on here which is median value which is
on here which is median value which is
on here which is median value which is actually a sales price so I’ve got my
actually a sales price so I’ve got my
actually a sales price so I’ve got my features and my labels and I’m passing
features and my labels and I’m passing
features and my labels and I’m passing those is my input function and then you
those is my input function and then you
those is my input function and then you have a separate input function for
have a separate input function for
have a separate input function for serving so serving is just a synonym for
serving so serving is just a synonym for
serving so serving is just a synonym for prediction or for inference so at the
prediction or for inference so at the
prediction or for inference so at the time when I have a user a third party
time when I have a user a third party
time when I have a user a third party user using my prediction service they
user using my prediction service they
user using my prediction service they may pass in data in a slightly different
may pass in data in a slightly different
may pass in data in a slightly different way than how I trained on so that’s why
way than how I trained on so that’s why
way than how I trained on so that’s why I’m allowed to define a separate serving
I’m allowed to define a separate serving
I’m allowed to define a separate serving input function which I do here and I
input function which I do here and I
input function which I do here and I actually am NOT going to pass it in any
actually am NOT going to pass it in any
actually am NOT going to pass it in any differently so the first part of the
differently so the first part of the
differently so the first part of the code is is the same and then there’s
code is is the same and then there’s
code is is the same and then there’s just some boilerplate code here that you
just some boilerplate code here that you
just some boilerplate code here that you don’t modify and the last step is I’m
don’t modify and the last step is I’m
don’t modify and the last step is I’m gonna package all of this into what’s
gonna package all of this into what’s
gonna package all of this into what’s called an experiment and an experiment
called an experiment and an experiment
called an experiment and an experiment is what allows me to actually do this
is what allows me to actually do this
is what allows me to actually do this distributed training without having to
distributed training without having to
distributed training without having to write any additional code and it’s just
write any additional code and it’s just
write any additional code and it’s just a wrapper so I’m just passing in
a wrapper so I’m just passing in
a wrapper so I’m just passing in pointers to all of these previous cells
pointers to all of these previous cells
pointers to all of these previous cells that I’ve defined it so my input
that I’ve defined it so my input
that I’ve defined it so my input function my estimator and then I am
function my estimator and then I am
function my estimator and then I am specifying that I want to train this for
specifying that I want to train this for
specifying that I want to train this for 3,000 steps and then at the end I want
3,000 steps and then at the end I want
3,000 steps and then at the end I want to do an evaluation step to see how how
to do an evaluation step to see how how
to do an evaluation step to see how how good this model is performing against my
good this model is performing against my
good this model is performing against my evaluation data so that’s all the
evaluation data so that’s all the
evaluation data so that’s all the tension flow code and so my code is now
tension flow code and so my code is now
tension flow code and so my code is now written and in order to train this on
written and in order to train this on
written and in order to train this on the cloud I need to package this as a
the cloud I need to package this as a
the cloud I need to package this as a typical Python file so here I’m just
typical Python file so here I’m just
typical Python file so here I’m just taking all these individual cells and
taking all these individual cells and
taking all these individual cells and pasting them into a single cell the code
pasting them into a single cell the code
pasting them into a single cell the code hasn’t changed and then at the end here
hasn’t changed and then at the end here
hasn’t changed and then at the end here there’s some boilerplate code that you
there’s some boilerplate code that you
there’s some boilerplate code that you would just copy and paste that
would just copy and paste that
would just copy and paste that tells up tells this program how to parse
tells up tells this program how to parse
tells up tells this program how to parse certain command-line arguments that the
certain command-line arguments that the
certain command-line arguments that the Google Cloud ml engine xmx so at this
Google Cloud ml engine xmx so at this
Google Cloud ml engine xmx so at this point I can actually train and I’m gonna
point I can actually train and I’m gonna
point I can actually train and I’m gonna train both locally and in the cloud so
train both locally and in the cloud so
train both locally and in the cloud so you can see how both of those work
you can see how both of those work
you can see how both of those work there’s a couple cloud variables that
there’s a couple cloud variables that
there’s a couple cloud variables that I’m defining here so this is the address
I’m defining here so this is the address
I’m defining here so this is the address of my Google Cloud Storage bucket that I
of my Google Cloud Storage bucket that I
of my Google Cloud Storage bucket that I want my model to be written to this is
want my model to be written to this is
want my model to be written to this is the name of my Google Cloud a project
the name of my Google Cloud a project
the name of my Google Cloud a project and this is the geographical region that
and this is the geographical region that
and this is the geographical region that I want my virtual machines to be spun
I want my virtual machines to be spun
I want my virtual machines to be spun upon and I’m just setting those as
upon and I’m just setting those as
upon and I’m just setting those as environment variables so I can reference
environment variables so I can reference
environment variables so I can reference them in the remaining cells and so first
them in the remaining cells and so first
them in the remaining cells and so first I’m gonna run this locally and in this
I’m gonna run this locally and in this
I’m gonna run this locally and in this case locally is actually still on the
case locally is actually still on the
case locally is actually still on the cloud because data lab is a cloud
cloud because data lab is a cloud
cloud because data lab is a cloud virtual machine and you’ll see that that
virtual machine and you’ll see that that
virtual machine and you’ll see that that that finished pretty quickly I have some
that finished pretty quickly I have some
that finished pretty quickly I have some more compilation warnings here which you
more compilation warnings here which you
more compilation warnings here which you can ignore this is just because I didn’t
can ignore this is just because I didn’t
can ignore this is just because I didn’t compile center flow from scratch on this
compile center flow from scratch on this
compile center flow from scratch on this machine so there’s certain optimizations
machine so there’s certain optimizations
machine so there’s certain optimizations it’s letting me know that I can take
it’s letting me know that I can take
it’s letting me know that I can take advantage of if I want to and if I want
advantage of if I want to and if I want
advantage of if I want to and if I want to run this on the cloud then I use a
to run this on the cloud then I use a
to run this on the cloud then I use a command line tool called G cloud and I
command line tool called G cloud and I
command line tool called G cloud and I just specify the the the path of my
just specify the the the path of my
just specify the the the path of my Python file that I had just written here
Python file that I had just written here
Python file that I had just written here to disk so that’s all my code cells just
to disk so that’s all my code cells just
to disk so that’s all my code cells just copied pasted it together and written to
copied pasted it together and written to
copied pasted it together and written to disk I’m writing a pointer to that so it
disk I’m writing a pointer to that so it
disk I’m writing a pointer to that so it knows where the code lives and a I’m
knows where the code lives and a I’m
knows where the code lives and a I’m referencing again my Google Cloud
referencing again my Google Cloud
referencing again my Google Cloud Storage bucket so it knows where to
Storage bucket so it knows where to
Storage bucket so it knows where to output the results to so I’m gonna run
output the results to so I’m gonna run
output the results to so I’m gonna run that and so so this is queued
that and so so this is queued
that and so so this is queued successfully and this is just running on
successfully and this is just running on
successfully and this is just running on one cloud cloud machine learning unit
one cloud cloud machine learning unit
one cloud cloud machine learning unit which is you can think of this is the
which is you can think of this is the
which is you can think of this is the equivalent of a local laptop now here’s
equivalent of a local laptop now here’s
equivalent of a local laptop now here’s the real power of me writing this using
the real power of me writing this using
the real power of me writing this using the estimators API is I can just add a
the estimators API is I can just add a
the estimators API is I can just add a argument here that says – – the scalar
argument here that says – – the scalar
argument here that says – – the scalar equals standard one
equals standard one
equals standard one and now that’s gonna run instead of on
and now that’s gonna run instead of on
and now that’s gonna run instead of on one CPU it’s going to run across ten
one CPU it’s going to run across ten
one CPU it’s going to run across ten CPUs and similarly if I want to run on a
CPUs and similarly if I want to run on a
CPUs and similarly if I want to run on a GPU instead of on a CPU I’ll change this
GPU instead of on a CPU I’ll change this
GPU instead of on a CPU I’ll change this skelter to basic GPU and in terms of
skelter to basic GPU and in terms of
skelter to basic GPU and in terms of pricing the GPU costs three times as
pricing the GPU costs three times as
pricing the GPU costs three times as much as a CPU to run on so as a user
much as a CPU to run on so as a user
much as a CPU to run on so as a user your price performance would be you just
your price performance would be you just
your price performance would be you just want to make sure the GPU speeds up your
want to make sure the GPU speeds up your
want to make sure the GPU speeds up your training at least 3x that’s your
training at least 3x that’s your
training at least 3x that’s your break-even point and so there’s certain
break-even point and so there’s certain
break-even point and so there’s certain algorithms where GPU can actually speed
algorithms where GPU can actually speed
algorithms where GPU can actually speed up your training 4050 X and there’s
up your training 4050 X and there’s
up your training 4050 X and there’s other types of applications where GPU
other types of applications where GPU
other types of applications where GPU isn’t that much faster than the CPU so
isn’t that much faster than the CPU so
isn’t that much faster than the CPU so just be aware of that when you’re
just be aware of that when you’re
just be aware of that when you’re deciding whether trip train on the CPU
deciding whether trip train on the CPU
deciding whether trip train on the CPU or a GPU and then lastly I am going to
or a GPU and then lastly I am going to
or a GPU and then lastly I am going to run across multiple GPUs so the way that
run across multiple GPUs so the way that
run across multiple GPUs so the way that you define this is a little bit
you define this is a little bit
you define this is a little bit different I create a yamo file where I
different I create a yamo file where I
different I create a yamo file where I where I specify my machine type and then
where I specify my machine type and then
where I specify my machine type and then I referenced that Yambol file as a
I referenced that Yambol file as a
I referenced that Yambol file as a command-line argument so it’s still not
command-line argument so it’s still not
command-line argument so it’s still not that complicated but it’s just a little
that complicated but it’s just a little
that complicated but it’s just a little bit different and so now I’m running
bit different and so now I’m running
bit different and so now I’m running across eight cloud GPUs and actually in
across eight cloud GPUs and actually in
across eight cloud GPUs and actually in this case those are nvidia tesla k 80s
this case those are nvidia tesla k 80s
this case those are nvidia tesla k 80s so i’m running across eight of those in
so i’m running across eight of those in
so i’m running across eight of those in parallel so if i go back to the cloud
parallel so if i go back to the cloud
parallel so if i go back to the cloud here and click on ml engine jobs you’ll
here and click on ml engine jobs you’ll
here and click on ml engine jobs you’ll see all those four jobs that i just
see all those four jobs that i just
see all those four jobs that i just kicked off then they’re spinning and you
kicked off then they’re spinning and you
kicked off then they’re spinning and you might be asking why this is still
might be asking why this is still
might be asking why this is still running if the one I ran locally
running if the one I ran locally
running if the one I ran locally finished pretty much instantly and the
finished pretty much instantly and the
finished pretty much instantly and the reason for that is when you’re running
reason for that is when you’re running
reason for that is when you’re running in the cloud it’s starting up the
in the cloud it’s starting up the
in the cloud it’s starting up the virtual machine from scratch so the
virtual machine from scratch so the
virtual machine from scratch so the startup time for that is about five
startup time for that is about five
startup time for that is about five minutes and then it runs the job so you
minutes and then it runs the job so you
minutes and then it runs the job so you wouldn’t actually want to run this
wouldn’t actually want to run this
wouldn’t actually want to run this particular job in the cloud because it’s
particular job in the cloud because it’s
particular job in the cloud because it’s such a small data set but when you’re
such a small data set but when you’re
such a small data set but when you’re thinking of larger data sets where it’s
thinking of larger data sets where it’s
thinking of larger data sets where it’s going to take hours or even days to run
going to take hours or even days to run
going to take hours or even days to run then that five minutes becomes pretty
then that five minutes becomes pretty
then that five minutes becomes pretty insignificant in the context of the
insignificant in the context of the
insignificant in the context of the horizontal scaling that you go
horizontal scaling that you go
horizontal scaling that you go and in terms of the the logging for any
and in terms of the the logging for any
and in terms of the the logging for any job as it’s running you can just click
job as it’s running you can just click
job as it’s running you can just click view logs here and it’ll give you a
view logs here and it’ll give you a
view logs here and it’ll give you a running output of of the logs so here it
running output of of the logs so here it
running output of of the logs so here it says waiting for job to be provisioned
says waiting for job to be provisioned
says waiting for job to be provisioned it’s still spinning up the VM and then
it’s still spinning up the VM and then
it’s still spinning up the VM and then when the job is done we can deploy that
when the job is done we can deploy that
when the job is done we can deploy that well we can inspect our results so I’m
well we can inspect our results so I’m
well we can inspect our results so I’m gonna go back into my data lab notebook
gonna go back into my data lab notebook
gonna go back into my data lab notebook here and I’m gonna inspect the results
here and I’m gonna inspect the results
here and I’m gonna inspect the results of the the job that I ran locally since
of the the job that I ran locally since
of the the job that I ran locally since that one’s already done and we’re going
that one’s already done and we’re going
that one’s already done and we’re going to inspect the results using a tool
to inspect the results using a tool
to inspect the results using a tool called tensor board
and so remember this review at this
and so remember this review at this point we’ve started with the raw data
point we’ve started with the raw data
point we’ve started with the raw data set we’ve done some exploration of that
set we’ve done some exploration of that
set we’ve done some exploration of that data we then wrote our tensorflow code
data we then wrote our tensorflow code
data we then wrote our tensorflow code to define how to interpret the specific
to define how to interpret the specific
to define how to interpret the specific columns of that data and also what type
columns of that data and also what type
columns of that data and also what type of algorithm that we wanted to use for
of algorithm that we wanted to use for
of algorithm that we wanted to use for training so in this case we used a two
training so in this case we used a two
training so in this case we used a two layer neural network and now we want to
layer neural network and now we want to
layer neural network and now we want to investigate the results of that training
investigate the results of that training
investigate the results of that training did it actually learn anything is it
did it actually learn anything is it
did it actually learn anything is it doing anything useful so here I’m gonna
doing anything useful so here I’m gonna
doing anything useful so here I’m gonna click on my loss diagram here and I’m
click on my loss diagram here and I’m
click on my loss diagram here and I’m gonna expand this and basically what
gonna expand this and basically what
gonna expand this and basically what you’re looking for here is that the this
you’re looking for here is that the this
you’re looking for here is that the this line goes down and to the right that
line goes down and to the right that
line goes down and to the right that means it’s learning something over time
means it’s learning something over time
means it’s learning something over time and you can see here’s my training step
and you can see here’s my training step
and you can see here’s my training step here so I have my 3,000 training steps
here so I have my 3,000 training steps
here so I have my 3,000 training steps and then I’m running one evaluation step
and then I’m running one evaluation step
and then I’m running one evaluation step at the end of it
at the end of it
at the end of it so just to do a sanity check here my
so just to do a sanity check here my
so just to do a sanity check here my evaluation step at the end is saying a
evaluation step at the end is saying a
evaluation step at the end is saying a loss of six point four times ten raise
loss of six point four times ten raise
loss of six point four times ten raise to seven which might seem like a lot but
to seven which might seem like a lot but
to seven which might seem like a lot but this is actually squared error that’s
this is actually squared error that’s
this is actually squared error that’s how these that’s how this is measured so
how these that’s how this is measured so
how these that’s how this is measured so if I just do some quick math here so I’m
if I just do some quick math here so I’m
if I just do some quick math here so I’m gonna do the square root of six point
gonna do the square root of six point
gonna do the square root of six point four six followed by or let’s just do
four six followed by or let’s just do
four six followed by or let’s just do six for six and then one two three four
six for six and then one two three four
six for six and then one two three four five zeroes so that’s that’s that same
five zeroes so that’s that’s that same
five zeroes so that’s that’s that same number and so that comes to about 8,000
number and so that comes to about 8,000
number and so that comes to about 8,000 so if I put this back in the in my
so if I put this back in the in my
so if I put this back in the in my original problem that means it’s
original problem that means it’s
original problem that means it’s predicting the selling price of a house
predicting the selling price of a house
predicting the selling price of a house with an average error of about eight
with an average error of about eight
with an average error of about eight thousand dollars which which is pretty
thousand dollars which which is pretty
thousand dollars which which is pretty good maybe it could be better but for
good maybe it could be better but for
good maybe it could be better but for any of you who ever sold a house or
any of you who ever sold a house or
any of you who ever sold a house or bought a house if you could know ahead
bought a house if you could know ahead
bought a house if you could know ahead of time within a thousand dollars what
of time within a thousand dollars what
of time within a thousand dollars what it was going to sell for you’d be pretty
it was going to sell for you’d be pretty
it was going to sell for you’d be pretty happy with that
happy with that
happy with that so I’ve inspected my results I am now
so I’ve inspected my results I am now
so I’ve inspected my results I am now happy with it and
happy with it and
happy with it and I’m going to now deploy my model for
I’m going to now deploy my model for
I’m going to now deploy my model for actually for for prediction so to do
actually for for prediction so to do
actually for for prediction so to do that there’s another g-cloud command
that there’s another g-cloud command
that there’s another g-cloud command here and I’m just again pointing to the
here and I’m just again pointing to the
here and I’m just again pointing to the to the model file that was created and
to the model file that was created and
to the model file that was created and what this is doing now is it’s pushing
what this is doing now is it’s pushing
what this is doing now is it’s pushing my model central model file to the cloud
my model central model file to the cloud
my model central model file to the cloud and it’s spinning up some infrastructure
and it’s spinning up some infrastructure
and it’s spinning up some infrastructure to actually back that and this is all
to actually back that and this is all
to actually back that and this is all just killing infrastructure so let’s say
just killing infrastructure so let’s say
just killing infrastructure so let’s say today that I only get you know let’s say
today that I only get you know let’s say
today that I only get you know let’s say I’ve just launched this API and I don’t
I’ve just launched this API and I don’t
I’ve just launched this API and I don’t have any users yet so I’m only getting
have any users yet so I’m only getting
have any users yet so I’m only getting maybe ten requests per day then it’s
maybe ten requests per day then it’s
maybe ten requests per day then it’s gonna have the minimal VM to back that
gonna have the minimal VM to back that
gonna have the minimal VM to back that but all of a sudden if that API goes
but all of a sudden if that API goes
but all of a sudden if that API goes viral and I’m getting thousands of
viral and I’m getting thousands of
viral and I’m getting thousands of requests per second then this service
requests per second then this service
requests per second then this service will just spin up more VMs in the
will just spin up more VMs in the
will just spin up more VMs in the background to handle those requests so
background to handle those requests so
background to handle those requests so it’s it’s a cloud native fully auto
it’s it’s a cloud native fully auto
it’s it’s a cloud native fully auto scaling service in that sense and I can
scaling service in that sense and I can
scaling service in that sense and I can see here if I go to my models section
see here if I go to my models section
see here if I go to my models section and then click on housing prices that is
and then click on housing prices that is
and then click on housing prices that is creating that model and this will take
creating that model and this will take
creating that model and this will take just about probably 15 more seconds
just about probably 15 more seconds
just about probably 15 more seconds because they’re just spinning up that VM
because they’re just spinning up that VM
because they’re just spinning up that VM to back it and then once that’s done I’m
to back it and then once that’s done I’m
to back it and then once that’s done I’m gonna actually test it out so I’m gonna
gonna actually test it out so I’m gonna
gonna actually test it out so I’m gonna write out a a couple of new records here
write out a a couple of new records here
write out a a couple of new records here for a prediction so I have all my
for a prediction so I have all my
for a prediction so I have all my predictive features here the crime rate
predictive features here the crime rate
predictive features here the crime rate the age of the house the
the age of the house the
the age of the house the student-to-teacher ratio in the in that
student-to-teacher ratio in the in that
student-to-teacher ratio in the in that area and I’m gonna write those out to a
area and I’m gonna write those out to a
area and I’m gonna write those out to a file and then I’m just gonna pass that
file and then I’m just gonna pass that
file and then I’m just gonna pass that file to the service and then here I get
file to the service and then here I get
file to the service and then here I get my actual predictions back so 24
my actual predictions back so 24
my actual predictions back so 24 thousand dollars and seventy thousand
thousand dollars and seventy thousand
thousand dollars and seventy thousand dollars respectively and keep in mind
dollars respectively and keep in mind
dollars respectively and keep in mind this data set is from 1978 so those
this data set is from 1978 so those
this data set is from 1978 so those those housing prices are actually in US
those housing prices are actually in US
those housing prices are actually in US dollars are passed the sanity check
dollars are passed the sanity check
dollars are passed the sanity check so just to review what we covered here
so just to review what we covered here
so just to review what we covered here is how to use tensor flows high-level
is how to use tensor flows high-level
is how to use tensor flows high-level estimator a
estimator a
estimator a p.i how to deploy it for distributed
p.i how to deploy it for distributed
p.i how to deploy it for distributed training on the cloud how to evaluate
training on the cloud how to evaluate
training on the cloud how to evaluate those results using the tool called
those results using the tool called
those results using the tool called tensor board and then how to deploy that
tensor board and then how to deploy that
tensor board and then how to deploy that resulting model to the cloud for online
resulting model to the cloud for online
resulting model to the cloud for online prediction and I also want to point out
prediction and I also want to point out
prediction and I also want to point out some important things that we didn’t
some important things that we didn’t
some important things that we didn’t cover so in this case we were using a
cover so in this case we were using a
cover so in this case we were using a data set that fit in memory and if you
data set that fit in memory and if you
data set that fit in memory and if you wanted to use a data set that was larger
wanted to use a data set that was larger
wanted to use a data set that was larger than memory or a quote-unquote big data
than memory or a quote-unquote big data
than memory or a quote-unquote big data set there’s some other things you would
set there’s some other things you would
set there’s some other things you would have to take into account we didn’t do
have to take into account we didn’t do
have to take into account we didn’t do any true feature engineering we just
any true feature engineering we just
any true feature engineering we just passed our CSV features as in without
passed our CSV features as in without
passed our CSV features as in without doing transformations because in this
doing transformations because in this
doing transformations because in this case we didn’t need to but in most cases
case we didn’t need to but in most cases
case we didn’t need to but in most cases you will need to and then lastly we
you will need to and then lastly we
you will need to and then lastly we didn’t do any automatic hyper parameter
didn’t do any automatic hyper parameter
didn’t do any automatic hyper parameter tuning which in a real data sense data
tuning which in a real data sense data
tuning which in a real data sense data science environment you would normally
science environment you would normally
science environment you would normally do that if you would like to learn how
do that if you would like to learn how
do that if you would like to learn how to do those things the last thing that I
to do those things the last thing that I
to do those things the last thing that I will plug here is there is a great
will plug here is there is a great
will plug here is there is a great course on Coursera that we release
course on Coursera that we release
course on Coursera that we release recently that covers how to do all those
recently that covers how to do all those
recently that covers how to do all those things
things
things it’s a one-week course it goes through a
it’s a one-week course it goes through a
it’s a one-week course it goes through a case study of predicting taxi cab prices
case study of predicting taxi cab prices
case study of predicting taxi cab prices in New York City and it’ll do kind of
in New York City and it’ll do kind of
in New York City and it’ll do kind of even a level deeper than we did today so
even a level deeper than we did today so
even a level deeper than we did today so so yeah apologies for losing the
so yeah apologies for losing the
so yeah apologies for losing the internet connection and not realizing I
internet connection and not realizing I
internet connection and not realizing I lost it for a couple minutes there but
lost it for a couple minutes there but
lost it for a couple minutes there but thank you for bearing with me and I have
thank you for bearing with me and I have
thank you for bearing with me and I have technically 50 seconds for questions but
technically 50 seconds for questions but
technically 50 seconds for questions but it you know if we have any I’m happy to
it you know if we have any I’m happy to
it you know if we have any I’m happy to take them and there’s one here
when collecting more data continuously
when collecting more data continuously how do you use this data to
how do you use this data to
how do you use this data to automatically retrain your model that’s
automatically retrain your model that’s
automatically retrain your model that’s the question yeah so that’s a good
the question yeah so that’s a good
the question yeah so that’s a good question and that’s a question that’s
question and that’s a question that’s
question and that’s a question that’s very relevant for certain types of
very relevant for certain types of
very relevant for certain types of problems like recommendation engines
problems like recommendation engines
problems like recommendation engines where you are getting new information
where you are getting new information
where you are getting new information about your environment and you want to
about your environment and you want to
about your environment and you want to retrain on a daily basis maybe even an
retrain on a daily basis maybe even an
retrain on a daily basis maybe even an hourly basis there’s other types of
hourly basis there’s other types of
hourly basis there’s other types of problems where what you know about the
problems where what you know about the
problems where what you know about the world doesn’t change for maybe six
world doesn’t change for maybe six
world doesn’t change for maybe six months or years at a time so you don’t
months or years at a time so you don’t
months or years at a time so you don’t have to worry about that the answer is
have to worry about that the answer is
have to worry about that the answer is you just have to have a orchestration
you just have to have a orchestration
you just have to have a orchestration plan in place so you need to have some
plan in place so you need to have some
plan in place so you need to have some type of messaging system that alerts you
type of messaging system that alerts you
type of messaging system that alerts you when there’s new data to return to train
when there’s new data to return to train
when there’s new data to return to train on and then that kicks off a new
on and then that kicks off a new
on and then that kicks off a new training job and when that you training
training job and when that you training
training job and when that you training job is kicked off your second decision
job is kicked off your second decision
job is kicked off your second decision point is how do I combine the results of
point is how do I combine the results of
point is how do I combine the results of that new training job with my old
that new training job with my old
that new training job with my old training job so I might just replacing
training job so I might just replacing
training job so I might just replacing it the old model completely with my new
it the old model completely with my new
it the old model completely with my new training data or maybe I want to do a
training data or maybe I want to do a
training data or maybe I want to do a sliding window where let’s take the
sliding window where let’s take the
sliding window where let’s take the example where I’m training on a daily
example where I’m training on a daily
example where I’m training on a daily basis I want to take today’s data and
basis I want to take today’s data and
basis I want to take today’s data and combine it with a sliding of the last
combine it with a sliding of the last
combine it with a sliding of the last seven days data that I trained on or
seven days data that I trained on or
seven days data that I trained on or maybe what I want to do is just an
maybe what I want to do is just an
maybe what I want to do is just an incumbent versus Challenger scenario
incumbent versus Challenger scenario
incumbent versus Challenger scenario where I take today’s data that I trained
where I take today’s data that I trained
where I take today’s data that I trained on evaluate the model against my success
on evaluate the model against my success
on evaluate the model against my success metric and see if it does better or
metric and see if it does better or
metric and see if it does better or worse than my incumbent model and if it
worse than my incumbent model and if it
worse than my incumbent model and if it does better I promote that to production
does better I promote that to production
does better I promote that to production or if it doesn’t do better I stay with
or if it doesn’t do better I stay with
or if it doesn’t do better I stay with the incumbent model so there’s a couple
the incumbent model so there’s a couple
the incumbent model so there’s a couple different ways that you can do that
different ways that you can do that
different ways that you can do that we’re working on some managed services
we’re working on some managed services
we’re working on some managed services coming out to make that orchestration
coming out to make that orchestration
coming out to make that orchestration piece even easier right now you have to
piece even easier right now you have to
piece even easier right now you have to do it a bit more piecemeal but it
do it a bit more piecemeal but it
do it a bit more piecemeal but it basically it’s a very similar to problem
basically it’s a very similar to problem
basically it’s a very similar to problem to the kind of continuous integration
to the kind of continuous integration
to the kind of continuous integration continual continuous deployment tools in
continual continuous deployment tools in
continual continuous deployment tools in the non machine learning sector ok we’re
the non machine learning sector ok we’re
the non machine learning sector ok we’re out of time and thank you very much
out of time and thank you very much
out of time and thank you very much speaking thank you
speaking thank you
speaking thank you you
”
Be First to Comment