GOTO 2015 • Modern Fraud Prevention using Deep Learning • Phil Winder

hello everyone.welcome this is modern

hello everyone.welcome this is modern fraud detection prevention using deep

fraud detection prevention using deep

fraud detection prevention using deep learning that title was submitted quite

learning that title was submitted quite

learning that title was submitted quite a long time ago so I’d say actually the

a long time ago so I’d say actually the

a long time ago so I’d say actually the talk is probably bit more about machine

talk is probably bit more about machine

talk is probably bit more about machine learning in general now we had a good

learning in general now we had a good

learning in general now we had a good talk earlier on in the day introducing

talk earlier on in the day introducing

talk earlier on in the day introducing some of the concepts we time it behind

some of the concepts we time it behind

some of the concepts we time it behind learning and I’m hoping to sort of build

learning and I’m hoping to sort of build

learning and I’m hoping to sort of build on them really so this talk is going to

on them really so this talk is going to

on them really so this talk is going to be a bit more technical there’s there’s

be a bit more technical there’s there’s

be a bit more technical there’s there’s no maths which you’ll be glad to hear

no maths which you’ll be glad to hear

no maths which you’ll be glad to hear there’s there’s also no code I’ve tried

there’s there’s also no code I’ve tried

there’s there’s also no code I’ve tried to you know explain myself using

to you know explain myself using

to you know explain myself using diagrams and pictures wherever I can but

diagrams and pictures wherever I can but

diagrams and pictures wherever I can but it is a much more technical talk so

it is a much more technical talk so

it is a much more technical talk so hopefully you can get you your teeth

hopefully you can get you your teeth

hopefully you can get you your teeth into it we’ve got the usual slides at

into it we’ve got the usual slides at

into it we’ve got the usual slides at the front saying please rate and engage

the front saying please rate and engage

the front saying please rate and engage so yeah my name’s Phil I’m with try fork

so yeah my name’s Phil I’m with try fork

so yeah my name’s Phil I’m with try fork but we’re in try fot leads so we’re

but we’re in try fot leads so we’re

but we’re in try fot leads so we’re quite distinct from the Danish

quite distinct from the Danish

quite distinct from the Danish mothership yeah I actually am a software

mothership yeah I actually am a software

mothership yeah I actually am a software engineer in my in my professional life

engineer in my in my professional life

engineer in my in my professional life machine learning is a just a bit more of

machine learning is a just a bit more of

machine learning is a just a bit more of a hobby I’m currently working on there

a hobby I’m currently working on there

a hobby I’m currently working on there and Apache meat sauce framework for

and Apache meat sauce framework for

and Apache meat sauce framework for elasticsearch yeah if you’d if you’d

elasticsearch yeah if you’d if you’d

elasticsearch yeah if you’d if you’d like to talk more about any of the

like to talk more about any of the

like to talk more about any of the subjects that I’m about to discuss then

subjects that I’m about to discuss then

subjects that I’m about to discuss then please see me or I’ll see some of my

please see me or I’ll see some of my

please see me or I’ll see some of my colleagues listed at the bottom there

colleagues listed at the bottom there

colleagues listed at the bottom there I’m gonna skip the marketing slides

I’m gonna skip the marketing slides

I’m gonna skip the marketing slides because you will not try and we’re

because you will not try and we’re

because you will not try and we’re split into three or four topics the the

split into three or four topics the the

split into three or four topics the the final one the architectures that is is

final one the architectures that is is

final one the architectures that is is more about how we would do this in

more about how we would do this in

more about how we would do this in production how we would do this in real

production how we would do this in real

production how we would do this in real life it’s it’s interesting but it’s not

life it’s it’s interesting but it’s not

life it’s it’s interesting but it’s not really the core thing of my my talk so

really the core thing of my my talk so

really the core thing of my my talk so I’m going to go through the first three

I’m going to go through the first three

I’m going to go through the first three sections and if we have time we might do

sections and if we have time we might do

sections and if we have time we might do the fourth but I’ll probably end up

the fourth but I’ll probably end up

the fourth but I’ll probably end up speaking for too long and I’ll probably

speaking for too long and I’ll probably

speaking for too long and I’ll probably drop that section I’m going to introduce

drop that section I’m going to introduce

drop that section I’m going to introduce the the reasons why we want to provide

the the reasons why we want to provide

the the reasons why we want to provide some new tools and techniques to apply

some new tools and techniques to apply

some new tools and techniques to apply to fraud to try and make the case to the

to fraud to try and make the case to the

to fraud to try and make the case to the business users as why you should pick up

business users as why you should pick up

business users as why you should pick up on some of these ideas and start

on some of these ideas and start

on some of these ideas and start to run with them I’m gonna then

to run with them I’m gonna then

to run with them I’m gonna then introduce the topic of machine learning

introduce the topic of machine learning

introduce the topic of machine learning and you’ve probably had quite a bit of

and you’ve probably had quite a bit of

and you’ve probably had quite a bit of experience already but if you haven’t

experience already but if you haven’t

experience already but if you haven’t that that’ll be the section that really

that that’ll be the section that really

that that’ll be the section that really explains what’s going on and and why it

explains what’s going on and and why it

explains what’s going on and and why it happens and I’ve also got quite a lot of

happens and I’ve also got quite a lot of

happens and I’ve also got quite a lot of demos as well some of the demos are

demos as well some of the demos are

demos as well some of the demos are quite simple and very general just to

quite simple and very general just to

quite simple and very general just to explain the concepts but the rest of the

explain the concepts but the rest of the

explain the concepts but the rest of the demos are all focused towards fraud

demos are all focused towards fraud

demos are all focused towards fraud prevention focus towards finance and

prevention focus towards finance and

prevention focus towards finance and specifically mortgages okay so let’s

specifically mortgages okay so let’s

specifically mortgages okay so let’s crack on so in order to to do any of

crack on so in order to to do any of

crack on so in order to to do any of this work we need to persuade some

this work we need to persuade some

this work we need to persuade some people to give us some money and there’s

people to give us some money and there’s

people to give us some money and there’s no better reason to get people to give

no better reason to get people to give

no better reason to get people to give us some money if there’s other money at

us some money if there’s other money at

us some money if there’s other money at risk in the UK we’ve got some UK

risk in the UK we’ve got some UK

risk in the UK we’ve got some UK specific facts here in the UK financial

specific facts here in the UK financial

specific facts here in the UK financial crime is defined as I can’t even read

crime is defined as I can’t even read

crime is defined as I can’t even read that screen so I have to read from here

that screen so I have to read from here

that screen so I have to read from here sorry fraud is an act of deception

sorry fraud is an act of deception

sorry fraud is an act of deception intended for personal gain or to cause a

intended for personal gain or to cause a

intended for personal gain or to cause a loss to another party so all of these

loss to another party so all of these

loss to another party so all of these facts and figures it specific to the UK

facts and figures it specific to the UK

facts and figures it specific to the UK but they’re they’re applicable to pretty

but they’re they’re applicable to pretty

but they’re they’re applicable to pretty much every country in the world anybody

much every country in the world anybody

much every country in the world anybody that’s trying to do wrong to do harm for

that’s trying to do wrong to do harm for

that’s trying to do wrong to do harm for their own financial gain is considered

their own financial gain is considered

their own financial gain is considered fraud we’ve got a UK mortgage fraud

fraud we’ve got a UK mortgage fraud

fraud we’ve got a UK mortgage fraud listed there in 2014 a 1.2 million

listed there in 2014 a 1.2 million

listed there in 2014 a 1.2 million properties bought and sold in the UK and

properties bought and sold in the UK and

properties bought and sold in the UK and 83 in every 10,000 of those applications

83 in every 10,000 of those applications

83 in every 10,000 of those applications were fraudulent so that’s not quite 1%

were fraudulent so that’s not quite 1%

were fraudulent so that’s not quite 1% 0.8 3% and when we say when you say

0.8 3% and when we say when you say

0.8 3% and when we say when you say fraud in that that aspect it’s not

fraud in that that aspect it’s not

fraud in that that aspect it’s not necessarily people being like hugely

necessarily people being like hugely

necessarily people being like hugely devious we’re going from the small scale

devious we’re going from the small scale

devious we’re going from the small scale where somebody’s maybe telling a few

where somebody’s maybe telling a few

where somebody’s maybe telling a few fibs about their employment history or

fibs about their employment history or

fibs about their employment history or how much they earn all the way up to

how much they earn all the way up to

how much they earn all the way up to huge huge you know international fraud

huge huge you know international fraud

huge huge you know international fraud in 2013 there was a story of two guys

in 2013 there was a story of two guys

in 2013 there was a story of two guys who had invented there a whole series of

who had invented there a whole series of

who had invented there a whole series of companies that invented estate agents

companies that invented estate agents

companies that invented estate agents that invented surveyors they’ve invented

that invented surveyors they’ve invented

that invented surveyors they’ve invented property businesses and builders and

property businesses and builders and

property businesses and builders and they had supposedly bought a huge tract

they had supposedly bought a huge tract

they had supposedly bought a huge tract of land which they were going to build

of land which they were going to build

of land which they were going to build you know lots of new houses on the

you know lots of new houses on the

you know lots of new houses on the invented or stole the identities of

invented or stole the identities of

invented or stole the identities of other people to take out mortgages on

other people to take out mortgages on

other people to take out mortgages on those respective houses so it turns out

those respective houses so it turns out

those respective houses so it turns out there were tens you know tens to

there were tens you know tens to

there were tens you know tens to hundreds of mortgage applications all

hundreds of mortgage applications all

hundreds of mortgage applications all going in for houses that hadn’t been

going in for houses that hadn’t been

going in for houses that hadn’t been built yet

built yet

built yet but as it turned out they just took that

but as it turned out they just took that

but as it turned out they just took that money paid off the original land the

money paid off the original land the

money paid off the original land the original debt they worked they owned the

original debt they worked they owned the

original debt they worked they owned the land and then just liked it they just

land and then just liked it they just

land and then just liked it they just ran off they completely invented a

ran off they completely invented a

ran off they completely invented a village bought loads and mortgages based

village bought loads and mortgages based

village bought loads and mortgages based upon that and then ran off how how can

upon that and then ran off how how can

upon that and then ran off how how can that even so that the total cost finally

that even so that the total cost finally

that even so that the total cost finally came to it was about 53 million pounds

came to it was about 53 million pounds

came to it was about 53 million pounds and managed to to run away with and they

and managed to to run away with and they

and managed to to run away with and they did finally get caught but they very

did finally get caught but they very

did finally get caught but they very nearly got away with it because it was

nearly got away with it because it was

nearly got away with it because it was just so embarrassing

just so embarrassing

just so embarrassing you know the mortgage company was so

you know the mortgage company was so

you know the mortgage company was so embarrassed to say that this had

embarrassed to say that this had

embarrassed to say that this had happened it almost never even got caught

happened it almost never even got caught

happened it almost never even got caught so it does it does get to quite a large

so it does it does get to quite a large

so it does it does get to quite a large scale and this this actually equates to

scale and this this actually equates to

scale and this this actually equates to approximately 1 billion pounds worth of

approximately 1 billion pounds worth of

approximately 1 billion pounds worth of fraudulent applications so it’s a huge

fraudulent applications so it’s a huge

fraudulent applications so it’s a huge huge number but my interestingly it’s

huge number but my interestingly it’s

huge number but my interestingly it’s not actually the worst case of fraud in

not actually the worst case of fraud in

not actually the worst case of fraud in the UK the worst is actually credit

the UK the worst is actually credit

the UK the worst is actually credit current account fraud so traditionally

current account fraud so traditionally

current account fraud so traditionally what would what people would do is to

what would what people would do is to

what would what people would do is to steal somebody’s information open a

steal somebody’s information open a

steal somebody’s information open a standard bank account current account

standard bank account current account

standard bank account current account some sort from from a traditional bank

some sort from from a traditional bank

some sort from from a traditional bank which you can do quite easily in the UK

which you can do quite easily in the UK

which you can do quite easily in the UK and then use the overdraft or use some

and then use the overdraft or use some

and then use the overdraft or use some facilities to actually withdraw some

facilities to actually withdraw some

facilities to actually withdraw some money and then and then run a runoff

money and then and then run a runoff

money and then and then run a runoff so that actually constitutes the most

so that actually constitutes the most

so that actually constitutes the most fraud in the UK but we’re talking a

fraud in the UK but we’re talking a

fraud in the UK but we’re talking a little bit about mortgages today and

little bit about mortgages today and

little bit about mortgages today and finally we’ve got UK real retail fraud

finally we’ve got UK real retail fraud

finally we’ve got UK real retail fraud much of the business in the UK is

much of the business in the UK is

much of the business in the UK is actually made up of small to medium

actually made up of small to medium

actually made up of small to medium sized enterprise

sized enterprise

sized enterprise the big guys actually don’t they make a

the big guys actually don’t they make a

the big guys actually don’t they make a significant part of the market but not

significant part of the market but not

significant part of the market but not not a huge part small to medium-sized

not a huge part small to medium-sized

not a huge part small to medium-sized businesses they’re estimated losing

businesses they’re estimated losing

businesses they’re estimated losing eighteen billion pounds every year to

eighteen billion pounds every year to

eighteen billion pounds every year to fraudulent transactions so that’s when

fraudulent transactions so that’s when

fraudulent transactions so that’s when somebody goes online buy some clothes or

somebody goes online buy some clothes or

somebody goes online buy some clothes or buy some food or buy some shopping of

buy some food or buy some shopping of

buy some food or buy some shopping of some kind on a credit card and then

some kind on a credit card and then

some kind on a credit card and then maybe they cancel a credit card as soon

maybe they cancel a credit card as soon

maybe they cancel a credit card as soon as I place the order so the the guys on

as I place the order so the the guys on

as I place the order so the the guys on the retail side of having to ship all of

the retail side of having to ship all of

the retail side of having to ship all of this stuff only to find that the person

this stuff only to find that the person

this stuff only to find that the person you know doesn’t exist or

you know doesn’t exist or

you know doesn’t exist or card stolen or stuff like that and that

card stolen or stuff like that and that

card stolen or stuff like that and that amounts to a huge amount as well another

amounts to a huge amount as well another

amounts to a huge amount as well another reason why businesses might want to look

reason why businesses might want to look

reason why businesses might want to look at some of these ideas is that

at some of these ideas is that

at some of these ideas is that legislation so we’ve got one end of the

legislation so we’ve got one end of the

legislation so we’ve got one end of the spectrum where there’s people actually

spectrum where there’s people actually

spectrum where there’s people actually doing wrong to their businesses you

doing wrong to their businesses you

doing wrong to their businesses you might want to try and protect yourself

might want to try and protect yourself

might want to try and protect yourself but also this legislation legal

but also this legislation legal

but also this legislation legal requirements that need to be put in

requirements that need to be put in

requirements that need to be put in place in order to comply two more in

place in order to comply two more in

place in order to comply two more in 2017 there’s new anti money laundering

2017 there’s new anti money laundering

2017 there’s new anti money laundering legislation coming in within the EU so

legislation coming in within the EU so

legislation coming in within the EU so it applies to all EU countries it’s

it applies to all EU countries it’s

it applies to all EU countries it’s extending extending money laundering

extending extending money laundering

extending extending money laundering rules that are already in place but the

rules that are already in place but the

rules that are already in place but the main changes are that the out of scope

main changes are that the out of scope

main changes are that the out of scope limit has dropped to a thousand euros

limit has dropped to a thousand euros

limit has dropped to a thousand euros the previously it was fifteen thousand

the previously it was fifteen thousand

the previously it was fifteen thousand euros and this applies to businesses

euros and this applies to businesses

euros and this applies to businesses that are handling financial transactions

that are handling financial transactions

that are handling financial transactions so it applies to banks obviously

so it applies to banks obviously

so it applies to banks obviously financial institutions credit agencies

financial institutions credit agencies

financial institutions credit agencies stuff like that it also applies to legal

stuff like that it also applies to legal

stuff like that it also applies to legal services in the state services it also

services in the state services it also

services in the state services it also applies to to gambling services

applies to to gambling services

applies to to gambling services basically anybody that’s handling and

basically anybody that’s handling and

basically anybody that’s handling and moving money around has to comply with

moving money around has to comply with

moving money around has to comply with this legislation and what this is saying

this legislation and what this is saying

this legislation and what this is saying is that anybody that has a transaction

is that anybody that has a transaction

is that anybody that has a transaction of over a thousand euros they need to

of over a thousand euros they need to

of over a thousand euros they need to prove to the authorities that they’re

prove to the authorities that they’re

prove to the authorities that they’re doing their due diligence in to prove

doing their due diligence in to prove

doing their due diligence in to prove that that person is a not being

that that person is a not being

that that person is a not being fraudulent and be not using the money

fraudulent and be not using the money

fraudulent and be not using the money for nefarious means like terrorism or

for nefarious means like terrorism or

for nefarious means like terrorism or something like that and finally then

something like that and finally then

something like that and finally then they’re they’re required to submit their

they’re they’re required to submit their

they’re they’re required to submit their information to a central registry of

information to a central registry of

information to a central registry of information and this there’s a well

information and this there’s a well

information and this there’s a well there’s obviously previously concerns

there’s obviously previously concerns

there’s obviously previously concerns there but that’s a bit unclear and how

there but that’s a bit unclear and how

there but that’s a bit unclear and how that’s actually going to be implemented

that’s actually going to be implemented

that’s actually going to be implemented so there’s financial reasons direct

so there’s financial reasons direct

so there’s financial reasons direct financial reasons why you want might you

financial reasons why you want might you

financial reasons why you want might you might want to do it is also legal

might want to do it is also legal

might want to do it is also legal reasons so how do we do it at the moment

reasons so how do we do it at the moment

reasons so how do we do it at the moment well if a traditional company was goat

well if a traditional company was goat

well if a traditional company was goat would go to a software house and ask for

would go to a software house and ask for

would go to a software house and ask for some software to do this they would

some software to do this they would

some software to do this they would probably come up with these some

probably come up with these some

probably come up with these some combination of these four general ideas

combination of these four general ideas

combination of these four general ideas we’ve got the origination based

we’ve got the origination based

we’ve got the origination based technique so most countries have a law

technique so most countries have a law

technique so most countries have a law that requires financial services to

that requires financial services to

that requires financial services to prove the

prove the

prove the they’re talking to the real person

they’re talking to the real person

they’re talking to the real person origination is that’s it that’s what

origination is that’s it that’s what

origination is that’s it that’s what origination is I won one thing I get

origination is I won one thing I get

origination is I won one thing I get really really annoyed about is banks in

really really annoyed about is banks in

really really annoyed about is banks in the UK they’ve got this awful technique

the UK they’ve got this awful technique

the UK they’ve got this awful technique of using automated phone systems to try

of using automated phone systems to try

of using automated phone systems to try and prove you are who you say you are so

and prove you are who you say you are so

and prove you are who you say you are so you go through the whole series of you

you go through the whole series of you

you go through the whole series of you know police typing your ID number please

know police typing your ID number please

know police typing your ID number please type in your address please type in your

type in your address please type in your

type in your address please type in your password please do this please do that

password please do this please do that

password please do this please do that and that takes about three and a half

and that takes about three and a half

and that takes about three and a half minutes and then as soon as you finally

minutes and then as soon as you finally

minutes and then as soon as you finally speak to a real person which is all you

speak to a real person which is all you

speak to a real person which is all you wanted to do in the first place

wanted to do in the first place

wanted to do in the first place as soon as you’ve written speak to a

as soon as you’ve written speak to a

as soon as you’ve written speak to a real person they ask all the same

real person they ask all the same

real person they ask all the same questions again and it turns out they do

questions again and it turns out they do

questions again and it turns out they do this because these businesses aren’t

this because these businesses aren’t

this because these businesses aren’t quite sure that the automated method

quite sure that the automated method

quite sure that the automated method really is proof enough that the personal

really is proof enough that the personal

really is proof enough that the personal methods are actually going through a

methods are actually going through a

methods are actually going through a variety does my head in and some some

variety does my head in and some some

variety does my head in and some some may be less secure instances such as

may be less secure instances such as

may be less secure instances such as insurance agencies and people that are

insurance agencies and people that are

insurance agencies and people that are not necessarily as interested in

not necessarily as interested in

not necessarily as interested in protecting security they can use some

protecting security they can use some

protecting security they can use some really quite dodgy methods like I’ve had

really quite dodgy methods like I’ve had

really quite dodgy methods like I’ve had some cases where people have asked me

some cases where people have asked me

some cases where people have asked me just for my date of birth or just for my

just for my date of birth or just for my

just for my date of birth or just for my postcode or something like that and

postcode or something like that and

postcode or something like that and they’re completely not secure your date

they’re completely not secure your date

they’re completely not secure your date of birth is basically a password you

of birth is basically a password you

of birth is basically a password you were given at birth you can’t change

were given at birth you can’t change

were given at birth you can’t change it’s fixed and you have to live with it

it’s fixed and you have to live with it

it’s fixed and you have to live with it so it’s the worst password that that can

so it’s the worst password that that can

so it’s the worst password that that can ever exist the next group of

ever exist the next group of

ever exist the next group of technologies are rules based so these

technologies are rules based so these

technologies are rules based so these are static rules that are usually

are static rules that are usually

are static rules that are usually provided by analysts saying that you

provided by analysts saying that you

provided by analysts saying that you know no transaction must be bigger than

know no transaction must be bigger than

know no transaction must be bigger than X or you can’t have so many transactions

X or you can’t have so many transactions

X or you can’t have so many transactions within a certain period of time

within a certain period of time

within a certain period of time something like that and they’re and

something like that and they’re and

something like that and they’re and they’re great and they’re okay and they

they’re great and they’re okay and they

they’re great and they’re okay and they catch a reasonable amount of fraud it’s

catch a reasonable amount of fraud it’s

catch a reasonable amount of fraud it’s usually the the accidental types and the

usually the the accidental types and the

usually the the accidental types and the basically the not so intelligent

basically the not so intelligent

basically the not so intelligent fraudsters would try and do something

fraudsters would try and do something

fraudsters would try and do something silly like this but also it u also catch

silly like this but also it u also catch

silly like this but also it u also catch all the good guys as well like like when

all the good guys as well like like when

all the good guys as well like like when you’re abroad you cards always declined

you’re abroad you cards always declined

you’re abroad you cards always declined the first time because they think it’s

the first time because they think it’s

the first time because they think it’s fraudulent or you know any trying to buy

fraudulent or you know any trying to buy

fraudulent or you know any trying to buy a new car from a guy and he you know

a new car from a guy and he you know

a new car from a guy and he you know takes cash and you try and pull out 1500

takes cash and you try and pull out 1500

takes cash and you try and pull out 1500 pounds out of the cash machine you can’t

pounds out of the cash machine you can’t

pounds out of the cash machine you can’t do it because it’s you know it’s against

do it because it’s you know it’s against

do it because it’s you know it’s against their static rules credit checks

their static rules credit checks

their static rules credit checks lots of agencies will gladly accept your

lots of agencies will gladly accept your

lots of agencies will gladly accept your money to provide you with a number

money to provide you with a number

money to provide you with a number that’s it and these numbers are supposed

that’s it and these numbers are supposed

that’s it and these numbers are supposed to represent the worthiness or the the

to represent the worthiness or the the

to represent the worthiness or the the risk that that person provides to your

risk that that person provides to your

risk that that person provides to your business and there is certainly a case

business and there is certainly a case

business and there is certainly a case there’s an argument to use them how

there’s an argument to use them how

there’s an argument to use them how accurate they are is another question

accurate they are is another question

accurate they are is another question aggregation and monitoring so this is

aggregation and monitoring so this is

aggregation and monitoring so this is more of a reactive type of solution

more asking the question exactly where

more asking the question exactly where do I put the line to say that’s Class A

do I put the line to say that’s Class A

do I put the line to say that’s Class A and that’s Class B and finally

and that’s Class B and finally

and that’s Class B and finally regression which is trying to predict a

regression which is trying to predict a

regression which is trying to predict a value based upon their previous inputs

value based upon their previous inputs

value based upon their previous inputs we’ve also got different types of

we’ve also got different types of

we’ve also got different types of learning as well learning is the key

learning as well learning is the key

learning as well learning is the key thing that’s this really enabled deep

thing that’s this really enabled deep

thing that’s this really enabled deep learning to to come to the forefront is

learning to to come to the forefront is

learning to to come to the forefront is that the new training techniques that

that the new training techniques that

that the new training techniques that have been developed are so much more

have been developed are so much more

have been developed are so much more powerful than they were in the past

powerful than they were in the past

powerful than they were in the past training can be split into supervised

training can be split into supervised

training can be split into supervised and unsupervised learning supervised

and unsupervised learning supervised

and unsupervised learning supervised learning is where you have an expected

learning is where you have an expected

learning is where you have an expected result so it’s a it’s labeled so you say

result so it’s a it’s labeled so you say

result so it’s a it’s labeled so you say that this raw data is supposed to belong

that this raw data is supposed to belong

that this raw data is supposed to belong to Class A this is supposed to be the

to Class A this is supposed to be the

to Class A this is supposed to be the number one or this person is fraudulent

the algorithm is then trained the

the algorithm is then trained the parameters of the algorithm and then

parameters of the algorithm and then

parameters of the algorithm and then tuned to try and produce that same

tuned to try and produce that same

tuned to try and produce that same result and the the measure of

result and the the measure of

result and the the measure of performance for that algorithm is

performance for that algorithm is

performance for that algorithm is compared to the true result versus the

compared to the true result versus the

compared to the true result versus the predicted Frizzle and then when you were

predicted Frizzle and then when you were

predicted Frizzle and then when you were to use this in in real life if you had

to use this in in real life if you had

to use this in in real life if you had new data coming in then you would use

new data coming in then you would use

new data coming in then you would use those pre learnt weights and you would

those pre learnt weights and you would

those pre learnt weights and you would predict an output based upon that for

predict an output based upon that for

predict an output based upon that for unsupervised

unsupervised

unsupervised you’ve got no results so you don’t know

you’ve got no results so you don’t know

you’ve got no results so you don’t know exactly what class it’s supposed to

exactly what class it’s supposed to

exactly what class it’s supposed to belong to algorithms are trained in you

belong to algorithms are trained in you

belong to algorithms are trained in you need to decide on on what’s going to

need to decide on on what’s going to

need to decide on on what’s going to provide you with a measure of how good

provide you with a measure of how good

provide you with a measure of how good your algorithms be trained so some some

your algorithms be trained so some some

your algorithms be trained so some some of them deciding whether data are close

of them deciding whether data are close

of them deciding whether data are close or far away so since this measure of

or far away so since this measure of

or far away so since this measure of distance between data the there’s also

distance between data the there’s also

distance between data the there’s also may be other reasons why you want to do

may be other reasons why you want to do

may be other reasons why you want to do it as well and you can provide your own

it as well and you can provide your own

it as well and you can provide your own we’re talking about

we’re talking about

we’re talking about customized or personalized customized

customized or personalized customized

customized or personalized customized functions to actually cost whether your

functions to actually cost whether your

functions to actually cost whether your output is going to be labeled as class 1

output is going to be labeled as class 1

output is going to be labeled as class 1 or class 2 if something is important but

or class 2 if something is important but

or class 2 if something is important but in the real in the real world most data

in the real in the real world most data

in the real in the real world most data is usually semi-supervised

is usually semi-supervised

is usually semi-supervised you usually start off with some label

you usually start off with some label

you usually start off with some label data and usually a lot more that is

data and usually a lot more that is

data and usually a lot more that is unlabeled so you can kind of combine

unlabeled so you can kind of combine

unlabeled so you can kind of combine these two things together to maybe you

these two things together to maybe you

these two things together to maybe you can use the labeled stuff to start to

can use the labeled stuff to start to

can use the labeled stuff to start to bring out some of the clusters and then

bring out some of the clusters and then

bring out some of the clusters and then apply the unlabeled data to you know

apply the unlabeled data to you know

apply the unlabeled data to you know really filling the pattern a bit more so

really filling the pattern a bit more so

really filling the pattern a bit more so let’s talk about some specific

let’s talk about some specific

let’s talk about some specific algorithms I’m going to talk about to

algorithms I’m going to talk about to

algorithms I’m going to talk about to every every guy’s got his own favorite

every every guy’s got his own favorite

every every guy’s got his own favorite algorithm this first one is called a

algorithm this first one is called a

algorithm this first one is called a decision tree and there’s various

decision tree and there’s various

decision tree and there’s various different types of decision tree but

different types of decision tree but

different types of decision tree but we’re going to stick to the simple one

we’re going to stick to the simple one

we’re going to stick to the simple one for now and they can be used for

for now and they can be used for

for now and they can be used for classification and regression and the

classification and regression and the

classification and regression and the idea is that they predict the target of

idea is that they predict the target of

idea is that they predict the target of the target value of a class or a value

the target value of a class or a value

the target value of a class or a value or something based upon some very simple

or something based upon some very simple

or something based upon some very simple decision rules so is it less than 10 or

decision rules so is it less than 10 or

decision rules so is it less than 10 or bigger than 10 is it is it labeled a or

bigger than 10 is it is it labeled a or

bigger than 10 is it is it labeled a or labeled B the example we’ve got there on

labeled B the example we’ve got there on

labeled B the example we’ve got there on the right is quite morbid actually this

the right is quite morbid actually this

the right is quite morbid actually this is a decision tree that’s been learned

is a decision tree that’s been learned

is a decision tree that’s been learned from the data provided from the Titanic

from the data provided from the Titanic

from the data provided from the Titanic manifests and this is predicting whether

manifests and this is predicting whether

manifests and this is predicting whether you’re going to survive if you were on

you’re going to survive if you were on

you’re going to survive if you were on the Titanic or not so the first question

the Titanic or not so the first question

the Titanic or not so the first question it asks is is the sex male so if it was

it asks is is the sex male so if it was

it asks is is the sex male so if it was yes then it goes down to one side of the

yes then it goes down to one side of the

yes then it goes down to one side of the tree on the Left if it was no it goes

tree on the Left if it was no it goes

tree on the Left if it was no it goes down the right side of the tree so if

down the right side of the tree so if

down the right side of the tree so if you were female you had a pretty good

you were female you had a pretty good

you were female you had a pretty good chance of 0.73 so 73% chance of

chance of 0.73 so 73% chance of

chance of 0.73 so 73% chance of surviving and that represents 36% of the

surviving and that represents 36% of the

surviving and that represents 36% of the entire population inside the Titanic or

entire population inside the Titanic or

entire population inside the Titanic or as if you were male and if you were

as if you were male and if you were

as if you were male and if you were above 9.5 then you’ve got a fairly big

above 9.5 then you’ve got a fairly big

above 9.5 then you’ve got a fairly big chance that you’re going to die

chance that you’re going to die

chance that you’re going to die unfortunately 61% of all males of a 9.5

unfortunately 61% of all males of a 9.5

unfortunately 61% of all males of a 9.5 died and you can see that you can go

died and you can see that you can go

died and you can see that you can go down the tree and you can make a

down the tree and you can make a

down the tree and you can make a decision based upon these rules so the

decision based upon these rules so the

decision based upon these rules so the idea of the algorithm is to train these

idea of the algorithm is to train these

idea of the algorithm is to train these parameters these rules these decision

parameters these rules these decision

parameters these rules these decision points to optimally make the right

points to optimally make the right

points to optimally make the right decision

decision

decision so it’s conceptually quite simple it can

so it’s conceptually quite simple it can

so it’s conceptually quite simple it can handle categorical data which is great

handle categorical data which is great

handle categorical data which is great because some algorithms can’t but it

because some algorithms can’t but it

because some algorithms can’t but it well decision trees specifically can

well decision trees specifically can

well decision trees specifically can ooph it quite badly but there are lots

ooph it quite badly but there are lots

ooph it quite badly but there are lots of methods

of methods

of methods to to use decision trees in a different

to to use decision trees in a different

to to use decision trees in a different way to prevent the overfitting so don’t

way to prevent the overfitting so don’t

way to prevent the overfitting so don’t worry about that too much and decision

worry about that too much and decision

worry about that too much and decision trees are usually one of the simplest

trees are usually one of the simplest

trees are usually one of the simplest and sometimes effective enough to solve

and sometimes effective enough to solve

and sometimes effective enough to solve a problem the next algorithm and what’s

a problem the next algorithm and what’s

a problem the next algorithm and what’s surrounded by lots of hype at the moment

surrounded by lots of hype at the moment

surrounded by lots of hype at the moment is deep learning so deep learning is

is deep learning so deep learning is

is deep learning so deep learning is it’s really good because you remember

it’s really good because you remember

it’s really good because you remember those classes of types of algorithms at

those classes of types of algorithms at

those classes of types of algorithms at the start there he actually does all of

the start there he actually does all of

the start there he actually does all of them he does the dimensionality

them he does the dimensionality

them he does the dimensionality reduction the classification the

reduction the classification the

reduction the classification the regression and the clustering it could

regression and the clustering it could

regression and the clustering it could do all of it it’s a holy grail of

do all of it it’s a holy grail of

do all of it it’s a holy grail of algorithms no other algorithm can

algorithms no other algorithm can

algorithms no other algorithm can actually do all the same things the idea

actually do all the same things the idea

actually do all the same things the idea is that it’s actually trying to model

is that it’s actually trying to model

is that it’s actually trying to model our learning process in our brain

our learning process in our brain

our learning process in our brain basically it seems to model the neurons

basically it seems to model the neurons

basically it seems to model the neurons and the synapses in your brain to do the

and the synapses in your brain to do the

and the synapses in your brain to do the similar sort of tasks it’s it’s

similar sort of tasks it’s it’s

similar sort of tasks it’s it’s simplified somewhat but that’s that’s

simplified somewhat but that’s that’s

simplified somewhat but that’s that’s the general idea so the hope here is

the general idea so the hope here is

the general idea so the hope here is that if we can produce a model that of

that if we can produce a model that of

that if we can produce a model that of our brain that then we can merit right

our brain that then we can merit right

our brain that then we can merit right algorithms to perform things that our

algorithms to perform things that our

algorithms to perform things that our brain can do quite easily like

brain can do quite easily like

brain can do quite easily like recognition classification things like

recognition classification things like

recognition classification things like that so the pros and cons again it’s

that so the pros and cons again it’s

that so the pros and cons again it’s very versatile can be used for lots of

very versatile can be used for lots of

very versatile can be used for lots of different tasks

different tasks

different tasks the key improvement really is that it

the key improvement really is that it

the key improvement really is that it begins to remove the requirement of

begins to remove the requirement of

begins to remove the requirement of feature engineering so with all of the

feature engineering so with all of the

feature engineering so with all of the other algorithms your algorithm will

other algorithms your algorithm will

other algorithms your algorithm will live or die based upon what features you

live or die based upon what features you

live or die based upon what features you give the input you need to work really

give the input you need to work really

give the input you need to work really hard with other algorithms to to say

hard with other algorithms to to say

hard with other algorithms to to say that this is the most important feature

that this is the most important feature

that this is the most important feature I’m going to keep that and use that but

I’m going to keep that and use that but

I’m going to keep that and use that but those are the ones are completely

those are the ones are completely

those are the ones are completely redundant I’m going to remove them and

redundant I’m going to remove them and

redundant I’m going to remove them and that takes a significant amount of time

that takes a significant amount of time

that takes a significant amount of time with deep learning it has the ability of

with deep learning it has the ability of

with deep learning it has the ability of internally during the training stage of

internally during the training stage of

internally during the training stage of either completely removing parameters or

either completely removing parameters or

either completely removing parameters or completely keeping parameters purely

completely keeping parameters purely

completely keeping parameters purely based upon how well it fits the data how

based upon how well it fits the data how

based upon how well it fits the data how well the training process goes so it

well the training process goes so it

well the training process goes so it removes the bias that comes from

removes the bias that comes from

removes the bias that comes from removing data or adding data that you’re

removing data or adding data that you’re

removing data or adding data that you’re not sure it should be there or not the

not sure it should be there or not the

not sure it should be there or not the the main con actually there’s a suppose

the main con actually there’s a suppose

the main con actually there’s a suppose there’s a couple of cons the biggest one

there’s a couple of cons the biggest one

there’s a couple of cons the biggest one is it can be hard to visualize as soon

is it can be hard to visualize as soon

is it can be hard to visualize as soon as you start getting into

as you start getting into

as you start getting into neural network sizes that are quite deep

neural network sizes that are quite deep

neural network sizes that are quite deep it can be quite hard to visualize and

it can be quite hard to visualize and

it can be quite hard to visualize and conceptualize I’m hopefully going to try

conceptualize I’m hopefully going to try

conceptualize I’m hopefully going to try and prove that wrong in a little bit but

and prove that wrong in a little bit but

and prove that wrong in a little bit but um that’s that’s the problem number one

um that’s that’s the problem number one

um that’s that’s the problem number one and problem number two can be quite

and problem number two can be quite

and problem number two can be quite computationally expensive but that’s

computationally expensive but that’s

computationally expensive but that’s that’s true for kind of lots of these

that’s true for kind of lots of these

that’s true for kind of lots of these algorithms really so how do they

algorithms really so how do they

algorithms really so how do they actually work well they all it works

actually work well they all it works

actually work well they all it works primarily by trying to conceptualize

primarily by trying to conceptualize

primarily by trying to conceptualize things so there’s this idea that that

things so there’s this idea that that

things so there’s this idea that that neural networks are acting like a

neural networks are acting like a

neural networks are acting like a hierarchy of of concepts and the the

hierarchy of of concepts and the the

hierarchy of of concepts and the the whole goal really is to take those

whole goal really is to take those

whole goal really is to take those images also take your data and produce a

images also take your data and produce a

images also take your data and produce a concept something that accurately

concept something that accurately

concept something that accurately describes what is provided at the input

describes what is provided at the input

describes what is provided at the input so we’ve got the couple of the concepts

so we’ve got the couple of the concepts

so we’ve got the couple of the concepts on the left there we’ve got a street an

on the left there we’ve got a street an

on the left there we’ve got a street an animal and a person but you can see that

animal and a person but you can see that

animal and a person but you can see that you don’t

you don’t

you don’t the to the bottom ones the person and

the to the bottom ones the person and

the to the bottom ones the person and the animal there they’re actually linked

the animal there they’re actually linked

the animal there they’re actually linked by another concept you know they’re both

by another concept you know they’re both

by another concept you know they’re both animals is just one of them’s human so

animals is just one of them’s human so

animals is just one of them’s human so the great thing about the delayering

the great thing about the delayering

the great thing about the delayering concept is that you can actually start

concept is that you can actually start

concept is that you can actually start to tag things that are similar but not

to tag things that are similar but not

to tag things that are similar but not quite the same based upon your training

quite the same based upon your training

quite the same based upon your training data so to be more specific this says is

data so to be more specific this says is

data so to be more specific this says is a an example of how you would go about

a an example of how you would go about

a an example of how you would go about conceptualizing an image so each pixel

conceptualizing an image so each pixel

conceptualizing an image so each pixel within the image that’s the dashed lines

within the image that’s the dashed lines

within the image that’s the dashed lines there that would be passed into the

there that would be passed into the

there that would be passed into the input of our deep learning and it would

input of our deep learning and it would

input of our deep learning and it would start to reduce concepts around those

start to reduce concepts around those

start to reduce concepts around those pixels so the first layer might decide

pixels so the first layer might decide

pixels so the first layer might decide that there’s a you know part of a tire

that there’s a you know part of a tire

that there’s a you know part of a tire or a pile of a rim or an end plate or

or a pile of a rim or an end plate or

or a pile of a rim or an end plate or something like that usually very small

something like that usually very small

something like that usually very small discreet kind of local things within the

discreet kind of local things within the

discreet kind of local things within the image the next layer might start to

image the next layer might start to

image the next layer might start to build in that concept and build a

build in that concept and build a

build in that concept and build a concept of a tire or a full wing or a

concept of a tire or a full wing or a

concept of a tire or a full wing or a real wing and then finally we get to the

real wing and then finally we get to the

real wing and then finally we get to the classification and in this case is an f1

classification and in this case is an f1

classification and in this case is an f1 car but you can imagine that if you then

car but you can imagine that if you then

car but you can imagine that if you then showed the algorithm a normal car it

showed the algorithm a normal car it

showed the algorithm a normal car it could reuse some of those concepts they

could reuse some of those concepts they

could reuse some of those concepts they all they still have wheels they still

all they still have wheels they still

all they still have wheels they still have you know cockpits or our bodies

have you know cockpits or our bodies

have you know cockpits or our bodies away probably don’t have wings I don’t

away probably don’t have wings I don’t

away probably don’t have wings I don’t know maybe maybe in Leeds I don’t don’t

know maybe maybe in Leeds I don’t don’t

know maybe maybe in Leeds I don’t don’t about Denmark

about Denmark

about Denmark but you can reuse some of these concepts

but you can reuse some of these concepts

but you can reuse some of these concepts and that kind of shows the applicability

and that kind of shows the applicability

and that kind of shows the applicability to not just not just problems that it’s

to not just not just problems that it’s

to not just not just problems that it’s already seen but also future problems

already seen but also future problems

already seen but also future problems that it hasn’t seen and so just to

that it hasn’t seen and so just to

that it hasn’t seen and so just to finish this section off really just

finish this section off really just

finish this section off really just machine learning in the news or deep

machine learning in the news or deep

machine learning in the news or deep learning in them in the news the the one

learning in them in the news the the one

learning in them in the news the the one I really like that’s accessible to

I really like that’s accessible to

I really like that’s accessible to anybody really is the Google the new

anybody really is the Google the new

anybody really is the Google the new Google Translate app that takes pictures

Google Translate app that takes pictures

Google Translate app that takes pictures of signs or text in a different language

of signs or text in a different language

of signs or text in a different language and it translates that text but the real

and it translates that text but the real

and it translates that text but the real the cool USP of the whole thing is that

the cool USP of the whole thing is that

the cool USP of the whole thing is that it actually takes the image and replaces

it actually takes the image and replaces

it actually takes the image and replaces the image with the correct text in your

the image with the correct text in your

the image with the correct text in your language so here we’ve got a Russian

language so here we’ve got a Russian

language so here we’ve got a Russian sign and it’s replaced it with the

sign and it’s replaced it with the

sign and it’s replaced it with the English here actually I say he says

English here actually I say he says

English here actually I say he says access the city but according to my

access the city but according to my

access the city but according to my friend who who speaks Russian it

friend who who speaks Russian it

friend who who speaks Russian it actually means exit to village so not

actually means exit to village so not

actually means exit to village so not access to City exit to village but it’s

access to City exit to village but it’s

access to City exit to village but it’s not quite as grandiose if we showed if

not quite as grandiose if we showed if

not quite as grandiose if we showed if Google showed us science and exit to

Google showed us science and exit to

Google showed us science and exit to village so it’s probably why they

village so it’s probably why they

village so it’s probably why they changed it and then we’ve got the the

changed it and then we’ve got the the

changed it and then we’ve got the the images at the bottom and this is a new

images at the bottom and this is a new

images at the bottom and this is a new chip developed by IBM it’s been a few

chip developed by IBM it’s been a few

chip developed by IBM it’s been a few years in the making actually but

years in the making actually but

years in the making actually but effectively it’s a a deep learning

effectively it’s a a deep learning

effectively it’s a a deep learning neural network type infrastructure

neural network type infrastructure

neural network type infrastructure inside a chip so obviously you’ve got

inside a chip so obviously you’ve got

inside a chip so obviously you’ve got the cause and you used to the cause

the cause and you used to the cause

the cause and you used to the cause imagine the cause parallelized massively

imagine the cause parallelized massively

imagine the cause parallelized massively so instead of having you know one call

so instead of having you know one call

so instead of having you know one call we’ve got tens of thousands in this case

we’ve got tens of thousands in this case

we’ve got tens of thousands in this case is actually a million there’s a a

is actually a million there’s a a

is actually a million there’s a a million neurons in this chip so it’s

million neurons in this chip so it’s

million neurons in this chip so it’s able to do a million parallel tasks all

able to do a million parallel tasks all

able to do a million parallel tasks all at the same time and when we go through

at the same time and when we go through

at the same time and when we go through some of the examples in in a minute

some of the examples in in a minute

some of the examples in in a minute we’re going to be talking about like

we’re going to be talking about like

we’re going to be talking about like image sizes like they’re 10 10 by 10 100

image sizes like they’re 10 10 by 10 100

image sizes like they’re 10 10 by 10 100 input pixels that go down to maybe 2 to

input pixels that go down to maybe 2 to

input pixels that go down to maybe 2 to 2 outputs on there 2 dimensions on the

2 outputs on there 2 dimensions on the

2 outputs on there 2 dimensions on the output so that’s kind of nothing in

output so that’s kind of nothing in

output so that’s kind of nothing in comparison to what this could do and

comparison to what this could do and

comparison to what this could do and this is actually in hardware as well so

this is actually in hardware as well so

this is actually in hardware as well so it’s super fast super low power and

it’s super fast super low power and

it’s super fast super low power and should produce some really interesting

should produce some really interesting

should produce some really interesting applications ok so it’s just to solidify

applications ok so it’s just to solidify

applications ok so it’s just to solidify the howdy learning works I’m going to

the howdy learning works I’m going to

the howdy learning works I’m going to take you through an example which is a

take you through an example which is a

take you through an example which is a description

description

description of some some numbers here so the the

of some some numbers here so the the

of some some numbers here so the the idea of this task is to recognize some

idea of this task is to recognize some

idea of this task is to recognize some handwritten digits and to classify them

handwritten digits and to classify them

handwritten digits and to classify them as a number from 0 to 9 so it’s a really

as a number from 0 to 9 so it’s a really

as a number from 0 to 9 so it’s a really classic here machine learning example

classic here machine learning example

classic here machine learning example but it’s really great to use in the

but it’s really great to use in the

but it’s really great to use in the example as an example because it’s very

example as an example because it’s very

example as an example because it’s very easy to understand very very easy for

easy to understand very very easy for

easy to understand very very easy for everybody to understand it’s just trying

everybody to understand it’s just trying

everybody to understand it’s just trying to recognize what that number is and the

to recognize what that number is and the

to recognize what that number is and the first thing we notice when we start

first thing we notice when we start

first thing we notice when we start looking at the data so the first step in

looking at the data so the first step in

looking at the data so the first step in any in any data analysis job is to have

any in any data analysis job is to have

any in any data analysis job is to have a look at the data and the first thing

a look at the data and the first thing

a look at the data and the first thing we notice is that if you actually if you

we notice is that if you actually if you

we notice is that if you actually if you look at that that top left number there

look at that that top left number there

look at that that top left number there so I’m not not completely sure whether

so I’m not not completely sure whether

so I’m not not completely sure whether that’s a 5 or a that’s 3 and this

that’s a 5 or a that’s 3 and this

that’s a 5 or a that’s 3 and this immediately brings problems because this

immediately brings problems because this

immediately brings problems because this data is actually labeled so every one of

data is actually labeled so every one of

data is actually labeled so every one of these examples you’ll see so each each

these examples you’ll see so each each

these examples you’ll see so each each number is an example you can see that

number is an example you can see that

number is an example you can see that it’s been inverted from maybe you’ve

it’s been inverted from maybe you’ve

it’s been inverted from maybe you’ve somebody written pen on white paper and

somebody written pen on white paper and

somebody written pen on white paper and it’s being inverted and then reduced to

it’s being inverted and then reduced to

it’s being inverted and then reduced to a fixed pixel size and then sent it as

a fixed pixel size and then sent it as

a fixed pixel size and then sent it as well and the first thing that we can see

well and the first thing that we can see

well and the first thing that we can see is we’re already not sure whether that’s

is we’re already not sure whether that’s

is we’re already not sure whether that’s a 3 or a 5 and so somebody’s gone

a 3 or a 5 and so somebody’s gone

a 3 or a 5 and so somebody’s gone through and labeled this data as being a

through and labeled this data as being a

through and labeled this data as being a 3 or a 5 but I’m not convinced that

3 or a 5 but I’m not convinced that

3 or a 5 but I’m not convinced that that’s actually correct so we’re giving

that’s actually correct so we’re giving

that’s actually correct so we’re giving our algorithm potentially dodgy data

our algorithm potentially dodgy data

our algorithm potentially dodgy data already so there are in mind whenever

already so there are in mind whenever

already so there are in mind whenever you’re trying to train data that your

you’re trying to train data that your

you’re trying to train data that your your label data might not be right in

your label data might not be right in

your label data might not be right in the first place because it’s usually

the first place because it’s usually

the first place because it’s usually it’s usually labeled by by humans so

it’s usually labeled by by humans so

it’s usually labeled by by humans so what we then do with each example is we

what we then do with each example is we

what we then do with each example is we feed it into an input layer so I’m

feed it into an input layer so I’m

feed it into an input layer so I’m trying to stay away from the term neural

trying to stay away from the term neural

trying to stay away from the term neural network although I’ve mentioned it a

network although I’ve mentioned it a

network although I’ve mentioned it a couple of times because that it’s been

couple of times because that it’s been

couple of times because that it’s been around since the 80s but it it sounds

around since the 80s but it it sounds

around since the 80s but it it sounds complicated but it’s really not all the

complicated but it’s really not all the

complicated but it’s really not all the neural network is you have a node where

neural network is you have a node where

neural network is you have a node where some data goes in and then you have have

some data goes in and then you have have

some data goes in and then you have have links to an annexe subset of nodes and

links to an annexe subset of nodes and

links to an annexe subset of nodes and those are those links all have weights

those are those links all have weights

those are those links all have weights that it’s as simple as that all we do is

that it’s as simple as that all we do is

that it’s as simple as that all we do is we alter the weights within the the

we alter the weights within the the

we alter the weights within the the network in order to perform a task so

network in order to perform a task so

network in order to perform a task so I’ll try and refrain from using that

I’ll try and refrain from using that

I’ll try and refrain from using that terminology so our input layer is

terminology so our input layer is

terminology so our input layer is usually the same size as the size of the

usually the same size as the size of the

usually the same size as the size of the data so here we’ve got made maybe 10 by

data so here we’ve got made maybe 10 by

data so here we’ve got made maybe 10 by 10 pixels so we’ve got 100 inputs

10 pixels so we’ve got 100 inputs

10 pixels so we’ve got 100 inputs have one input for each pixel we then

have one input for each pixel we then

have one input for each pixel we then pass that data through to what’s known

pass that data through to what’s known

pass that data through to what’s known as a hidden layer and we call it hidden

as a hidden layer and we call it hidden

as a hidden layer and we call it hidden layer a bit basically because it’s not

layer a bit basically because it’s not

layer a bit basically because it’s not an input or an output it’s something in

an input or an output it’s something in

an input or an output it’s something in the middle it’s not directly observable

the middle it’s not directly observable

the middle it’s not directly observable and the way in which they’re connected

and the way in which they’re connected

and the way in which they’re connected is with a weight and during the training

is with a weight and during the training

is with a weight and during the training process those weights could be you know

process those weights could be you know

process those weights could be you know completely removed by setting it to zero

completely removed by setting it to zero

completely removed by setting it to zero or you know completely kept by sitting

or you know completely kept by sitting

or you know completely kept by sitting it’s all one and that’s all the training

it’s all one and that’s all the training

it’s all one and that’s all the training process is doing so what’s really great

process is doing so what’s really great

process is doing so what’s really great at this point is that those weights

at this point is that those weights

at this point is that those weights actually they combine in the next layer

actually they combine in the next layer

actually they combine in the next layer so you might have learnt that the

so you might have learnt that the

so you might have learnt that the weights that have been learned for that

weights that have been learned for that

weights that have been learned for that one particular neuron in the hidden

one particular neuron in the hidden

one particular neuron in the hidden layer can actually be treated as like a

layer can actually be treated as like a

layer can actually be treated as like a feature this is this is the beginnings

feature this is this is the beginnings

feature this is this is the beginnings of a concept so it’s saying that given

of a concept so it’s saying that given

of a concept so it’s saying that given that one neuron that one item in the

that one neuron that one item in the

that one neuron that one item in the hidden layer there that has that has

hidden layer there that has that has

hidden layer there that has that has certain weights on each of the input

certain weights on each of the input

certain weights on each of the input pixels so if we if if we were to make

pixels so if we if if we were to make

pixels so if we if if we were to make that the output layer there we could

that the output layer there we could

that the output layer there we could imagine that if that was the the output

imagine that if that was the the output

imagine that if that was the the output layer for the number one the weights

layer for the number one the weights

layer for the number one the weights would represent a shape that looks

would represent a shape that looks

would represent a shape that looks something like the number one generally

something like the number one generally

something like the number one generally in hidden layers you have multiple

in hidden layers you have multiple

in hidden layers you have multiple hidden layers so you’re trying to get

hidden layers so you’re trying to get

hidden layers so you’re trying to get the algorithm to learn these small steps

the algorithm to learn these small steps

the algorithm to learn these small steps these small increments of of concept and

these small increments of of concept and

these small increments of of concept and what we can actually do is to say that

what we can actually do is to say that

what we can actually do is to say that for for that one hidden layer we can go

for for that one hidden layer we can go

for for that one hidden layer we can go back and say what does the input layer

back and say what does the input layer

back and say what does the input layer have to look like in order to fully

have to look like in order to fully

have to look like in order to fully activate that one neuron and only that

activate that one neuron and only that

activate that one neuron and only that one neuron so this is an example of that

one neuron so this is an example of that

one neuron so this is an example of that hidden feature layer here and it might

hidden feature layer here and it might

hidden feature layer here and it might look a bit abstract but you you can just

look a bit abstract but you you can just

look a bit abstract but you you can just about start to make out that it’s

about start to make out that it’s

about start to make out that it’s starting to learn this kind of ghostly

starting to learn this kind of ghostly

starting to learn this kind of ghostly images of numbers in there and that’s

images of numbers in there and that’s

images of numbers in there and that’s because it’s starting to learn some of

because it’s starting to learn some of

because it’s starting to learn some of these concepts if you were to use a

these concepts if you were to use a

these concepts if you were to use a number of hidden layers and say you know

number of hidden layers and say you know

number of hidden layers and say you know don’t don’t try and learn the number all

don’t don’t try and learn the number all

don’t don’t try and learn the number all in one go it might come up with features

in one go it might come up with features

in one go it might come up with features that are like edges maybe it could learn

that are like edges maybe it could learn

that are like edges maybe it could learn the edge of the stick of a7 or maybe you

the edge of the stick of a7 or maybe you

the edge of the stick of a7 or maybe you can start to learn some curves of a nine

can start to learn some curves of a nine

can start to learn some curves of a nine or something like that and these are the

or something like that and these are the

or something like that and these are the hidden features that are in the middle

hidden features that are in the middle

hidden features that are in the middle of all these these networks

of all these these networks

of all these these networks so then finally we would produce an

so then finally we would produce an

so then finally we would produce an output layer which usually amounts to

output layer which usually amounts to

output layer which usually amounts to the number of possible classifications

the number of possible classifications

the number of possible classifications that we want to make so for our output

that we want to make so for our output

that we want to make so for our output layer we would have 10 we would have 0

layer we would have 10 we would have 0

layer we would have 10 we would have 0 to 9 and each one of those nodes would

to 9 and each one of those nodes would

to 9 and each one of those nodes would represent a number and at the output

represent a number and at the output

represent a number and at the output layer if we were to actually put one of

layer if we were to actually put one of

layer if we were to actually put one of these examples in you’d never get 100%

these examples in you’d never get 100%

these examples in you’d never get 100% you always get this the we’re talking

you always get this the we’re talking

you always get this the we’re talking earlier about how they’re they’re not

earlier about how they’re they’re not

earlier about how they’re they’re not deterministic but you kind of they are

deterministic but you kind of they are

deterministic but you kind of they are deterministic in the sense that they

deterministic in the sense that they

deterministic in the sense that they have fixed weight so you can follow the

have fixed weight so you can follow the

have fixed weight so you can follow the path of those weights through the data

path of those weights through the data

path of those weights through the data however we’re never quite sure like

however we’re never quite sure like

however we’re never quite sure like going back to that previous example

going back to that previous example

going back to that previous example we’re never quite sure whether it’s a 5

we’re never quite sure whether it’s a 5

we’re never quite sure whether it’s a 5 or a 3 so we’re going to the algorithm

or a 3 so we’re going to the algorithm

or a 3 so we’re going to the algorithm will probably decide that I’m 50 percent

will probably decide that I’m 50 percent

will probably decide that I’m 50 percent sure that it’s a 5 but there’s a 40%

sure that it’s a 5 but there’s a 40%

sure that it’s a 5 but there’s a 40% chance there could be a 3 so all of the

chance there could be a 3 so all of the

chance there could be a 3 so all of the numbers that are generated basically the

numbers that are generated basically the

numbers that are generated basically the the classification is made by picking

the classification is made by picking

the classification is made by picking the highest of those numbers so in this

the highest of those numbers so in this

the highest of those numbers so in this case would say that the 5 is the

case would say that the 5 is the

case would say that the 5 is the classification for this example because

classification for this example because

classification for this example because that add the highest value at the output

but what’s really cool as well is that

but what’s really cool as well is that we can actually rather than try and tell

we can actually rather than try and tell

we can actually rather than try and tell it to classify the objects by only

it to classify the objects by only

it to classify the objects by only having 10 outputs we can actually

having 10 outputs we can actually

having 10 outputs we can actually produce the same number of outputs and

produce the same number of outputs and

produce the same number of outputs and inputs and say ask the algorithm please

inputs and say ask the algorithm please

inputs and say ask the algorithm please try and reconstruct the image based upon

try and reconstruct the image based upon

try and reconstruct the image based upon your hidden you know concepts and

your hidden you know concepts and

your hidden you know concepts and representations so what we can do here

representations so what we can do here

representations so what we can do here is given a certain output please reduce

is given a certain output please reduce

is given a certain output please reduce reproduce that input and then we could

reproduce that input and then we could

reproduce that input and then we could do some comparison to see how well it’s

do some comparison to see how well it’s

do some comparison to see how well it’s performed so this is an example of what

performed so this is an example of what

performed so this is an example of what a reconstruction actually looks like and

a reconstruction actually looks like and

a reconstruction actually looks like and if I just flick backwards or forwards

if I just flick backwards or forwards

if I just flick backwards or forwards between what was real what was the real

between what was real what was the real

between what was real what was the real input and what was the learned concepts

input and what was the learned concepts

input and what was the learned concepts about that you can kind of see that the

about that you can kind of see that the

about that you can kind of see that the learned concepts are kind of like a

learned concepts are kind of like a

learned concepts are kind of like a drunk blurred version of the real number

drunk blurred version of the real number

drunk blurred version of the real number and that’s because they’re kind of

and that’s because they’re kind of

and that’s because they’re kind of learning they did what the most likely

learning they did what the most likely

learning they did what the most likely look is for that particular number and

look is for that particular number and

look is for that particular number and and what’s really interesting is in the

and what’s really interesting is in the

and what’s really interesting is in the real data with what we won’t show

real data with what we won’t show

real data with what we won’t show whether that’s 3 or 5 but if you look at

whether that’s 3 or 5 but if you look at

whether that’s 3 or 5 but if you look at the drunk verse

the drunk verse

the drunk verse it actually looks a little bit more than

it actually looks a little bit more than

it actually looks a little bit more than a five and this is saying that the

a five and this is saying that the

a five and this is saying that the algorithm was decided um well but it’s

algorithm was decided um well but it’s

algorithm was decided um well but it’s probably been labeled as a five so that

probably been labeled as a five so that

probably been labeled as a five so that so the algorithm has has learnt that of

so the algorithm has has learnt that of

so the algorithm has has learnt that of those features as a five so when you try

those features as a five so when you try

those features as a five so when you try and reconstruct it it looks more like a

and reconstruct it it looks more like a

and reconstruct it it looks more like a five and then finally we talked about

five and then finally we talked about

five and then finally we talked about dimensionality reduction so what we can

dimensionality reduction so what we can

dimensionality reduction so what we can do is take that high dimensional output

do is take that high dimensional output

do is take that high dimensional output so in this case we have ten discrete

so in this case we have ten discrete

so in this case we have ten discrete classes from zero to nine and we can

classes from zero to nine and we can

classes from zero to nine and we can flatten them into space so we don’t have

flatten them into space so we don’t have

flatten them into space so we don’t have ten dimensions to plot all our data so

ten dimensions to plot all our data so

ten dimensions to plot all our data so we can’t we can’t plot the 50% of the

we can’t we can’t plot the 50% of the

we can’t we can’t plot the 50% of the five to thirty percent of the for the

five to thirty percent of the for the

five to thirty percent of the for the twenty percent of the three and so on

twenty percent of the three and so on

twenty percent of the three and so on and so on all on a graph because we

and so on all on a graph because we

and so on all on a graph because we don’t have that many dimensions so what

don’t have that many dimensions so what

don’t have that many dimensions so what we can do is flatten all of that into

we can do is flatten all of that into

we can do is flatten all of that into two dimensions and this is what this

two dimensions and this is what this

two dimensions and this is what this process is here and what it shows you is

process is here and what it shows you is

process is here and what it shows you is how well the data are clustering

how well the data are clustering

how well the data are clustering together so we can see if I have stand

together so we can see if I have stand

together so we can see if I have stand very close to my screen I can see that

very close to my screen I can see that

very close to my screen I can see that the number Seven’s at the bottom are

the number Seven’s at the bottom are

the number Seven’s at the bottom are quite well clustered there the number of

quite well clustered there the number of

quite well clustered there the number of eights are okay in the top left but then

eights are okay in the top left but then

eights are okay in the top left but then we’ve also got some very strange

we’ve also got some very strange

we’ve also got some very strange features like so let’s take the five and

features like so let’s take the five and

features like so let’s take the five and a three example you see the fives in the

a three example you see the fives in the

a three example you see the fives in the orange in the middle they’re pretty well

orange in the middle they’re pretty well

orange in the middle they’re pretty well mixed with the three and that’s kind of

mixed with the three and that’s kind of

mixed with the three and that’s kind of because there must be quite a lot of

because there must be quite a lot of

because there must be quite a lot of examples that look like a five or look

examples that look like a five or look

examples that look like a five or look like a three so they’re quite well mixed

like a three so they’re quite well mixed

like a three so they’re quite well mixed so that means to actually perform the

so that means to actually perform the

so that means to actually perform the classification the algorithm is gonna

classification the algorithm is gonna

classification the algorithm is gonna have to work really hard to try and you

have to work really hard to try and you

have to work really hard to try and you know pull those apart so this is what

know pull those apart so this is what

know pull those apart so this is what you would generally do on the output is

you would generally do on the output is

you would generally do on the output is you would you would try and visualize

you would you would try and visualize

you would you would try and visualize the data in such a way that we as humans

the data in such a way that we as humans

the data in such a way that we as humans can couldn’t understand it that could be

can couldn’t understand it that could be

can couldn’t understand it that could be in 2d or in 3d okay so hopefully that

in 2d or in 3d okay so hopefully that

in 2d or in 3d okay so hopefully that that section kind of introduced you to

that section kind of introduced you to

that section kind of introduced you to two deep learning and some of the ideas

two deep learning and some of the ideas

two deep learning and some of the ideas and some of the terminology so when I

and some of the terminology so when I

and some of the terminology so when I come to some of the financial demos

come to some of the financial demos

come to some of the financial demos there this should be much easier to

there this should be much easier to

there this should be much easier to understand so first example is a

understand so first example is a

understand so first example is a traditional example using a rules-based

traditional example using a rules-based

traditional example using a rules-based approach and in this case we’ve been a

approach and in this case we’ve been a

approach and in this case we’ve been a little bit fancy we use in graph

little bit fancy we use in graph

little bit fancy we use in graph database typically graphed over it

database typically graphed over it

database typically graphed over it databases aren’t used as much as we’d

databases aren’t used as much as we’d

databases aren’t used as much as we’d like but they do perform really well in

like but they do perform really well in

like but they do perform really well in a

a in a fraud based scenario so just

in a fraud based scenario so just

in a fraud based scenario so just quickly recap if you don’t know a graph

quickly recap if you don’t know a graph

quickly recap if you don’t know a graph database is a another new SQL database

database is a another new SQL database

database is a another new SQL database but its power really is the description

but its power really is the description

but its power really is the description of the data so the data can only ever be

of the data so the data can only ever be

of the data so the data can only ever be either a node or a relationship a node

either a node or a relationship a node

either a node or a relationship a node is like a thing or a noun whereas a

is like a thing or a noun whereas a

is like a thing or a noun whereas a relationship is is a link or a

relationship is is a link or a

relationship is is a link or a relationship or a or a verb that

relationship or a or a verb that

relationship or a or a verb that basically connects two concepts together

basically connects two concepts together

basically connects two concepts together and the key selling point really is that

and the key selling point really is that

and the key selling point really is that sometimes you’ve got data that is just

sometimes you’ve got data that is just

sometimes you’ve got data that is just better described in a graph like

better described in a graph like

better described in a graph like structure so for example when we’re

structure so for example when we’re

structure so for example when we’re talking about fraud and and finance and

talking about fraud and and finance and

talking about fraud and and finance and stuff

stuff

stuff you’ve got the concepts of people and

you’ve got the concepts of people and

you’ve got the concepts of people and accounts and those people and accounts

accounts and those people and accounts

accounts and those people and accounts are all linked to different things

are all linked to different things

are all linked to different things they’re linked to an address a link to a

they’re linked to an address a link to a

they’re linked to an address a link to a current account and so on so for example

current account and so on so for example

current account and so on so for example we’ve got the traditional the

we’ve got the traditional the

we’ve got the traditional the traditional social media use case where

traditional social media use case where

traditional social media use case where we’ve got bobs these Bobby’s friends

we’ve got bobs these Bobby’s friends

we’ve got bobs these Bobby’s friends with Jane we’ve got a chair contained

with Jane we’ve got a chair contained

with Jane we’ve got a chair contained within a room Jane bought a book and so

within a room Jane bought a book and so

within a room Jane bought a book and so on but the real power is that once

on but the real power is that once

on but the real power is that once you’ve modeled it in this way you can

you’ve modeled it in this way you can

you’ve modeled it in this way you can perform complex queries that you

perform complex queries that you

perform complex queries that you wouldn’t be able to do in a traditional

wouldn’t be able to do in a traditional

wouldn’t be able to do in a traditional relational database so when you wanted

relational database so when you wanted

relational database so when you wanted to do so to go back to the social media

to do so to go back to the social media

to do so to go back to the social media example again when you wanted to do like

example again when you wanted to do like

example again when you wanted to do like who is friends with my friend you have

who is friends with my friend you have

who is friends with my friend you have to do some crazy joined with your SQL in

to do some crazy joined with your SQL in

to do some crazy joined with your SQL in order to get that to work with a graph

order to get that to work with a graph

order to get that to work with a graph database you can just pop you can just

database you can just pop you can just

database you can just pop you can just hop through the graph it makes it really

hop through the graph it makes it really

hop through the graph it makes it really really fast so in their fraud situation

really fast so in their fraud situation

really fast so in their fraud situation we might model our data to something

we might model our data to something

we might model our data to something like this we might have an account

like this we might have an account

like this we might have an account holder in the middle and they have

holder in the middle and they have

holder in the middle and they have relationships with phone numbers or

relationships with phone numbers or

relationships with phone numbers or national insurance numbers things like

national insurance numbers things like

national insurance numbers things like that and then we can perform queries on

that and then we can perform queries on

that and then we can perform queries on that if we would like to but when you

that if we would like to but when you

that if we would like to but when you start viewing that in detail and

start viewing that in detail and

start viewing that in detail and actually viewing how these connections

actually viewing how these connections

actually viewing how these connections are connecting things together

are connecting things together

are connecting things together interesting patterns start to come out

interesting patterns start to come out

interesting patterns start to come out and especially if you’re visualizing it

and especially if you’re visualizing it

and especially if you’re visualizing it in this way as well it’s much easier to

in this way as well it’s much easier to

in this way as well it’s much easier to visualize data in this way than it is in

visualize data in this way than it is in

visualize data in this way than it is in a table for example so in this example

a table for example so in this example

a table for example so in this example we’ve got three account holders in red

we’ve got three account holders in red

we’ve got three account holders in red having the red yep they’re red and

having the red yep they’re red and

having the red yep they’re red and they’re linked in various different ways

they’re linked in various different ways

they’re linked in various different ways we’ve got all three of them are sharing

we’ve got all three of them are sharing

we’ve got all three of them are sharing the same address so who could be dodgy I

the same address so who could be dodgy I

the same address so who could be dodgy I actually had a person in another talk

actually had a person in another talk

actually had a person in another talk excuse me

excuse me

excuse me that III was suggesting that all three

that III was suggesting that all three

that III was suggesting that all three people sharing the same address that

people sharing the same address that

people sharing the same address that could be dodgy and and she was like no

could be dodgy and and she was like no

could be dodgy and and she was like no no no no when thousands of people are

no no no when thousands of people are

no no no when thousands of people are sharing the same address then it’s dodgy

sharing the same address then it’s dodgy

sharing the same address then it’s dodgy three is fine don’t worry about it so

three is fine don’t worry about it so

three is fine don’t worry about it so I’m like okay so but we could set up a

I’m like okay so but we could set up a

I’m like okay so but we could set up a rule there to say you know how many

rule there to say you know how many

rule there to say you know how many people are using the same address and

people are using the same address and

people are using the same address and you could do that in the traditional

you could do that in the traditional

you could do that in the traditional database but where the power really

database but where the power really

database but where the power really comes in is when you start linking these

comes in is when you start linking these

comes in is when you start linking these these things together and searching for

these things together and searching for

these things together and searching for these larger rings and groups within the

these larger rings and groups within the

these larger rings and groups within the data so if we imagine that directly two

data so if we imagine that directly two

data so if we imagine that directly two people aren’t sharing the same national

people aren’t sharing the same national

people aren’t sharing the same national insurance number for example which is

insurance number for example which is

insurance number for example which is illegal in the UK maybe there’s a third

illegal in the UK maybe there’s a third

illegal in the UK maybe there’s a third party which is linking these National

party which is linking these National

party which is linking these National Insurance numbers together so you

Insurance numbers together so you

Insurance numbers together so you actually start to form these rings

actually start to form these rings

actually start to form these rings within the data which are kind of not

within the data which are kind of not

within the data which are kind of not not natural this shouldn’t really be

not natural this shouldn’t really be

not natural this shouldn’t really be rings in the data and graph databases

rings in the data and graph databases

rings in the data and graph databases are really good at viewing and spotting

are really good at viewing and spotting

are really good at viewing and spotting these rings so that’s the kind of

these rings so that’s the kind of

these rings so that’s the kind of technology that would exist in the wild

technology that would exist in the wild

technology that would exist in the wild today if we were asked to to perform a

today if we were asked to to perform a

today if we were asked to to perform a job like this but where we’re really

job like this but where we’re really

job like this but where we’re really interested in is bringing some machine

interested in is bringing some machine

interested in is bringing some machine learning techniques to some of these

learning techniques to some of these

learning techniques to some of these ideas so the first idea I had was quite

ideas so the first idea I had was quite

ideas so the first idea I had was quite a typical one really and that’s why

a typical one really and that’s why

a typical one really and that’s why that’s why I did it because it was quite

that’s why I did it because it was quite

that’s why I did it because it was quite easy to do but basically if we could use

easy to do but basically if we could use

easy to do but basically if we could use vocal fingerprints for origination it

vocal fingerprints for origination it

vocal fingerprints for origination it would just solve just the the main

would just solve just the the main

would just solve just the the main reasons really it would save the user a

reasons really it would save the user a

reasons really it would save the user a significant amount of time the user

significant amount of time the user

significant amount of time the user experience would would you know be huge

experience would would you know be huge

experience would would you know be huge hugely improved not having to wait on

hugely improved not having to wait on

hugely improved not having to wait on the phone for 20 minutes just because

the phone for 20 minutes just because

the phone for 20 minutes just because some stupid automated system took you to

some stupid automated system took you to

some stupid automated system took you to the wrong place so if we can use their

the wrong place so if we can use their

the wrong place so if we can use their person’s voice as a form of

person’s voice as a form of

person’s voice as a form of authentication origination then we’ll be

authentication origination then we’ll be

authentication origination then we’ll be able to save time be able to save

able to save time be able to save

able to save time be able to save machines and be able to save their the

machines and be able to save their the

machines and be able to save their the power of people on the other end of the

power of people on the other end of the

power of people on the other end of the phone so to do this what we’d have to do

phone so to do this what we’d have to do

phone so to do this what we’d have to do is to record the customers voice

is to record the customers voice

is to record the customers voice we then pre-process the data in some way

we then pre-process the data in some way

we then pre-process the data in some way to clean it up and put it in a format

to clean it up and put it in a format

to clean it up and put it in a format that’s that’s capable of being put into

that’s that’s capable of being put into

that’s that’s capable of being put into an algorithm in this case we would trade

an algorithm in this case we would trade

an algorithm in this case we would trade a deep learning model but it could be

a deep learning model but it could be

a deep learning model but it could be any algorithm and then we’d store that

any algorithm and then we’d store that

any algorithm and then we’d store that fingerprint for future verification in

fingerprint for future verification in

fingerprint for future verification in the online scenario so once you’ve got

the online scenario so once you’ve got

the online scenario so once you’ve got set up the user would come on you’d

set up the user would come on you’d

set up the user would come on you’d rerecord his voice again maybe against

rerecord his voice again maybe against

rerecord his voice again maybe against the preset phrase maybe against new

the preset phrase maybe against new

the preset phrase maybe against new phrase and then you’d compare that

phrase and then you’d compare that

phrase and then you’d compare that result of the fingerprint and that would

result of the fingerprint and that would

result of the fingerprint and that would prove whether that person is you know

prove whether that person is you know

prove whether that person is you know really who they say they are so this is

really who they say they are so this is

really who they say they are so this is the pre-processing stage in action so

the pre-processing stage in action so

the pre-processing stage in action so this is a bit of signal processing which

this is a bit of signal processing which

this is a bit of signal processing which is converting the the time signature of

is converting the the time signature of

is converting the the time signature of the the audio file into frequency into

the the audio file into frequency into

the the audio file into frequency into the frequency domain so what you’re

the frequency domain so what you’re

the frequency domain so what you’re seeing there is a plot of the frequency

seeing there is a plot of the frequency

seeing there is a plot of the frequency components versus time so red is strong

components versus time so red is strong

components versus time so red is strong and that green blue a color is is weak

and that green blue a color is is weak

and that green blue a color is is weak so it’s saying that you know you can see

so it’s saying that you know you can see

so it’s saying that you know you can see there the gaps in between the data

there the gaps in between the data

there the gaps in between the data they’re a kind of where that paused to

they’re a kind of where that paused to

they’re a kind of where that paused to say the words and I think if we’re if it

say the words and I think if we’re if it

say the words and I think if we’re if it works yeah so this is some example data

works yeah so this is some example data

works yeah so this is some example data that I used in my learning and this is

that I used in my learning and this is

that I used in my learning and this is three examples of three people saying

three examples of three people saying

three examples of three people saying the same phrase don’t ask me what that

the same phrase don’t ask me what that

the same phrase don’t ask me what that phrase actually means I don’t know what

phrase actually means I don’t know what

phrase actually means I don’t know what anything but anyway you can tell

anything but anyway you can tell

anything but anyway you can tell yourself that those three voices sounded

yourself that those three voices sounded

yourself that those three voices sounded sometimes a little bit different but in

sometimes a little bit different but in

sometimes a little bit different but in that last example completely different

that last example completely different

that last example completely different and what we’re trying to do is to to

and what we’re trying to do is to to

and what we’re trying to do is to to make the deep learning think the same

make the deep learning think the same

make the deep learning think the same okay so once we’ve put it into our deep

okay so once we’ve put it into our deep

okay so once we’ve put it into our deep learning model we’ve done the training

learning model we’ve done the training

learning model we’ve done the training and we’ve produced an output our output

and we’ve produced an output our output

and we’ve produced an output our output in this case is between these three

in this case is between these three

in this case is between these three different people so you could have three

different people so you could have three

different people so you could have three outputs and then again we’ve compressed

outputs and then again we’ve compressed

outputs and then again we’ve compressed that we’ve squashed that under the

that we’ve squashed that under the

that we’ve squashed that under the screen into two dimensions and this is a

screen into two dimensions and this is a

screen into two dimensions and this is a plot that shows how close all of those

plot that shows how close all of those

plot that shows how close all of those voices were between so we’ve got a

voices were between so we’ve got a

voices were between so we’ve got a couple of different points in there and

couple of different points in there and

couple of different points in there and the the different colors there – Bob

the the different colors there – Bob

the the different colors there – Bob Steve and Dave they correspond to the

Steve and Dave they correspond to the

Steve and Dave they correspond to the three different examples the three

three different examples the three

three different examples the three different people giving the example

different people giving the example

different people giving the example sorry and each individual point is a

sorry and each individual point is a

sorry and each individual point is a specific phrase that they said so we had

specific phrase that they said so we had

specific phrase that they said so we had ten ten different phrases that they said

ten ten different phrases that they said

ten ten different phrases that they said and you can see that all of these

and you can see that all of these

and you can see that all of these examples are clustering together quite

examples are clustering together quite

examples are clustering together quite well so if we then took another they’re

well so if we then took another they’re

well so if we then took another they’re the same people but using a different

the same people but using a different

the same people but using a different spoken example so not the same examples

spoken example so not the same examples

spoken example so not the same examples how would that perform

how would that perform

how would that perform new data so I think we go again so the

new data so I think we go again so the

new data so I think we go again so the top line now in the results that was the

top line now in the results that was the

top line now in the results that was the the raw result the raw output of those

the raw result the raw output of those

the raw result the raw output of those three neurons throught for that file and

three neurons throught for that file and

three neurons throught for that file and it’s saying that one of the new your

it’s saying that one of the new your

it’s saying that one of the new your honors have 0.98

honors have 0.98

honors have 0.98 the 10.1 another 100.1 as well and

the 10.1 another 100.1 as well and

the 10.1 another 100.1 as well and that’s saying that you know Bob

that’s saying that you know Bob

that’s saying that you know Bob definitely pretty sure 19 percent sure

definitely pretty sure 19 percent sure

definitely pretty sure 19 percent sure that that was definitely Bob

there you go 97 percent chance that was

there you go 97 percent chance that was Steve there 96 percent it was Dave so

Steve there 96 percent it was Dave so

Steve there 96 percent it was Dave so that was that example quite a simple

that was that example quite a simple

that was that example quite a simple example in sense that it only used a

example in sense that it only used a

example in sense that it only used a very small data set but it’s you know

very small data set but it’s you know

very small data set but it’s you know it’s instructive and it kind of points

it’s instructive and it kind of points

it’s instructive and it kind of points towards things that we could do in the

towards things that we could do in the

towards things that we could do in the future given much more data I mean like

future given much more data I mean like

future given much more data I mean like every phone call we pick up these days

every phone call we pick up these days

every phone call we pick up these days there’s always a we are recording your

there’s always a we are recording your

there’s always a we are recording your voice for verification training purposes

voice for verification training purposes

voice for verification training purposes so there must be huge vast databases of

so there must be huge vast databases of

so there must be huge vast databases of people’s voices out there ok so next is

people’s voices out there ok so next is

people’s voices out there ok so next is ample decision trees so this is an

ample decision trees so this is an

ample decision trees so this is an example of decision tree that we showed

example of decision tree that we showed

example of decision tree that we showed earlier on and this is predicting

earlier on and this is predicting

earlier on and this is predicting mortgage default so amazingly two banks

mortgage default so amazingly two banks

mortgage default so amazingly two banks – – sorry two mortgage providers in the

– – sorry two mortgage providers in the

– – sorry two mortgage providers in the u.s. went bust as usual of course and

u.s. went bust as usual of course and

u.s. went bust as usual of course and were bailed out by the US taxpayer so we

were bailed out by the US taxpayer so we

were bailed out by the US taxpayer so we owned by the US government so Freddie

owned by the US government so Freddie

owned by the US government so Freddie but Freddie Mac and Fannie Mae and as

but Freddie Mac and Fannie Mae and as

but Freddie Mac and Fannie Mae and as part of their I don’t know as part of

part of their I don’t know as part of

part of their I don’t know as part of their reprisal basically a slap on the

their reprisal basically a slap on the

their reprisal basically a slap on the wrist the the government forced them to

wrist the the government forced them to

wrist the the government forced them to release lots of their data to the public

release lots of their data to the public

release lots of their data to the public and amazingly they they publicized a

and amazingly they they publicized a

and amazingly they they publicized a whole data set of mortgage applications

whole data set of mortgage applications

whole data set of mortgage applications and also historical accounts of what

and also historical accounts of what

and also historical accounts of what happened to those mortgage applications

happened to those mortgage applications

happened to those mortgage applications so you can say that they did told us

so you can say that they did told us

so you can say that they did told us whether that person then defaulted in

whether that person then defaulted in

whether that person then defaulted in the future so the task here is given

the future so the task here is given

the future so the task here is given some given some oh dear I’m running over

some given some oh dear I’m running over

some given some oh dear I’m running over time off to speed up given some data is

time off to speed up given some data is

time off to speed up given some data is it possible to predict whether that

it possible to predict whether that

it possible to predict whether that person’s going to default so the first

person’s going to default so the first

person’s going to default so the first the first problem is the whole data

the first problem is the whole data

the first problem is the whole data cleaning problem like we saw

cleaning problem like we saw

cleaning problem like we saw the previous talk it’s the vast majority

the previous talk it’s the vast majority

the previous talk it’s the vast majority of time to spend cleaning data

of time to spend cleaning data

of time to spend cleaning data I’m gonna skip over that so if we were

I’m gonna skip over that so if we were

I’m gonna skip over that so if we were to flatten all of the data that was

to flatten all of the data that was

to flatten all of the data that was recovered into a an image before we put

recovered into a an image before we put

recovered into a an image before we put it through the algorithm this is kind of

it through the algorithm this is kind of

it through the algorithm this is kind of what it looks like it’s very

what it looks like it’s very

what it looks like it’s very intermingled and mixed can’t quite

intermingled and mixed can’t quite

intermingled and mixed can’t quite understand what’s going on so a decision

understand what’s going on so a decision

understand what’s going on so a decision tree is is learning all of these rules

tree is is learning all of these rules

tree is is learning all of these rules and based upon the outcome of those

and based upon the outcome of those

and based upon the outcome of those rules is rather yes the person defaulted

rules is rather yes the person defaulted

rules is rather yes the person defaulted no they didn’t default so we had

no they didn’t default so we had

no they didn’t default so we had approximately 20,000 samples total 50-50

approximately 20,000 samples total 50-50

approximately 20,000 samples total 50-50 split a random forest classifier so it’s

split a random forest classifier so it’s

split a random forest classifier so it’s a type of decision tree algorithm but is

a type of decision tree algorithm but is

a type of decision tree algorithm but is better does not over fit as much only 11

better does not over fit as much only 11

better does not over fit as much only 11 input features so the main problem here

input features so the main problem here

input features so the main problem here is I don’t actually think we’ve got

is I don’t actually think we’ve got

is I don’t actually think we’ve got enough data to do a really good job but

enough data to do a really good job but

enough data to do a really good job but we’ll see what we can do and the one

we’ll see what we can do and the one

we’ll see what we can do and the one great thing about decision trees is that

great thing about decision trees is that

great thing about decision trees is that actually gives you a measure of

actually gives you a measure of

actually gives you a measure of importance for all of those variables so

importance for all of those variables so

importance for all of those variables so here we’ve got the variables that were

here we’ve got the variables that were

here we’ve got the variables that were inputted to the algorithm at the bottom

inputted to the algorithm at the bottom

inputted to the algorithm at the bottom and it shows their respective importance

and it shows their respective importance

and it shows their respective importance of those variables on there on the

of those variables on there on the

of those variables on there on the left-hand side so you can see actually

left-hand side so you can see actually

left-hand side so you can see actually the credit score is in second place so

the credit score is in second place so

the credit score is in second place so I’m not sure that the credit reference

I’m not sure that the credit reference

I’m not sure that the credit reference agencies would be too happy that you

agencies would be too happy that you

agencies would be too happy that you know they could only explain 0.25 of the

know they could only explain 0.25 of the

know they could only explain 0.25 of the data so 25% of the data could only be

data so 25% of the data could only be

data so 25% of the data could only be explained by the credit score alone so

explained by the credit score alone so

explained by the credit score alone so not not a great result for them and

not not a great result for them and

not not a great result for them and actually the most important measure was

actually the most important measure was

actually the most important measure was the HPI origination which was the house

the HPI origination which was the house

the HPI origination which was the house price index origination for that local

price index origination for that local

price index origination for that local area so this is saying that a person who

area so this is saying that a person who

area so this is saying that a person who took out a mortgage in a very local area

took out a mortgage in a very local area

took out a mortgage in a very local area it’s very dependent on the prices within

it’s very dependent on the prices within

it’s very dependent on the prices within that area as to whether they’re going to

that area as to whether they’re going to

that area as to whether they’re going to default or not and this is kind of a

default or not and this is kind of a

default or not and this is kind of a typical really in the US you can see

typical really in the US you can see

typical really in the US you can see like vast tracts of like places like

like vast tracts of like places like

like vast tracts of like places like Detroit that you know as soon as some of

Detroit that you know as soon as some of

Detroit that you know as soon as some of the jobs left everybody just lost their

the jobs left everybody just lost their

the jobs left everybody just lost their jobs in the whole house price area then

jobs in the whole house price area then

jobs in the whole house price area then crashed and then people couldn’t afford

crashed and then people couldn’t afford

crashed and then people couldn’t afford to sell because they couldn’t sell it so

to sell because they couldn’t sell it so

to sell because they couldn’t sell it so that’s kind of why that’s so important

that’s kind of why that’s so important

that’s kind of why that’s so important interesting result and then final

interesting result and then final

interesting result and then final example I’m having to move rather

example I’m having to move rather

example I’m having to move rather quickly here because I’ve only got two

quickly here because I’ve only got two

quickly here because I’ve only got two minutes left but is it possible to take

minutes left but is it possible to take

minutes left but is it possible to take that data

that data

that data and try and see whether there’s

and try and see whether there’s

and try and see whether there’s something strange going on without in

something strange going on without in

something strange going on without in the data so basically this is an

the data so basically this is an

the data so basically this is an unlabeled example we’re not telling it

unlabeled example we’re not telling it

unlabeled example we’re not telling it what to learn here so how do we do that

what to learn here so how do we do that

what to learn here so how do we do that well there’s a deep learning technique

well there’s a deep learning technique

well there’s a deep learning technique called an autoencoder which basically it

called an autoencoder which basically it

called an autoencoder which basically it takes the inputs and it restricts the

takes the inputs and it restricts the

takes the inputs and it restricts the number of hidden neurons to only a few

number of hidden neurons to only a few

number of hidden neurons to only a few concepts he’s saying you’ve really got a

concepts he’s saying you’ve really got a

concepts he’s saying you’ve really got a pick and choose what data you use and

pick and choose what data you use and

pick and choose what data you use and generate some concepts that are really

generate some concepts that are really

generate some concepts that are really quite strict and then we try and

quite strict and then we try and

quite strict and then we try and reproduce the output again and we’re

reproduce the output again and we’re

reproduce the output again and we’re comparing the output against the input

comparing the output against the input

comparing the output against the input as a measure of how well we’ll have done

as a measure of how well we’ll have done

as a measure of how well we’ll have done so basically those restrictions in the

so basically those restrictions in the

so basically those restrictions in the middle maybe only two neurons you know

middle maybe only two neurons you know

middle maybe only two neurons you know yes and no something like that is that

yes and no something like that is that

yes and no something like that is that possible to reconstruct the data so we

possible to reconstruct the data so we

possible to reconstruct the data so we can do that so there’s the same data as

can do that so there’s the same data as

can do that so there’s the same data as before slightly it’s a different random

before slightly it’s a different random

before slightly it’s a different random sample so it might look slightly

sample so it might look slightly

sample so it might look slightly different we’ve got an input layer a

different we’ve got an input layer a

different we’ve got an input layer a number of hidden layers that are

number of hidden layers that are

number of hidden layers that are compressing the data down into smaller

compressing the data down into smaller

compressing the data down into smaller and smaller neurons and then we’re

and smaller neurons and then we’re

and smaller neurons and then we’re reconstructing again back to the input

reconstructing again back to the input

reconstructing again back to the input layer and doing a comparison to see how

layer and doing a comparison to see how

layer and doing a comparison to see how well we did but what we can do then is

well we did but what we can do then is

well we did but what we can do then is plot in two or three D one of those

plot in two or three D one of those

plot in two or three D one of those hidden layers to actually view those

hidden layers to actually view those

hidden layers to actually view those concepts and what we’ve learnt and

concepts and what we’ve learnt and

concepts and what we’ve learnt and finally this is the result of that

finally this is the result of that

finally this is the result of that process and the left-hand side we’ve got

process and the left-hand side we’ve got

process and the left-hand side we’ve got a 2d representation and you can start to

a 2d representation and you can start to

a 2d representation and you can start to see there’s actually some structure

see there’s actually some structure

see there’s actually some structure within that data so most generally you

within that data so most generally you

within that data so most generally you can see that the people that defaulted

can see that the people that defaulted

can see that the people that defaulted the each ruse on that graph or on the on

the each ruse on that graph or on the on

the each ruse on that graph or on the on the left-hand side and the people that

the left-hand side and the people that

the left-hand side and the people that didn’t default on the right-hand side

didn’t default on the right-hand side

didn’t default on the right-hand side and within there if you look on the

and within there if you look on the

and within there if you look on the right-hand side there’s a couple of

right-hand side there’s a couple of

right-hand side there’s a couple of orange dots and that’s saying that the

orange dots and that’s saying that the

orange dots and that’s saying that the vast majority of people in there didn’t

vast majority of people in there didn’t

vast majority of people in there didn’t default but one or two people did now an

default but one or two people did now an

default but one or two people did now an analyst might start to ask why so it

analyst might start to ask why so it

analyst might start to ask why so it could be something quite innocent you

could be something quite innocent you

could be something quite innocent you know maybe the person lost his

know maybe the person lost his

know maybe the person lost his high-powered job went to prison

high-powered job went to prison

high-powered job went to prison something like that but it’s kind of

something like that but it’s kind of

something like that but it’s kind of indicative that something else is going

indicative that something else is going

indicative that something else is going on and this is where the analyst would

on and this is where the analyst would

on and this is where the analyst would come in and start investigating that

come in and start investigating that

come in and start investigating that data so these are completely unlabeled

data so these are completely unlabeled

data so these are completely unlabeled and the algorithm has absolutely no idea

and the algorithm has absolutely no idea

and the algorithm has absolutely no idea what it means

what it means

what it means and it still takes human to do some

and it still takes human to do some

and it still takes human to do some analysis and to do some investigation to

analysis and to do some investigation to

analysis and to do some investigation to figure out what has happened but these

figure out what has happened but these

figure out what has happened but these kinds of tools lead the analysts in the

kinds of tools lead the analysts in the

kinds of tools lead the analysts in the right direction as opposed to just

right direction as opposed to just

right direction as opposed to just taking a random Sam

taking a random Sam

taking a random Sam and then finally on the right hand side

and then finally on the right hand side

and then finally on the right hand side we’ve got a 3d representation of the

we’ve got a 3d representation of the

we’ve got a 3d representation of the same data and this is where it becomes

same data and this is where it becomes

same data and this is where it becomes really really powerful you can imagine

really really powerful you can imagine

really really powerful you can imagine like if you could get that graph and you

like if you could get that graph and you

like if you could get that graph and you can like look into it and and move it

can like look into it and and move it

can like look into it and and move it and turn it around and you can start to

and turn it around and you can start to

and turn it around and you can start to see clusters in 3d space and that’s when

see clusters in 3d space and that’s when

see clusters in 3d space and that’s when it starts to become immersive and given

it starts to become immersive and given

it starts to become immersive and given enough time it takes it takes a certain

enough time it takes it takes a certain

enough time it takes it takes a certain amount of time for any analyst to

amount of time for any analyst to

amount of time for any analyst to analyze data but given enough time they

analyze data but given enough time they

analyze data but given enough time they will be able to learn to see patterns

will be able to learn to see patterns

will be able to learn to see patterns within that data which will help them to

within that data which will help them to

within that data which will help them to investigate things that they haven’t

investigate things that they haven’t

investigate things that they haven’t seen before and I think I better stop

seen before and I think I better stop

seen before and I think I better stop there because I’ve completely run out of

there because I’ve completely run out of

there because I’ve completely run out of time so thank you very much for

time so thank you very much for

time so thank you very much for listening

”

GOTO 2015 • Modern Fraud Prevention using Deep Learning • Phil Winder

Be First to Comment

Leave a Reply Cancel reply