Looking at debit and credit card fraud

Summary

This article, written jointly by a mathematician and a barrister, looks at some of the statistical issues raised by court cases based on fraud involving chip and PIN cards. It provides examples and insights that statistics teachers should find helpful.

INTRODUCTION

Recent publicity around the world, including court cases, has raised questions about the safety of chip and PIN cards from fraudulent attack, for example by cloning. This article shows how some of the legal issues raised are relevant to statistics teaching at school and college level.

There are similarities, but also important differences, between statistical hypothesis testing and the concept of weighing evidence in a court of law. Many students find legal situations interesting and fairly easy to understand at an intuitive level. Relating them to statistical processes can thus be motivating and provide them with deeper insights.

COURT CASES

In a typical court case the claimant is an individual whose account has been debited as a result of one or more allegedly unauthorized transactions; the defendant is a bank or building society. A common counterclaim for damages for breach of contract is that the claimant did not observe the security conditions attached to the card and so made it possible for it to be used fraudulently.

In such cases at least one disputed transaction has taken place. The question before the court is which of four possible explanations is the most likely.

• 1
  A thief has stolen the money from the bank following a breach of the card's security conditions. The claimant is at fault.
• 2
  A thief has stolen the money without a breach of the card's security conditions. The bank is at fault.
• 3
  The claimant was responsible for the withdrawals and is making a dishonest claim. Clearly the claimant is at fault.
• 4
  The bank has made an error and so is at fault.

In a recent case (Job v Halifax PLC) in England (England and Wales, Northern Ireland and Scotland are the three separate judicial systems for the UK) the claimant had a total of £2100 taken from his account in eight withdrawals. He stated that he had neither made them himself nor authorized any third party to do so, but his bank refused to compensate him. The judge ruled in favour of the bank, saying that in the absence of a history of successful fraudulent attacks on chip and PIN transactions, and of any evidence of systems failure, the transactions could be taken at face value, and the claimant's card and PIN could be assumed to have been used.

A SURVEY

Feeling that the judge's statement merited further investigation, the authors conducted a small-scale survey. The participants were post-16 mathematics teachers at the annual Mathematics in Education and Industry (MEI) conference in the UK in 2009 (further comments on the sample are given at the end of this article); they provided a textbook opportunity sample. Eighty out of 250 questionnaires were returned.

The survey was based around people's experience with their banks (and building societies). These institutions use software designed to detect suspicious transactions; when such a transaction is detected, they contact the card holder to check whether the payment should be made. The survey asked people if they had been contacted by their banks in this way or if unauthorized withdrawals had been made from their accounts. The outcomes are summarized in table 1.

Table 1.  Summary of questionnaire outcomes

Transactions with bank query			Transactions without bank query			Totals
	People	Cases		People	Cases	People	Cases
Transactions queried	46	158
Unauthorized transactions	11	19	Unauthorized withdrawals	21	23	23	42
No security breach had occurred	9	16	No security breach	13	13	16	29
A security breach had occurred	3	3	A security breach had occurred	9	10	9	13

Regarding the outcomes given in table 1:

• • 
  Where the numbers in the “People” rows and columns appear not to add up, it is because some people feature in more than one entry.
• • 
  Only card transactions from January 2006 onwards were considered, post-dating the introduction of chip and PIN technology in the UK.

Information from the survey makes it possible to say something about the probabilities of the four explanations. The starting point must be that a disputed transaction has occurred. It is then necessary to consider the relative probabilities of the various possible explanations. Miscarriages of justice have occurred, for example in cases in England involving cot deaths, because courts have equated the probability of someone being innocent with the probability that the event occurred in the first place rather than with the probability of an innocent explanation, given that it has occurred. This error is known as the “Prosecutor's Fallacy.”

The first two explanations of what occurred presuppose the transaction was definitely a fraudulent attack. The survey identified 42 such attacks. Thirteen of these attacks could be explained by security lapses and 29 could not. So the data would suggest that given that an attack has occurred, the conditional probabilities of the two explanations are (security breach) and (no security breach). A security breach is the fault of the claimant and so a just outcome would be in favour of the bank. However, the opposite is true if there has been no security breach; the fault lies with the bank and so the case should be decided in favour of the claimant.

COURT VERDICTS AND HYPOTHESIS TESTS

In civil cases a judgement is based on “the balance of probabilities,” but what does this mean? Taken literally, it would imply that the judgement in such cases should always be made in favour of the claimant (assuming the data from the survey are reasonably representative), but that is not what happens in practice.

It is instructive to compare what happens in a court of law with carrying out a hypothesis test.

Criminal cases

In a criminal case, where the presumption of innocence applies, there is a clear null hypothesis:

• • 
  H₀: The accused is innocent.

A hypothesis test is carried out at a stated level of significance, and this may be described as “We can't be certain about rejecting the null hypothesis but there is a probability of being wrong that we are prepared to accept and that is the significance level.” In a court case the term “beyond reasonable doubt” conveys much the same idea.

Just as in the case of a hypothesis test, there is the possibility that the verdict of a court is incorrect, leading to either a Type 1 or a Type 2 error, as illustrated in table 2.

Table 2.  Outcomes of a criminal court case

H0 : The accused is innocent	Court convicts	Court acquits
Really innocent	Incorrect verdict Type 1 error	Correct verdict
Really guilty	Correct verdict	Incorrect verdict Type 2 error

Civil cases

In the situation described at the beginning of this article, the court was asked to decide between two parties, the claimant and the bank. This was a civil case and so presumption of innocence could not apply to either party. The court had to be strictly neutral between the claimant and the bank.

In a civil case there is no single null hypothesis. However, the analogy with hypothesis testing is still helpful. Think about a case involving two parties, A and B. Two null hypotheses are effectively being tested simultaneously, each with an alternative hypothesis corresponding to a one-tailed test.

• • 
  H₀(A): Party A is in the right, H₁(A): Party A is in the wrong
• • 
  H₀(B): Party B is in the right, H₁(B): Party B is in the wrong

Some of the evidence presented to the court may allow an estimate to be made of the probability of each of the two parties being in the right. The situation is illustrated in figure 1, with a significance level of S.

Figure 1. Balance of probabilities

In the example in this article, the survey suggests probabilities of that the claimant was in the right and for the bank. Even with a significance level as high as 20%, this would be well into the indeterminate grey region in figure 1 and so outside either of the critical regions. So the survey data would not, on their own, support a balance of probabilities argument in favour of either party. Other evidence would be needed, and the nature of the evidence will depend on which party had the burden of proof on a particular point at issue. Generally, a party discharges its burden when the judge can say it is more probable than not. If the probabilities are equal, the burden has not been discharged.

Situations where no data are available to a court are common, and so when the term “probability” is used it may be an expression of belief rather than the outcome from any calculation. Such an expression of belief is necessarily subjective, and different people are likely to assign different numerical values. This is close to a Bayesian interpretation.

THE OTHER TWO EXPLANATIONS

The claimant is a fraudster

The survey also allows something to be said about the third of the four explanations, that the claimant was responsible for the withdrawals and was making a dishonest claim, deliberately trying to defraud the bank.

The 29 cases of unexplained attacks came from the 80 people who returned the questionnaire, and they were reported by 16 people. This would suggest a probability of that a randomly selected individual has experienced an attack. However, there are two problems with this estimate.

• • 
  Many of those who did not return the questionnaire may have felt they had nothing interesting to report. If this was the case for all those who did not return the questionnaire, the probability would be reduced to .
• • 
  The claimant is not a randomly selected individual but one of a very small group of people involved in such cases.

The importance of this probability is not its actual value but that it is not zero. Such attacks can happen, and so it is entirely possible that the claimant is telling the truth. This is the extent of the interpretation that is possible from this figure. Whether a court judges the claimant to be telling the truth must depend on other evidence.

The bank is at fault

The survey provided some insight into the final explanation, that the bank is at fault. One respondent wrote:

We went to the bank and spoke at length with the manager. We were fully reimbursed and had a grovelling apology. (The sum involved was substantial, and was removed twice by different “managers” leaving our account frozen.)

This one example shows that the banks are not infallible. To test a bank being at fault in a court of law, the claimant will need to obtain disclosure of a variety of documents including, but not limited to, the bank's systems and procedures.

WHAT HAPPENS NEXT

The survey went on to ask those who had had unauthorized withdrawals what happened next. In nearly all cases the bank had refunded the money; one respondent described the experience in these words.

The bank described two transactions in the space of 3 or 4 hours. One for about £40 in a Marks and Spencers in London and the other for over £500 at an expensive restaurant/club in London. I was in Paris at the time of these transactions. The bank refunded both amounts after I filled in a form. . . . I assume that someone had managed to clone my card somehow.

However, in one case the bank had not given a refund. In such a situation, the next course of action open to someone in the UK is to contact the independent Financial Ombudsman. In 2009 the ombudsman investigated nearly 20,000 cases; with such a large number some incorrect judgements are to be expected. A few of these cases may become the subject of legal proceedings.

However, the decision to initiate legal proceedings is fraught with anxiety: by the time legal action is contemplated, much of the evidence might have been destroyed by the card issuer; finding a lawyer who is familiar with digital evidence is exceedingly rare, even in the twenty-first century, and the most effective deterrent of all is the fear of being liable for the other party's costs where a case is shifted out of the small claims track owing to the complexity, thereby depriving the claimant of the shield against costs orders.

It is worth noting that the survey data came from teachers of advanced mathematics. They are a highly coherent group of people, better able than most to present their case to a bank. There is a danger that some people do not get a refund from their banks because they do not express themselves well, and that the same could be true if they go to the ombudsman. (One respondent spoke of a friend in just this situation.) So it is possible that the survey data give the impression that fewer people are victims of injustice than is actually the case.

THE BANKS

Clearly such fraud can cost the banks a lot of money. It is not surprising that they go to considerable trouble and expense to detect unauthorized transactions and so prevent incorrect payments being made.

The survey provided information about the success of detection, and this is given in table 3. The total number of transactions for those who answered the questionnaire over the 3½ years has been estimated as 140,000 (i.e. 500 transactions per person per year, a little under 10 a week); this figure and those derived from it are given in parentheses.

Table 3.  Outcomes from banks querying transactions

Transactions	Authorized	Unauthorized	Total
Queried	139	19	158
Not queried	(139,819)	23	(139,842)
Total	(139,958)	42	(140,000)

Table 3 illustrates the problems faced by the banks. They check a large number of transactions, query quite a small proportion of them (in this case about 0.1%) and succeed in stopping some unauthorized transactions. However, despite all this effort, the data from the survey suggest that they only catch about half of the attacks.

The outcomes of the test that a bank applies can be described using the terms in table 4.

Table 4.  Description of the possible outcomes of a bank's test

Transactions	Authorized	Unauthorized
Queried	False positives	True positives
Not queried	True negatives	False negatives

In this description, a “positive” is a transaction that the bank's test has identified as suspect. The identification is “false” if the transaction was in fact authorized and it is “true” if the transaction was unauthorized. Similarly, a “negative” is a transaction that the test has not identified as suspect, and this non-identification may be true or false.

Transactions are checked by computer software that is designed to give a warning when a possible attack is detected. If the software gives a warning when there is no attack, a false positive results; 139 of these are recorded in table 3, and apart from some inconvenience, they are quite harmless. However, there is a cost to the bank because members of staff spend time contacting account holders.

By contrast, if the software fails to give a warning when there is an attack, a false negative occurs, resulting in unauthorized withdrawals. These are the serious cases; there are 23 of them in table 3.

The number of false negatives can be reduced by making the warning criteria in the software more severe, but the effect will inevitably be that the number of false positives rises; the more severe warning criteria will pick out more authorized transactions. Thus the fewer the false negatives, the greater the number of false positives and the associated staff costs. This mirrors the familiar trade-off between Type 1 and Type 2 errors in statistical hypothesis testing.

In an ideal world a bank would aim for zero false negatives. However, the reality is that a point is reached where the cost of further reducing the false negatives is greater than the additional savings to be made. Unsurprisingly, banks do not divulge information about their detection policies; they are a major element in an ongoing struggle with criminals, each trying to outwit the other. However, because of the cost involved, it would be surprising if many banks pursue a policy of reducing false negatives to zero.

SAMPLE SIZE

The survey described in this article involved a small and non-random sample. However, neither point affects the major findings.

• • 
  Card fraud does occur and ordinary people are victims of it.
• • 
  Card fraud is not always a consequence of a breach of card security.
• • 
  When a case involving an unauthorized withdrawal comes to court, it is not valid to use a balance of probabilities argument, without other evidence, to decide whether or not there had been a breach of card security.
• • 
  Bank errors do occur.
• • 
  The software that banks use to check for suspect transactions is only partially effective.

The survey provided evidence to support these findings, and in some cases this was backed up by estimated probabilities. However, in none of these cases are the actual values of the probabilities particularly important. What matters is whether or not they are zero or, in the case of the balance of probabilities argument, reasonably close together.

A further important point is that the situation is not static. Criminals are continually developing new techniques, and the banks are finding ways of countering them. So the idea that there are accurate probabilities to be found is a myth; they are changing all the time.

ATINER Conference

The Athens Institute for Education and Research (ATINER) will be holding their 7th conference on Mathematics from the 17th to the 20th June 2013. See http://www.atiner.gr/mathematics.htm for more details. The key date to note at the moment is the abstract deadline of 19th November 2012.