So you think you have all the data? Causes and consequences of selection bias

Abstract – There is no doubt that modern data science holds tremendous promise for improving the human condition. However, substantial challenges must first be overcome. This talk addresses one of these challenges – that of selection bias in large data sets. Failure to acknowledge and tackle selection bias means that mistaken conclusions can be drawn. The situation is often further complicated by the fact that selection bias is often concealed, and may not be apparent from the data. This talk gives examples from the wider data science literature, and also some new examples from my own work. Statistical tools for tackling selection bias are also described.

Biography – David Hand is Emeritus Professor of Mathematics at Imperial College, London. He received a BA in mathematics from Oxford University, and an MSc and PhD in statistics from Southampton University. He has served as President of the Royal Statistical Society and of the International Federation of Classification Societies. He is a fellow of the British Academy, the Royal Statistical Society, the Institute of Mathematics and its Applications, and an Honorary Fellow of the Institute of Actuaries. He was awarded the Royal Statistical Society’s Guy Medal in Silver in 2002 and the Credit Collections and Risk Award for Contributions to the Credit Industry in 2012. He is a non-executive director on the Board of the UK’s Statistics Authority, chairs the Board of the UK’s Administrative Data Research Network, and serves on the European Statistical Advisory Committee.

David is the founding editor of Statistics and Computing, and previously edited JRSS-C: Applied Statistics. He has published 300 scientific papers and 28 books, including Principles of Data Mining, Information Generation, Measurement Theory and Practice, The Improbability Principle, and The Wellbeing of Nations.