What Matters About How We Count

We tend to get what we measure, so we should measure what we want.

It's a bit of a strange feeling, but one I think I need to do more often - reading, and reading that challenges, or at least makes me more cautious about, the things that I'm more sure of. Counting: How We Use Numbers to Decide What Matters, is one of these such books. Deborah Stone shines a harsh light on one of the key assumptions in modern life: feelings aren't facts, numbers are.

As a Data Analyst, I use numbers - a lot. I've got to trust them, and I've got to make sure that I can tell stories with them. And it's these moments where you might think "numbers are numbers, so as long as you're correct mathematically, you're right".

But, even when you're correct in your sums and calculations, it's the counting that matters, and it's how we count that matters. As Stone says, it's not so much the calculations that can lead you astray, but the way in which society today takes for granted the decisions that go into including or excluding members from a set.

Particularly when working with people-related data at an organisational level, it's crucial to be aware of this. You don't need to question a number every time you use them, but you should question them. Who was included, and why? Who was excluded, and why?

And it's plainly not the case that statistics and analysis have no value. Entire systems of government and industries rely on it to make sure that things run as smoothly as possible - based on those assumptions.

So, it's really those assumptions that can be problematic. In the case of organisational data, you probably don't even have the opportunity to go back and re-count - once the numbers are in your data lake/base/warehouse/mart it's there, unquestionable because some time ago, a business analyst and data modeller sat down and decided on what counts and what doesn't. Who counts and who doesn't.

It sounds obvious but needs to be said: To know what a number means, you have to know what the counters included in their counts.

Counting was a pleasure to read - when it comes down to it, it's simply saying that numbers are unlike life in that life is full of subjectivity and ambiguity, it's got judgement calls and people with their own interests. It's entirely true, of course - life isn't math, and I'm alright with that.

But it would be a mistake to believe, after reading the book, that you should use numbers and statistics less to try to solve problems in the world. It would be a problem to throw statistics to the wind, and make decisions based on gut feel. After all, that's how we got to where we are. largely - people deciding to do things based on what they thought was right - rarely actually wanting to do harm - from their perspective, of course - but often motivated to maintain the status quo, which itself was (and is) problematic, or else to improve their lot in life, indifferent to the cost to others'.

The lesson to take away from the book, I think, is that we should all try to be more comfortable with interrogating statistics and data. Not with a view to discredit and disprove something because it would disadvantage us - or more likely, reduce our own relative advantage - or because, as the aphorism goes, there are lies, damned lies, and statistics.

Rather, the approach should be to try to understand better those figures in which we have a genuine interest - whether we want them to be right or wrong - with an honest curiosity. The numbers represent the same facts (yes, to belabour Stone's point, the same facts pre-counted and pre-sorted for your consumption) whether or not we understand it, after all.

Through understanding the numbers we encounter on a daily basis, though, we gain two understandings: what we consider important enough to count, and how we're counting them. Numbers are advanced, as Stone writes, in service of people's arguments. Asking more about the numbers arms us better than simply parroting numbers we want to believe are true, and trying to dismantle numbers we want to be false.