Impressively, in the last few days I’ve been chided both for failing to recognize the divine jazz-like looseness of blogging and for failing to recognize the requisite statistical rigor of blogging. The former accusation is harmless enough; the latter is the same argument used since time immemorial to wave aside cultural criticism on the whole — indeed, any knowledge that isn’t quantitatively generated — and that’s a problem.
It’s impossible to gather enough data to meet statistical standards while understanding the context that generated each point in that data. That’s the purpose of statistics in social science: to create discrete phenomena out of messy reality in an attempt to generalize where generalization is helpful. But generalization will never get you more than a few good options for what circumstances generated that data, a few strong correlations (which, the statisticians so ceaselessly remind us, is not causation). If you want to talk about not just the what but the why, you need to venture into conjecture anyway.
And conjecture based on those same generalizations doesn’t draw from nearly as deep a knowledge base as knowledge based on more comprehensive understanding of particular situations. Maybe those situations are anomalous — though in this case, the fact that I was using personal experience to illuminate a broader statistical study whose results lined up with that experience indicates that “fluke” is putting it strongly — and they’re certainly statistically insignificant. But until we can invent statistics to explain connections we don’t yet know exist, we’ll never be able to use them to explain social forces and cultural trends, only to record them after the fact like starlight on an Earth-bound telescope. Cultural criticism is by its nature tentative and speculative and flexible, but to say you can only explain a phenomenon after you’ve zoomed out too far to really examine it is missing the point entirely.
This is particularly appreciated with The Flaming Libs discussing something along the same lines today, too, inspired by the XKCD cartoon, but on the wrong side.
First, as much as I truly pity those that prize empiricism above creativity or think the only way we can talk about understanding the world is through proven facts, I don’t think you’re giving statistics their due. In fact, you’re vastly underestimating and completely strawmanning statistics pretty irresponsibly.
Yes, to start thinking about why not just what, you need to start in the land of conjecture — but while you can tell me a great story for a lot of things that aren’t true, well-done statistics do show causality and can describe the here and now, even many aspects of culture. Also, statistics does often tell us things we didn’t know before, it’s not just a method of verification. Finally, statistically based knowledge is without a doubt richer — when you abstract 50 levels from an original hypothesis proving the empirics of every step along the way, your 51st step is much more valuable than if you’ve simply sat there talking.
My usual line of argument in these situations is that numbers can’t describe everything we care about, which you do make towards the end. But I can’t get on board with the rest.
Points I will concede:
-”You can tell a great story about a lot of things that aren’t true.” Absolutely, and I struggle with it. But sometimes it takes hearing a story told at this scale to realize that what’s being described is true, because it resonates, even when you didn’t understand or see it before. (This is particularly true when moving from “Statistics tell me I’m probably X” to “oh, wow, this is exactly what happened to me!”
-”Statistics does often tell us things we don’t know before.” Yeah, I overstated that point.
-”Statistically based knowledge is without a doubt richer.” Than knowledge that could be statistically based but isn’t? Sure. Than knowledge that couldn’t be verified quantitatively if we tried? Ehh, apples to oranges (and this gets to the “anything we care about” thing).
The only time I wish I were able/had to put a subject, which would read, “Agreed”.
1. Yes
2. Yes
3. Very Yes
The only thing I’d add on point three is that I think there’s a lot lot lot of areas where people assume empiricism can’t belong but where it can yield real insight, even if it’s only as a component of a larger theoretical undertaking.
P.S. “It’s impossible to gather enough data to meet statistical standards while understanding the context that generated each point in that data.” is completely and utterly untrue.
Counterexample? Preferably one that actually honors the context of the quote, i.e. social forces and/or causal chains?