I’ll start this post with a disclaimer. I’m not a mathematician. My dissertation involved some statistics, and I knew enough to get help from someone more qualified than me. I learned a lot about how statistics really work from that experience, most notably that I shouldn’t trust my impressions about how the numbers really would end up. Our intuitions can fool us into thinking we’ve found a pattern when one doesn’t really exist.
Let me offer a hypothetical example related to brass pedagogy. Let’s say you’re a brass teacher working with beginners and you notice that out of your ten students, three improve with a flat chin embouchure while the other just don’t seem to make it work right. You find it curious that only 30% can make a flat chin work (after all, that’s what you’ve been told is correct), so you write a method book and point this out. A graduate student looking for a research project for his thesis reads your book and decides to put together a study to check this. He tests 90 brass players and finds that 45 of them improve more with instruction that recommends a bunched chin. Continue reading