Statistics Ad Absurdum

13 Jul

In the 1970s, orchestras began using blind auditions, playing behind a screen so the jury that cannot see them. As a result, the number of women in orchestras increased five fold… except they didn’t. That “fact” was vastly exaggerated.

Memory is a funny thing – the first factoid you hear often sticks and becomes truth, whether not it’s disproven later. What amazes me is how often that “fact” gets blown out of proportion. Take the “blind audition” story; I had never heard this until I took our DEI (diversity, equity, and inclusion) training at work.

Side note: My workplace does this training right. Compared to the horror stories I’ve heard, they’re not trying to shove a political agenda, or punish you for being white… just ways to be civil to each other. The only part that is “dicey” is the gender expression and identity section, but to be fair, we’re all trying to figure that out. How are we supposed to teach it?

So I’ve gone through this training around ten times–not because I was being punished, but because I work in the department that trains this. So I was asked to “produce” this class (handle the technical problems for this online class) as a favor for another trainer. Therefore, I’ve seen the video where they mention the “blind audition” research as an example of gender bias about ten times. This time, I decided to fact-check this.

The research was called “Orchestrating Impartiality: The Impact of ‘Blind’ Auditions on Female Musicians” by Claudia Goldin and Cecilia Rouse, originally published in the September 2000 issue of the American Economic Review. You can read the original study here, but I’ll warn you, it’s 27 pages long. Even I didn’t feel like going through all of it, so if you feel like going through Andrew Gelman’s blog about it, he lays out the stats pretty well in a shorter form. The point is that they looked at nine different orchestras (the “Big Five” and four regional ones) and compared their hiring rates across fifty years of data, then saw how that changed when blind auditions were added. The end result? On p. 737 of the original study, before blind auditions, women were hired 10 percent of the time; after, 35 percent. So there was an increase… but certainly NOT five fold!

What I found more interesting–and Gelman doesn’t mention–is that the paper is 65 pages long. But wait, Marcus, you just said it was 27 pages long? The research is… the other half of the document is the 741 times (!) this article has been cited in other research. As someone who has written many, many research papers, this might explain the exaggeration. When you’re just referencing the results (like I’m doing now) in your own paper, you simplify. You don’t mention that the research found women were 25 percent higher to get to the next round of auditions, not hired. I didn’t mention it either… to make a point. When you simply copy the stats to make your point, it’s much easier to exaggerate.

So 25 percent becomes 35 percent, becomes fifty percent, becomes fivefold. Why? This is the telephone game writ large. Original research is read by other researcher and used in their research, which is read by a journalist who writes an article, which is read by a DEI trainer who decides to use it in a class. Along the way, it’s easy to forget the exact number. Or because 25 percent isn’t dramatic enough, you say 50 percent, although I’d prefer to believe that was simply the writer not going back to confirm their facts. Because people want to have hard facts to support your empirical view, that women tend to get hired less than men, and a slight increase doesn’t quite make your point, does it?

Now the research didn’t cover the numbers of applicants, rather they focused on the proportion, because “symphony orchestras do not vary much in size and have virtually identical numbers and types of jobs.” (p. 717) They don’t hire that many people per year. It did mention that almost all harpists are women and that the New York Philharmonic (as of 2000) had 35 percent women. Is it possible that less women are going into those jobs? After all, women only comprise of 3.4% of all construction trades. Now is that because, like higher math or construction fields, you have to deal with a lot of pricks who hit on you or say “girls can’t do X?” Quite possibly. Music majors were the biggest pricks I ever met in college; my theory is that the less jobs available in your career, the more competitive they have to be. Female nurses comprise of 75% of the workforce, elementary school teachers has a higher gender bias towards women.

Is it possible that different genders are favor different types of work? That working nights as a violinist is not conducive to women who want to have kids? Or am I exaggerating? 🙂 Let me know in the comments below.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Tales from a broken doll

Short stories, poetry, musings and rambling.

Poteci de dor

"Adevărul, pur şi simplu, e rareori pur şi aproape niciodată simplu" - Oscar Wilde

O Miau do Leão

Uma pequena voz da Flandres

A Life's Journey

Little things matter 🌼


A dreamer, who loves to muse her world and penned it down✍️ Each words in this blog lay close to my soul🧡

Talkin' to Myself

I'm listening

Nature Whispering

From Sunset to Dawn

Riverside Peace

Discover how God works through his creation and Scripture to show us his love.

I didn't have my glasses on....

A trip through life with fingers crossed and eternal optimism.

Looking to God

Seek first the kingdom of God and his righteousness. (Matthew 6:33)


We may see things that we don't even imagine.

Decaf White

No Sugar


Mere khayal aap tak..

The Haute Mommy Handbook

Motherhood Misadventures + Creative Living

Hangaku Gozen

For we cannot tarry here, We must march my darlings

A Cornered Gurl

I am more than breath & bones.

%d bloggers like this: