Quantifying Citations

One of the goals that I had in revising my book manuscript over the past year was to cite more women authors.This was partly in response to critiques from peer reviewers, but also because it is the right thing to do. Citational politics is part of academic life and the growing interest in quantitative assessment (various indices, impact factors, and so on) means that it’s not just about appearances and giving credit where it is due, but it also has direct financial consequences. 

The distribution of citations in my original manuscript was pretty disappointing with only 35% of the works in the bibliography having at least one woman as author and 77% having at least one man. This is obviously not what I set out to do when I started writing this book and it appears that an assessment of the book as very white and very male was a fair one and one that I took to heart.

After I made a series of substantive revisions over the last year, I was excited to run the numbers on my bibliography again and see whether my revisions improved the situation.

Sadly, they did not. While I increased the percentage of references with at least one woman as author to 40%, I also increased the number of references with at least one man as author to 84%. 

This was pretty demoralizing to realize. When I dug deeper into my numbers, I did notice some reasons for optimism.

First, for citations dating to 2020 or later, 48% of the citations have at least one woman author and 67% have at least one man. 

For citations dating to 2015 or later, this number stays roughly stable with 46% of my citations having at least one woman author and 67% having a man.

For citations since 2010 and 2000, the percentage of references with at least one woman author stays relatively stable at 42% and 40% respectively and 73% and 70% respectively for references with at least one man as an author.

References dating to before 2000, however, are a shit show with merely 13% of the references including at least one woman and 91% including at least one man. Some of this can be attributed to the outsized place that Bill Rathje and Michael Schiffer have in both archaeology of the contemporary world and my book, but even then, these numbers are ghastly.

This quantitative work has taught me three things:

First, over the past decade there has been a good bit of conversation about structural biases and inequality. My bibliography is a depressing example of this. Even as I honestly tried to shift the balance toward more work by women, historical traditions of practice in my discipline continue to keep a firm thumb on the scale and my own reading and writing practices. 

As my book manuscript goes out once again for review over the next few weeks, I reckon I will have one more opportunity to work on my citation practices and will continue to try to work to redress what is clearly a shortcoming in my book. 

Secondly, if and when the book is accepted and typeset, I hope that I can do some more sophisticated analysis of the content of the book. After all, it is easy enough to pepper one’s work with some throw away references as a way to shift a bias one way or another. And, of course, this isn’t entirely superficial as various automated reference searches (e.g. Google Scholar) don’t care whether the citation is a “see also” or part of a more in-depth discussion. As institutions look toward i10 and H -indices as measures of a scholars reach and impact, these numbers matter.

On the other hand, a five page discussion of a work may only garner a handful of citations in the text and may only result in a single bibliographic entry. This is particular true for dissertations where authors don’t have as substantial “back catalogue” of work that warrant referencing. I hope to come up with a systematic way to measure how much of my book is devoted to various authors, but since this will be a pretty arduous task, it might make better sense to do this at after the book is typeset and when the final references are established. 

Finally, I still intend to make this data available and include an appendix to my book discussion what I did and what I had hoped to do.

As a start, here’s a copy of my bibliography from which I collected the data discussed above.

And you can read some of my earlier writing and thinking about citations: here, here, and here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s