Friday, September 2, 2011

How do you read the literature? Thoughts on academic maturation

How much should you trust the scientific literature? Reflecting on my own academic maturation, as well as observing on-line discussions of dinosaur paleontology for over 15 years (yikes, I'm getting old!), I have concluded that most of us pass through three stages: 1) Credulity; 2) Cynicism; and 3) Maturity.

This is inevitably one's first stop on the journey through the scientific literature: accepting everything that's published at face value. Credulity is also paired with the assumption that the most recent publication must be the most conclusive. For instance, let's say Dr. X described a new species in 2001. Dr. Y published a new paper in 2010, saying that the new species is invalid. Dr. Y must be correct, because she had the last word, right?

Another symptom of this stage is fanboy(girl)-ism. Anything published by Dr. Glamour is the bee's knees (it's widely featured in the news media, so it must be true)! Wow, Dr. Glamour published a new theory on the dinosaur extinction - it will revolutionize the science! Any nay-sayers are just jealous, or afraid of change.

I hit this stage during high school and college.

Suddenly, everything comes crashing down. You talk to another paleontologist, who tells you that Dr. Glamour's work isn't actually that highly regarded. Maybe he has a reputation for massaging his data just a little too much, or conveniently omits contradictory evidence in his papers. Then you find out that Dr. Z has just published a paper saying that Dr. X was actually correct in the first place, and Dr. Y's synonymization was a little too hasty. Your obvious conclusion: the scientific literature is untrustworthy. Everything ever written is a steaming pile of unreliable ramblings.

Most people don't go through a full-blown case of cynicism, of course. Usually we just get an incomplete case. Everything written by Dr. Glamour (but only some of the stuff by Dr. Y) is untrustworthy, etc. A related syndrome focuses on the methodology; a paper is considered horrible because it used or didn't use a particular technique.

I hit this stage between the end of my undergrad and the early to middle parts of my graduate career.

Most of us reach this stage only after a lengthy amount of time in the field (or the end of our graduate student career). Our BS detectors are honed to an appropriate level, and we accept that many of the papers out there aren't half-bad, and a minor mistake or two isn't enough to relegate research to the dustbin.

For my part, I still occasionally waver between cynicism and maturity; I might cast an exceptionally suspicious eye on research coming from certain researchers or using certain techniques (even if it's not necessarily warranted). Maybe I even have a little credulity at first, if it's a technique or area of science I'm not yet completely familiar with. At the same time, having been around the block a few times as a scientist, I am a little more understanding when it comes to the perceived shortcomings of a paper. As long as the basic science is still good, live and let live. A paper can have a fantastic morphological description, but a pretty weak discussion. With a little practice reading the literature, it's becoming easier and easier to pick up on the high and low points of a publication.

Summing it up
We all relate to the academic literature in different ways, depending on our life experience, scientific goals, and "academic maturity." It's up to us - with the help of trusted friends and colleagues - to continually work to improve our own approaches.


Jay said...

I think you're right Andy, as i feel I can relate to passing through these stages....or am I just showing credulity by accepting this? ;)

Suppossing not, i would guessed myself not to have reached staged 3 yet, but wavering between 1 & 2 depending on case.

Erin said...

Scientific literature is growing and one must know where he is. As learning depends on 'academic maturity', I'd say 'start early'.

Mike Taylor said...

I want to know who your "Dr. Glamour" was -- Bakker?

(I appreciate the good, British spelling of Glamour, by the way.)

Andy said...

I had many, many Dr. Glamours. . .I won't name names in this public forum, though ;-) (as I actually like and respect many Dr. Glamours as colleagues!).

Heinrich Mallison said...

hehe, yeah, there is more than just a kernel of truth in your post. But you should also mention that there are people out there for whom maturity is the same as cynicism: folks who are known to publish more junk than good stuff.

Karen said...

I'm not sure what stage I am... I read an article and usually wish I had more background knowledge better be able to assess the article. But there's no time to do the background reading...

Karen said...

I also admit to having a great deal of respect for Dr. Writer. His science was sometimes a bit speculative, but if I could write like him, I'd be in heaven.