Shrimp on a treadmill? A few years ago, you may have noticed a small uproar develop over research involving placing shrimp on underwater treadmills. The largest outcry involved claims of wasteful government spending, resulting in scrutiny for many researchers and research grants. The NPR has a great report on the situation if you’re interested in reading more.
So if you heard about that or got caught up in it in some way, which side is to be believed? Were the shrimp-on-treadmill studies wasteful or worth it? Here I’ll offer up some tips on checking for yourself what scientific studies mean for people and the world.
As an example for readers, I’ll use one of the papers published by the researchers who did the treadmill-shrimp study so we can look together to see if something presented as ridiculous might have bigger implications for society. I’ll put my example in a separate post.
Start with what you encounter first: journalism. Most journalistic pieces on science will relate whatever research is being talked about to the world at large. A news piece will usually have interviews with the researchers involved and one or more researchers in the field who were not involved that will discuss impacts of the study. This can range from obvious things like new technology and health benefits to studies that will advance the field (testing new methods or techniques, for example).
Journalism should not be the last step if you have further questions, though—especially if the benefits or implications of a study still aren’t obvious outside of one specific field. You have to move on to the paper itself (the following tips all assume you can access the full paper—in some cases, you will not be able to).
The two most important sections of the paper to read for broader implications are the Introduction and Discussion (or Conclusions).
The Introduction will start with broad context, explaining what field is being studied, questions and progress within that field, and often the relation of that field of science to society. It will usually narrow down in scope until it addresses the specific research questions of the study. In longer papers, you’ll get a much longer, more involved background while in short ones it may feel rushed and not include as much context, but don’t worry about that for now.
The Discussion will talk about the findings of the paper and what they mean in the context of the field and for science. Usually, the Discussion will take an opposite, mirrored effect to the Intro: broadening in scope until it finally addresses larger societal implications.
Most of your questions, therefore, will probably be answered by the researchers themselves in the Intro and Discussion. But what if you still don’t feel like it has much impact on society? You may have to dig into the background research.
Start with the list of references at the end of the paper. Go through the Intro and Discussion again, and wherever you see a citation (usually an author’s name and date or sometimes just a number) for a piece of information that seems like it could be part of the big picture, find it in the list of references and try to find that paper. With some background (detailing why the field was explored in the first place) or side studies (which link the research to something more relevant to society), you should find broader implications very quickly.
Sometimes this process may result in detective work where you continually find new papers to skim to try and broaden your scope, but by this point, you may be wondering if that original paper really means that much if you have to get so far removed to see the impacts.
The answer to this musing, of course, is perhaps it doesn’t. If a paper doesn’t seem to advance a field (by clearing up a definition, testing new methodologies, or investigating important hypotheses), perhaps it’s not particularly important. But this will rarely be the case. It shouldn’t take too much searching to find links between the paper and important, relevant topics (important and relevant to some sector of society, anyway).
A lot of what we view as important science is very subjective. Take space exploration, for example. Many people see it as crucially important to the existence of humanity and imperative to fulfilling our species’ drive for discovery. However, some see it as a waste. This stance seems impossible to me, but the views of people are as varied and diverse as the people themselves.
If you can’t see why a paper is important, before disdaining it try to have an open mind. Perhaps there is a perspective from which a study or bit of research is very important. Perhaps for some people, it is life or death, or will affect their livelihoods, or will save endangered species (another occasionally divisive topic which for some people is extremely important and worthy). If you want to practice at this, try looking for opinion pieces about the scientific field or find the webpages of the scientists involved in the study to get a better feel for how it affects their lives. Expanding your horizons before casting doubt and shame is always the best option in my experience.
There is a side note to this process as well: funding. In the case of the shrimp-on-treadmills study, most of the outrage seemed to stem from the fact that there was government funding involved. When it is a case of whether public funds are being put to good use, try researching the funding agencies. The primary federal scientific funding agency is the National Science Foundation. The NSF receives thousands of project proposals every year, selecting only about a quarter of them to which to allot funding. It has strict merit review criteria as part of its reviewing process. Specifically, from Chapter 4, Section A of the Proposal and Award Policies and Procedures Guide:
“When evaluating NSF proposals, reviewers will be asked to consider what the proposers want to do, why they want to do it, how they plan to do it, how they will know if they succeed, and what benefits could accrue if the project is successful. These issues apply both to the technical aspects of the proposal and the way in which the project may make broader contributions. To that end, reviewers will be asked to evaluate all proposals against two criteria:
- Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge; and
- Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.
The following elements should be considered in the review for both criteria:
1. What is the potential for the proposed activity to:
a. Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and
b. Benefit society or advance desired societal outcomes (Broader Impacts)?
2. To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?”
The NSF already evaluates what it gives funding to based on broader impacts to society, so your tax dollars (at least on the federal level) are most likely not going to waste.
I hope my tips have helped you and will continue to help you in evaluations of the merit of research. It can be hard, at first, to see why a hyper-specific scientific study might be important—science is usually incremental and diverse. But if you really want to know, the answers are likely already there.
To the reader: do you have tips or advice of your own for assessing how important a given study is to society? Are there any fields of science you think are wasteful or not worth publicly funding? Or the reverse: any fields (or specific studies) usually seen as obscure and pointless that you can relate to a bigger scale?