Chapter 2 Scientific Literature
This chapter explains how to find, read & cite scientific literature.
How do you find reliable information? How do you even know that information is trustworthy? And once you’ve found something, how do you communicate it in such a way that others can easily find the original source?
2.1 Finding Scientific Literature
There are many good databases that you can consider to find peer-reviewed scientific literature. Four popular ones are shown in table 2.1. My general advice is to access through Leiden University. This allows you to access articles from hundreds of publishers that might otherwise be behind a paywall.
| Database | Stengths | Weaknesses |
|---|---|---|
| Google Scholar | Very easy to use. | Search results not reproducible; limited to 1000 articles; includes non-peer-reviewed publications. |
| PubMed | Strong focus on clinical and biomedical literature. | Mostly limited to clinical and biomedical literature. |
| Semantic Scholar | Very broad, includes all kinds of research. Has an easy search function and features to find related articles. Tries to search for what you mean (semantic). | Does not allow you to search for specific combinations of words in specific parts of an article. |
| Web of Science | Very broad, includes all kinds of research. Has a very precise search function. Tries to search for what you asked (literal). | Does not try to interpret what you meant to search for. |
Three important things to note here:
- There is no “winner”. These databases each have their own use.
- Which database you prefer, may differ depending on what you’re doing (e.g., learning about a new topic, investigating a health claim, conducting a meta-analysis).
- The table shown here is a simplistic summary of the findings of a comprehensive comparison by Gusenbauer & Haddaway (2020).4 If literature review interests you, I highly recommend their article.
Personally, I mainly use PubMed, because most research I am involved in concerns cellular, molecular, and medical biology. However, for conservation, evolution, ecology, etc., PubMed really falls short, and better alternatives would be Web of Science or Semantic Scholar.
Think of Google Scholar like you would Wikipedia: It is quick and easy to use, and a great resource to get acquainted with a topic. But for actual research, Google Scholar is rarely appropriate, because it yields irreproducible search results.4–6 That is to say, you and I might find different articles using the same search terms. Personalized search results are antithetical to objective research, so use Google Scholar with caution.
2.2 Types of Scientific Literature
The main types of articles are shown in the table below:
| Type | Description | Use |
|---|---|---|
| Original research article | New data produced by a study and the conclusions drawn, usually with a narrow scope. | Find potential explanations for a specific phenomenon; Cite the original source for an observation made. |
| Narrative review | Current understanding of a subject, usually written by an expert, usually with a broad scope. | Get introduced to a new topic; Read up on the latest developments in a field. |
| Systematic review | A comprehensive review that explicitly states how studies were included, usually with a narrow scope. | Get high-quality information compiled from multiple studies in a transparent, reproducible manner. |
| Meta-analysis | A systematic review with a statistical analysis of the combined results of included studies, usually with a very narrow scope. | Get quantative measures of the current consensus on a subject and the extent of publication bias. |
Other types include pilot studies, letters to the editor and methodological studies.7
When you don’t know much about a topic yet, it is best to start with articles that have a broad scope. Narrative reviews (often simply referred to as ‘reviews’) are best suited for this purpose. Narrative reviews are subjective. They represent the findings and opinions of the author(s). Often, a narrative review will disclose little, if any information on how studies were included—a problem that makes this type of review susceptible to cherry picking.8
If you want an objective review of a certain topic, you should look for systematic reviews. These must adhere to strict quality standards to avoid cherry picking and more generally, to be reproducible.9 In principle, if you were to write a systematic review using the same database(s) and selection procedure, you should obtain the same results. The specific differences between narrative and systematic reviews are explained in more detail in Callcut & Branson (2009).8
Besides cherry picking, reviews are susceptible to another kind of bias. Publication bias occurs because studies with significant results have a greater chance of publication. Hence, studies with insignificant results are underrepresented in literature, and any analysis of published literature will overestimate effects and differences. This type of bias cannot be avoided, but it can be quantified in meta-analysis,10 and even corrected for.11 If it is found to be low, the conclusions can be stated with greater confidence, and vice versa.
Ideally, you would cite a high-quality meta-analysis comprised of a large number of studies, with minimal publication bias, for each claim you make. In reality, you have to make do with what’s available and adjust the confidence of your writing accordingly. Meta analyses are incredibly time-consuming and hence often limited in scope. Systematic reviews are generally slightly broader in scope but provide less concrete information on the current level of evidence. Narrative reviews are inferior in terms of evidence, but in return usually have a very broad scope.
2.3 Efficient Reading
A scientific article is not a log or a lab journal. Rather than written chronologically (in the order experiments were done), articles are typically divided into sections, with strict rules on what goes where (introduction, methods, results, discussion).
This is important, because it standardizes what information can be found where. However, it also means you don’t read an article from beginning to end. More realistically, literature review looks something like this:
.](figures/FlowchartReading.png)
Figure 2.1: How you rougly go through an article during literature review. Questionable research practices are explained in the next section.
In the most pessimistic case, where you know neither the topic, nor the methodology, you’ll be going through most of the article. But if this is your third article on pancreatic cancer, do you really need to another introduction that explains the relevance of this type of research? More articles are being published than you can ever hope to have the time to read, so I strongly suggest to use figure 2.1 as a guideline.
Quickly going through an article is a skill that takes time and practice. The first couple of articles, you will constantly be looking up terms and going back and forth between sections. But by the end of your bachelor internship, you will have seen so many articles that reading one has become a fairly trivial task. Of course, there will always be easier and harder articles to read, and especially an article outside your area of expertise takes longer to understand. If an article is particularly difficult to read, that is often not a reflection of your reading skill, but rather of the quality of the writing. More on that in the next section.
2.4 Judging the Quality of an Article
It can be hard to confidently state an article is of high quality, especially when you haven’t read many yet. Fortunately, identifying a poor quality article is a lot easier:
- Spelling errors;
- Low resolution figures;
- Flaws in analysis;
- Obscure or predatory journal;
- Questionable research practices.
1 & 2 are self-explanatory. 3 requires some basic knowledge of statistics, which you will learn later on in your education. Below, I will discuss 4 and 5.
Obscure journals are very easy to identify, by looking up the impact factor of the journal. In short, a lower impact factor means this journal is not cited often. That does not necessarily imply poor quality, but it does warrant more careful inspection of quality. Also be wary not to blindly trust an article simply because it is published in a high impact factor journal, like Nature (that would be an appeal to authority). Poor quality articles will occasionally slip through the cracks of the peer-reviewing process.
Predatory journals deceive scientists into publication for quick financial (or other) gain.12 These often have poor, opaque quality review, meaning that they are littered with articles that would be rejected in other journals.
Questionable research practices (QRP) is a collective term for practices in scientific research that are likely to result in bias, false conclusions, or difficulty to replicate the results.13 The most obvious, deliberate forms of scientific misconduct are plagiarism, fabrication and falsification, but there are many more subtle problems with how some research is conducted.14 Common QRPs are summarized in table 2.3. It is worth your while to explore the full extent of common errors (including unintentional ones) in the field of research you are conducting literature review on. Identifying one or more QRPs in a scientific article should immediately lower your trust in any conclusions drawn by the authors.
| QRP | Description |
|---|---|
| overclaiming | Placing too much confidence in a finding given the sample size. |
| overgeneralizing | Generalizing the conclusions to a larger population than the one sampled from. (For example, a study on Dutch biology students in Leiden with a conclusions about ‘students’.) |
| lack of a control group | An effect is asserted without including a proper control group in the study design. |
| inappropriate control group | A comparison between groups that systematically differ in more ways than those compared (e.g., comparing young men to old women.) |
| cherry-picking (citations) | Selectively citing literature in agreement with the authors’ views or hypotheses. |
| cherry-picking (data) | Omitting data not in agreement with the hypothesis, or the rest of the data. Deleting outliers. |
| cherry-picking (results) | Only mentioning significant results in the conclusion or abstract, without mentioning negative findings. |
| HARKing | Basing the ‘hypothesis’ in the introduction on the results found in the data. (Hypothesizing After the Results are Known.) |
| p-hacking | A variety of practices aimed at changing the analysis, the data, or the data collection, until a significant result is found. Often occurs unintentionally. |
Occasionally, articles are written specifically to help the reader learn to judge the quality of articles.15 You can always try and search for these.
2.5 Citing Literature
When you cite literature, you have to cite the original source. That means that if you want to cite something you read in a review, you have to look up the article that the review used for this claim. The exception to this rule is when you want to cite the overarching conclusions made by the review itself—in this case you cite the review.
2.5.1 Reasons for Citing
Important reasons for citing include:
- To provide evidence for a claim;
- To refer the reader to further details;
- To give proper attribution to the work of others.
The first two reasons are self-evident, but the third is just as important. Quoting, or even paraphrasing other literature without referring to it, is a form of plagiarism. This includes articles of poor quality, and even unpublished work. If you used it, you have to cite it.
Citations are not needed for:
- Original ideas;
- Claims that are easily observable;
- Claims that are considered common knowledge within your field.
For example: This page may help students learn to cite,\(^\text{no reference here}\) most people use smartphones on a daily basis,\(^\text{nor here}\) and smoking causes cancer.\(^\text{nor here}\)
2.5.2 Citation Style
When you publish an article to a journal, you must adhere to the citation style of that journal. If you are not publishing to a scientific journal, you can use whatever style you like. For example, this page is using the style of the journal Nature.
Instead of learning a specific style, I recommend learning how to automatically generate citations with a reference manager. This will also save you the arduous process of manually writing and formatting all your references.
Some options are:
- Use a reference manager with a free version, like Zotero or Mendeley;
- Use a paid reference manager, for which you have a discount at Leiden University, like EndNote;
- Keep your own list of references (see below).
For documents written in RStudio, like analyses, or even this website, you can manage your references within RStudio without additional software. If you’re interested, I explain this in the video below:
Assignment 2
- Choose a topic within biology of your liking;
- Find at least one article you think is of poor quality;
- Find at least one article you think is of good quality;
- Explain why you think these articles are good or not (min. 400 words);
- Add a reference section under your assignment where you refer to both articles in the style of a journal of your choosing. You are of course allowed to use a reference manager.
Upload your assignment to Brightspace as a Word or PDF file. Include your full name and student number. Deadline: 3 December