Whether for Journal Club, or while on the road of lifelong learning, being able to read journal articles is a core skill physicians must master. While it is tempting to merely read guidelines, summaries, or others’ analyses of articles, this will not help you become the best possible physician you can be. Choosing and evaluating articles in the context of your practice will allow you to determine how the conclusions apply to the patients you care for. So, what’s the right way to approach an article? In the immortal words of Julie Andrews, “Let’s start at the very beginning, a very good place to start.”

Choosing What to read

An often overlooked aspect of reading journal articles is choosing which ones to read. Obviously, in a Journal Club setting, it’s pretty easy: you read the articles that are going to be discussed that evening. In the rest of your academic life, how do you sort through the 2.5 million scientific articles published each year (Boon, 2017)? Obviously, the vast majority of these articles are not even remotely germane to emergency medicine and can be safely ignored. But, even looking at only quasi-relevant medical articles can become overwhelming quite quickly.

The first tactic you can use is to focus on relevant and high-impact journals. Within the world of emergency medicine, as you are undoubtedly aware, The Annals of Emergency Medicine is arguably the most important journal to pay attention to. But there are plenty of other journals that are likely worth paying attention to; The Journal of Emergency Medicine, The Western Journal of Emergency Medicine (WestJEM), and Academic Emergency Medicine are probably the core EM Journals to read. Subscriptions to these journals are typically available to you through memberships in professional societies at very reasonable costs for residents. Leafing through your journals when they arrive and, at minimum, reading the abstracts is a good use of your time.

Unfortunately, if you only look at the EM journals, you can miss out on interesting and useful articles. While you may not need to subscribe to BMJ, JAMA or The New England Journal of Medicine, you can typically get access to a couple of free articles a month for creating a free online account, as well as sign up for e-mail alerts for relevant articles. (Unsurprisingly, there have been some good COVID-related articles in these more mainstream journals.) It is also worth paying attention to the EM Blogosphere (we’ll discuss FOAM and blogs in a future post) to help you locate articles in journals that may not be on your typical radar.

Finally, when looking for what articles to read, be aware that most subspecialties / professional societies have their own journals as well, and if you have a niche interest, it is useful to subscribe to one of those journals as well. As an example, I subscribe the Prehospital and Emergency Care and Wilderness and Environmental Medicine to satisfy my niche interests. As you progress through your EM career, you’ll get a sense of what journals might be worth your subscription dollars. Whatever you decide to subscribe to, I would again recommend, at minimum, reading the abstracts of each issue.

Bear in mind that just because an article is in a reputable journal, that doesn’t necessarily make it high quality article. You shouldn’t treat something in the New England Journal as gospel just because of its source. Regardless of how you find them, you should subject articles to the same level of scrutiny.

Whether assigned for Journal Club, or selected by you from your subscriptions, you’ve put together a number of articles you’d like to read. How do you go about doing so?

Title and Authors

Obviously, the first thing you’ll see for an article is the title. It may sound stupid, but don’t blow past the title. A well written title may help you focus on the key facets of the article, as well as the subject matter, the outcome, patient population, or study type, helping you decide whether to commit your time and effort to the article. 

Especially early on in your academic career, it can be tempting to overlook the authors. After all, it’s a list of names of people you don’t know that slows you down in getting to the good parts. However, by skipping the authors you miss out on a few important benefits.

 Even if you don’t personally know the authors, you might recognize a name, and it will allow you to better contextualize the paper. A pediatric-related paper from the PECARN group would be attention grabbing, as would a pulmonary embolism paper from Jeffrey Kline, or an ECG paper from Amal Mattu. 

As a final note on authors, take a moment to look at conflicts of interests and sources of funding for the authors (often contained in a footnote). You should be more suspicious of conclusions when the authors have an apparent financial incentive to produce a certain outcome (e.g., drug-trial sponsored by the manufacturer.)

As with choosing papers from important journals, remember that a respected author is no guarantee of a perfect paper, and a sponsored paper does not automatically equal a bad paper. Do your due diligence!

Abstract

Read abstracts. Read lots of them. As I said before, you owe it to yourself to read the abstracts in any journals you subscribe to, and it doesn’t take that long. In reading the abstract you should be able to get a sense of:

1. Is this paper relevant to my patients?
2. Is it potentially practice changing / practice affirming?
3. Are the methods (study design) and results going to convince me of the conclusions of the paper?

Your answers to these questions will help guide you as to whether the rest of the paper is worth your time. If you decide the paper is worth your full attention, you can begin looking at the rest of the paper. I find that the best approach to reading scholarly papers is to start with identifying the “PICO” of the paper. (You may sometimes see this written as PICOT or PICOTS as well.)

 As you read through the paper keep this framework in mind, perhaps even noting down your answers to each of these questions. This will help you answer the questions of whether this paper is relevant to you and whether you should affirm / change your practice based on its results.

The Patient / Population may include things like age, gender, socioeconomic status, underlying health problems, etc. The “S” in PICOTS can refer to “setting” (i.e., emergency department, inpatient, outpatient, etc.). Obviously, a study on patients in an ambulatory care setting will be different to you than one on ED patients. Other factors that may be relevant include the type of hospital the study was performed at – resources at a tertiary academic center may be vastly different than at a small community hospital, and the conclusions drawn may not be applicable to where you practice. Also, as an example, consider that a study on in-hospital cardiac arrest may not be germane to the patients you care for in the ED who suffer out-of-hospital cardiac arrest.

The Intervention is what is different in one group relative to the Comparison (Control) group. It may be easy to understand this in a randomized controlled trial, yet you should be able to readily identify the Intervention and Comparison while evaluating any study that involves comparing two groups. If you cannot, that should be a red flag against the study. Ask yourself, are these Intervention and Comparison useful? A trial is most useful when the Comparison group is receiving a standard of care treatment, and drug manufacturers will sometimes offer comparisons that make their new therapy appear better by comparing it to something which is obviously worse. (A new pain medication compared to 2 milligrams of morphine, a novel antiplatelet agent compared to placebo rather than to aspirin, etc.)

Be aware that, as you evaluate the Intervention and Comparison, they may not be explicitly what the authors state. There may be unaccounted for differences between what is done to the two groups. (A treatment that involves frequent nursing assessments may show benefit purely due to the increased nursing interactions, rather than to the actual effect of the treatment, as an example.)

Finally, what is the Outcome of the study? An ideal study should have ONE primary Outcome that is patient oriented. Why is one primary outcome important? When you design a study, create the method, choose the number of participants, etc., you are trying to balance your chances of finding a statistical difference when one really exists. You do not want to over- or under-call the results based on random chance (we’ll talk more about Type I and Type II errors later). The more outcomes you look for, the more likely it is that random chance is playing a factor in your conclusions. Secondary outcomes in a paper should be treated differently than primary outcomes and should be hypothesis generating (suggesting a direction for future research). Similarly, when a subgroup is pulled out of the larger group and touted as showing a statistically significant difference, this too may be due to random chance and should be hypothesis generating.

Patient-oriented outcomes matter more than other outcomes (lab values, vital signs, etc.). Mortality is often thought of as the ultimate patient-oriented outcome. Does an intervention that changes your cholesterol level but not your mortality really matter? Also, over what timeframe (when PICO is written as PICOTS, the “T” refers to timing or timeframe) is the outcome being observed? Some studies show a difference in 90-day mortality but no difference after 1 year. A few extra months of (quality) life are good, but only reporting (or overemphasizing) the 90-day mortality makes the intervention look way better than it really is. 

Finally, as you read the paper, identify what the study design is. This is another topic we’ll spend more time on later.

Introduction

You’ll naturally be drawn to this section first. You don’t need to be. You can likely save it for later, or potentially skip most of it. The last paragraph or so of the introduction will likely help to identify the clinical question the authors are seeking to answer if you couldn’t figure it out from the abstract. And a well-written abstract should have made it clear!

The introduction will typically provide background clinical context for the study. As you immerse yourself in the medical literature, you’ll likely already know this stuff. It may be useful to skim. You may want to come back to it after you’ve read the rest of the paper – especially if some “established fact” in the paper piqued your curiosity and you want to see where that information came from.

Figures, Tables and Results

I recommend reading these sections first, as they are typically the most information dense. They will help you answer your PICO questions and decide if the rest of the paper is worth your time. As a reminder, when reading the results, pay particular attention to the Outcome – is there a single, patient-oriented Primary Outcome? Is it statistically significant? Is it clinically significant? As a general rule, when conducting a study, and writing a paper, researchers are looking for a result that is “statistically-significant,” which often means a p-value of less than 0.05, or a confidence interval that doesn’t cross zero. It is up to you, the astute physician, to decide if the results are clinically significant.

Be aware of harms associated with the intervention. The authors will likely be focused on the benefit of the intervention, but a good study will report the good and the bad together in the results. Make sure to scrutinize whether you genuinely believe that the benefits outweigh the harms.

Looking at the tables and figures can help you understand the results and know what to pay attention to when reading the paper. One specific table I think you should learn to look at early on is “Table 1.” In a trial, Table 1 is typically a comparison of the demographics between the Intervention and Comparison Groups. Remember that the goal of randomization is to make the two groups as similar as possible. Sometimes the groups are randomly not as similar as they should be, other times the study design may (intentionally or unintentionally) create dissimilar groups. If the groups are different, the results may be due to the underlying difference between the groups rather than the effect of the intervention. If the Control group is older, has more comorbidities, or more severe disease, the Intervention may look better by virtue of it being applied to a healthier population.

Methods

If you found this introductory article useful, I’d like to use future articles to dive more into the methods section of reading scientific literature. There’s a lot of meat there. For starters, ask yourself, does the method the authors chose to answer the question seem like the right one? If it isn’t the right one, why did they choose this method? Does the method they chose introduce potential confounding of the results? Would one hypothetically be able to reproduce the methods the authors used?

One easy thing to check is if the authors used Best Practices for their chosen method. Experts have put together consensus guidelines on how to perform different types of studies – PRISMA for Systematic Reviews, CARE for Case Reports, etc. It’s worth your time to check and see if the appropriate guidelines were followed.

Conclusion / Discussion

While the results are a reporting of facts – “what we found with this study” – the Discussion section is a chance for the authors to interpret those facts. At this point, you’ve hopefully drawn your own conclusions from the paper. Compare your conclusions with those of the authors. Do you agree with what they think their results mean? Why or why not? Pay particular attention to the limitations identified by the authors. If you had concerns while reading the paper, did the authors identify those issues as well? While it doesn’t necessarily improve the applicability of the paper, I am always reassured when the authors have the self-awareness to note where their study comes up short.

References +

You don’t need to read the entire reference section. But, as mentioned with the introduction, if there are particular background pieces of information, or “established facts” that strike you, it can be useful to go back and see where those were derived from. Ask me about Giardia or Backboards some time and we can go down the rabbit hole of medical truth!

If you really want to understand a topic and how a particular paper fits into it, you may want to look for who has cited it. (This works better for older papers.) Generally speaking, the more a particular paper is cited, the more important it is. If you look up the paper you’re reading in PubMed or Google Scholar, you can see how many times the paper has been cited (and even read those papers, if you like). 

Putting it all together

Obviously, this has been a lengthy blog post. As a quick reminder, here are the steps I think you should take in reading a journal article:

1. Read lots of titles an abstracts (especially in relevant journals)
2. Remember to glance at the authors and see if any names ring bells.
3. Approach the article with the PICO framework.
4. Read Results, Tables and Figures first.
5. You can probably skip most of the introduction.
6. When assessing the methods, does the approach taken by the authors seem like the correct approach to answer their clinical question? And did they follow the best practice guidelines for their study design, if applicable.
7. When looking at the conclusion, do the authors’ conclusions match yours? Did they identify their own limitations appropriately?
8. If you want to dive into the literature on this topic, look at the papers cited by the paper you’re reading, and the papers that cite it.

Happy reading!

Written by: Dylan Kellogg, MD, MSMEd

Edited by: Hakkam Zaghmout, MD

References

Boon, S. (2017). 21^st Century Science Overload. Canadian Science Publishing. Retrieved from http://blog.cdnsciencepub.com/21st-century-science-overload/.

Deng. (2020). PICO, PICOTS, PICOTT Framework for Clinical Questions as a Way to Design Clinical Trials. On Biostatistics and Clinical Trials. Retrieved from http://onbiostatistics.blogspot.com/2020/03/pico-picots-picott-framework-for.html.

Tanquary, G. & Cortez, E. (2019). Reading a Journal Article: A Simplified Approach. EM Resident. Retrieved from https://www.emra.org/emresident/article/reading-journals/.

Rahul. (2017). How to Read a Paper. Rush Emergency Medicine Education. https://rushemergencymedicine.org/2017/07/24/how-to-read-a-paper/