How supplements are rated on the evidence strength scale (and how to read these ratings)
How supplements are rated on the evidence strength scale — what the research says, straightforward and factual. u Bucha.
On one pharmacy shelf, there are supplements backed by hundreds of randomized controlled trials and those supported only by two test tubes and an advertisement on a wellness portal. The label reveals nothing. That’s why more and more consumers are looking for tools for self-assessment - and rightly so. The evidence hierarchy is a system that scientists and regulators have developed to distinguish 'works on cells in the lab' from 'works in humans in a randomized trial'. Understanding this system protects against overpaying for substances with weak evidence and helps choose those backed by solid science. This article explains the basics - step by step, without jargon where possible.
KEY INFORMATION
• The hierarchy of evidence classifies studies according to credibility: meta-analyses of RCTs at the top, expert opinions and in vitro studies at the bottom.
• RCT (randomized controlled trial) is the gold standard - only it allows distinguishing the effect of the supplement from the placebo effect and confounding factors.
• Examine.com uses an A-F scale for specific effects of a given supplement - an A rating means numerous consistent RCTs, a D rating means only data from cells or animals.
• “Promising results” in the description of a supplement means: an interesting mechanism, but no RCT in humans - do not buy based on this.
• NIH ODS and Cochrane Library are the two most reliable free sources for evidence assessment for supplements.
What is the hierarchy of evidence and why does it exist?
The hierarchy of scientific evidence exists because not all studies are equal. A study observing 50 people taking vitamin C for a year may show a correlation between supplementation and fewer colds. However, those who choose to supplement with vitamin C may also eat better, sleep more, and avoid stress - and it may be these factors, not the vitamin, that explain the effect. This is known as selection bias. The hierarchy of evidence classifies research designs based on how well they control for such confounding factors.Sackett et al., Evidence-Based Medicine, 2000).
At the top of the hierarchy are meta-analyses of randomized controlled trials (RCTs). They gather data from multiple independent studies and analyze them together - which provides greater statistical power and allows for the detection of effects that might be too small in a single study. Just below are individual large RCTs. Further down are cohort studies (tracking a group of people over time without randomization), cross-sectional studies, and case reports. At the very bottom: in vitro studies (on cells in the lab), animal studies, and expert opinions without empirical evidence.
What is an RCT and why is it the gold standard?
A randomized controlled trial (RCT) is a study design in which participants are randomly assigned to a group receiving the substance being tested (e.g., a supplement) or to a control group (placebo or another comparator). Randomization eliminates systematic differences between groups before treatment - if groups are randomly selected from the same population, they differ only in what they receive, not in who they are. Therefore, the difference in outcomes between groups can be attributed to the substance, not to other factors.
Double-blinding - when neither the participant nor the researcher knows who receives what - eliminates another source of error: the nocebo/placebo effect and unconscious favoritism of one group during measurements. A placebo-controlled study with double-blinding, conducted on a large group, reporting hard endpoints (e.g., fractures, hospitalizations, deaths, laboratory results) is the gold standard of methodological reliability. Many supplements have never undergone such a study.
Why don’t all supplements have RCTs? First - RCTs are expensive (millions of euros or dollars). Pharmaceutical companies fund them because patent protection allows them to recoup their investment. Supplements are usually natural substances that cannot be patented - so there is no economic incentive to fund costly studies. Second - regulators (EMA, FDA) require RCTs for drugs, not for supplements. A supplement can hit the market without any clinical trials.
We have noticed that many supplement descriptions confuse “biological mechanism” with “proven effect.” For example: “magnesium activates over 300 enzymes” is a biochemical fact - but that does not mean that magnesium supplementation in a person without a deficiency improves the function of those enzymes. The mechanism must be paired with clinical evidence (RCT) for the claim “magnesium affects sleep” to have a scientific basis. This second step is missing in most product descriptions.
Table: hierarchy of scientific evidence for supplements
Below is a simplified hierarchy with examples for popular supplements.
| Poziom | Type of study | What it says | Example of a supplement |
|---|---|---|---|
| 1 (highest) | Meta-analysis of RCTs (systematic review) | Strongest evidence; controls random and systematic error | Creatine and muscle strength (dozens of meta-analyses) |
| 2 | Single large RCT (≥200 participants, blinded) | Strong evidence; may have population limitations | D-mannose and UTI recurrences (Kranjčec 2014, n=308) |
| 3 | Small RCTs (<100 participants, short duration) | Signal evidence; requires replication | Ashwagandha and cortisol (most studies) |
| 4 | Cohort studies, cross-sectional | Correlation; does not prove causation | Omega-3 and heart disease (observational studies) |
| 5 | Animal studies | Mechanism potentially significant; lack of human data | Many “new” nootropics |
| 6 (lowest) | In vitro studies, expert opinions | Hypothesis; no clinical evidence | Most “breakthrough” substances from advertisements |
How to read Examine.com ratings - scale A-F
Examine.com is one of the most popular tools for evaluating evidence for supplements. The platform assigns ratings not to supplements as a whole, but to specific effects - a separate rating for “effect on testosterone levels,” a separate one for “effect on muscle strength,” a separate one for “effect on sleep quality.” This is an important nuance: a supplement may have an A rating for one application and a D rating for another.
What do the individual ratings mean? Rating A: many large, consistent RCTs confirm the effect in several independent research groups. Example: creatine and strength/muscle mass gain. Rating B: several well-designed RCTs with generally consistent results, but with some caveats (e.g., only one population, short observation time). Rating C: limited number of RCTs, conflicting results, or methodological weaknesses. Rating D: no RCTs in humans; only data from animal studies or in vitro. Rating F: no evidence or existing evidence is negative.
Interpretation trap: Examine.com evaluates the strength of evidence, not the size of the effect. A supplement may have an A rating for a very small, clinically insignificant effect. For example, melatonin has an A rating for “reducing sleep onset time” - the effect is about 7-8 minutes reduction in sleep latency, which is statistically significant, but for someone with severe insomnia, it may be clinically insufficient.Ferracioli-Oda et al., PLOS ONE, 2013). The strength of evidence and the magnitude of effect are two different dimensions.
GRADE - a system used by Cochrane and clinical guidelines
The Cochrane Library and most scientific societies use the GRADE system (Grading of Recommendations Assessment, Development and Evaluation) to classify evidence. GRADE assesses four parameters for each review: risk of bias, inconsistency, indirectness, and imprecision. Based on these, it assigns ratings: high, moderate, low, or very low certainty of evidence (Cochrane, 2024).
Practical example: a Cochrane review on vitamin D and falls in older adults may rate the certainty of evidence as “moderate” - this means that the true effect is likely close to the estimated one, but there is a chance that further studies may change the conclusion. “Low certainty” means: the effect may be radically different from the estimated one. When evaluating a supplement, “low certainty of evidence in GRADE” should be a signal for caution, even if the direction of the effect is positive.
We have noticed that in the Polish information space about supplements, two types of sources dominate: manufacturer websites (with an obvious interest in presenting data positively) and “health” articles that replicate claims without verifying sources. Examine.com and NIH ODS are in English, which poses a barrier for many readers. Therefore, in articles on this blog, we always cite specific studies with a link to PubMed or PMC - so you can check the original text, not its interpretation by a marketer.
Common pitfalls in interpreting supplement research
A few heuristics that protect against overinterpreting supplement data. First: conflict of interest. Studies funded by the supplement manufacturer statistically have a higher chance of a positive outcome than independent studies. This is a confirmed regularity in the scientific literature - the phenomenon is called “industry bias” or “sponsorship bias.” This does not mean that all industry studies are fabricated, but it signals the need for verification through independent replications.
The second pitfall: surrogate outcomes (surrogate endpoints). A study may show that supplement X raises the level of substance Y in the blood - but does raising Y translate into real health benefits (fewer diseases, longer life)? Not always. A classic example: beta-carotene in cohort studies correlated with a lower risk of lung cancer, so RCTs with supplementation were conducted. The result: a higher risk of lung cancer in the supplementing group.ATBC Study Group, NEJM, 1994). Correlation was not causation.
The third pitfall: reference base. A supplement may be “effective” compared to placebo, but worse than another available drug or non-pharmacological intervention. When assessing the value of a supplement, it is worth asking: “effective compared to what?” - not just “effective or not.”
Frequently Asked Questions
What is the hierarchy of scientific evidence?
The hierarchy of evidence is a classification of types of studies according to their methodological reliability. At the top: meta-analyses of RCTs and systematic reviews (the strongest evidence). At the bottom: in vitro studies and expert opinions. The higher in the hierarchy, the lower the risk that the result is a methodological artifact. The system was created to assist in making clinical decisions based on the best available data (Sackett et al., 2000).
What do the ratings A, B, C mean in Examine.com?
Examine.com rates the strength of evidence for a specific effect of a supplement on a scale of A-F. Rating A: numerous consistent RCTs (e.g., creatine and muscle strength). Rating B: a few good RCTs with some caveats. Rating C: limited or conflicting data. Rating D: only animal studies or in vitro. Rating F: no evidence. Important: an A rating for one application does not mean A for another in the same supplement.
What is the difference between an RCT and an observational study?
RCTs randomly assign participants to groups, eliminating the influence of confounding factors. An observational study merely records what people do and what outcomes they have - but supplementing individuals may generally have a healthier lifestyle, which distorts the results. RCTs are the gold standard for assessing the effectiveness of supplements because only they can prove causality, not just correlation.
What does it mean that a supplement has “promising results”?
“Promising results” means: there is a biologically plausible mechanism and preliminary data (usually from animal studies or small pilot trials in humans), but not enough RCTs to confirm effectiveness. It is an encouragement for further research, not a confirmation of efficacy. This is not a sufficient basis for purchase and use.
How to check the strength of evidence for a specific supplement?
Three reliable and free sources: Examine.com (scale A-F for specific effects), NIH ODS (sheets for hundreds of substances with evidence ratings), Cochrane Library (meta-analyses, GRADE system). Avoid information from manufacturer websites and blogs without links to original studies.
This article is for informational and educational purposes and does not replace consultation with a doctor. If you are pregnant, breastfeeding, taking medications, or have chronic conditions, consult the use of supplements or herbs with a specialist.
Author: Michał Waluk · Published: 2026-05-04 · Updated: 2026-05-04
