Type of Evidence

Comments

“Level of Evidence”

(US Agency for Health Care Policy

and Research 1992)

Note: Levels I and II indicate a significant clinical effect – if a good trial shows only a minor clinical effect its Ranking will be reduced

EXTRACT

ranking

Ia

•     formal meta-analysis of substantial controlled clinical trials

Formal meta-analyses are being published for a growing number of herbal medicines – although not unchallenged they remain the highwater mark of scientific consensus; some “meta-analyses” have less formal criteria and may be ranked with good systematic reviews as Ib’s.

Ia

Ib

•     double-blind random-assignment placebo- or other controlled clinical trials of sufficient size to support statistical significance

•     reviews of substantial controlled clinical trials

The gold-standard of evidence for a specific effect and often well-suited to studying herbal remedies; apart from problems of logistics and lack of patent protection the main difficulties are with establishing reliable outcome measures for many of the low-morbidity indications for these treatments, and the high recruitment therefore necessary; good systematic reviews will also merit this ranking.

Ib

IIa

•     good controlled clinical trials

This category is used for well-controlled studies whose conclusions however are diminished by minor doubts on, for example, the robustness of outcome measures, power calculations and conclusions, as well as on presentation, declarations of interest and other methodological issues. This is the maximum ranking for studies on healthy subjects and ex vivo studies.

IIa

IIb

•     controlled clinical trials not fully reported or with more significant flaws

•     open or partially-controlled clinical trials with human subjects

•     other clinical trials with humans

Results confounded by relatively strong non-specific effects; the best examples (IIb) will include large open studies and rigorous case study designs; poorly reported studies even of controlled trials may be down-ranked; a very poorly reported clinical trial may be ranked  as III

IIb

IIc

or

III

IIb

•     in vivo studies on oral consumption of whole plant extracts or crude plant extracts in close to therapeutic doses

It is relatively rare to have observations of animal consumption of plants in realistic doses and even here therapeutic usefulness is limited by interspecies variations

III

IIb

•     in vitro studies of whole plant extracts or major plant constituents where action is topical or affects gut surface receptors

Some value in illuminating traditional or clinical data but only where rationale for activity based on non-systemic mechanisms; most useful for plant constituents which act primarily on the body surface or internal surface receptors

III

III

•     systematised observations of traditional use;

•     systematised practitioner accounts

•     rigorous single-case reports

Valuable resource for further studies and can validate other conclusions; still however insufficient basis for efficacy alone.

III

•     other in vivo studies

Very little bearing on effect of plant in clinical context

IV

•     other in vitro studies

Almost no bearing on effect of plant in clinical context

IV

•     popular claims and reputations;

      non-systematised observations or ethnobotanical records

Potentially registered in the "EXTRACT traditional use score" (attached); impossible to rule out non-specific (incl placebo) effects for individual claims

IV

•     non-formal review papers

Where a paper reports on other papers without constructing a rigorous review process to weight them for and against the argument they are entered in the EXTRACT database for interest only.

R