Cochrane reviews are considered by many to be the “gold standard” or the final word in medical conversation on a topic. We explored the eleven most relevant Cochrane reviews on herbal medicine and identified that frequently herbal medicines in the included studies had not been sufficiently well characterised. If data on the effects of the plant parts are unavailable, effects of co-active ingredients need to be considered and the plausibility of the study medications for the specific indications discussed. Effect sizes calculated from exploratory studies would be best used to determine the sample sizes required for future confirmatory studies, rather than as definitive reports of intervention effects. Reviews should be comprehensive, including discussion of putative adverse events and possible drug interactions. We suggest that the guidelines for preparing Cochrane reviews be revised and offer assistance in this task.