In August 2024 the Federal Trade Commission finalized a rule banning the sale and purchase of fake reviews, with civil penalties of over $50,000 per violation. The agency wouldn’t have bothered if fake reviews were rare. They’re not, and supplements are one of the categories where they’re most common, because the products are cheap to make, expensive to sell, and almost impossible for a buyer to evaluate on their own.
So you read reviews. Everyone does. The question is what they can actually tell you about a bag of powder or a bottle of capsules, and the answer is: less than you’d think, but more than nothing if you know what to look for.
What a review can’t tell you about a botanical
Start with the limits, because they’re specific to this category.
A reviewer can tell you how a product tasted, whether it arrived on time, whether the capsules were consistent, and how the company handled a return. They can’t tell you what was in it. Botanicals vary batch to batch in ways a customer has no way to detect. A study in Scientific Reports that analyzed 53 commercial kratom products found the main alkaloid varied about fourfold between samples, and some products had entirely different chemical profiles from others despite all being the same plant. A five-star review from January says nothing about the batch shipping in August.
Reviews also can’t tell you about contaminants. When the FDA tested 30 kratom products for heavy metals, it found lead and nickel across the board, some at levels it considered unsafe for daily intake. None of those products had reviews saying “contains lead.” Customers can’t taste it.
And a reviewer’s experience of an effect is one person’s report on one day. It’s a data point about them, not a specification of the product.
What a review can tell you
Plenty, if you read the right ones for the right things.
Shipping and packaging complaints are reliable. If twenty people say the bag arrived torn or the seal was broken, that’s a real pattern.
Consistency complaints are worth weight. “This batch was different from my last one” is one of the most useful things a customer can write, because it’s exactly the problem lab testing exists to catch, and it tells you whether the vendor’s quality control is working.
Customer service reviews are reliable and underrated. How a company handles a wrong shipment or a return request tells you more about whether they’ll stand behind the product than any product description.
Taste and texture reports are honest, if only because there’s no reason to fake them.
Eight signs a review section is being gamed
The FTC rule targets the sellers, but it doesn’t scrub the internet. Here’s what to look for:
- A cluster of five-star reviews posted within the same few days, with similar phrasing
- Reviews that read like ad copy, naming the product’s full title and repeating marketing terms
- Reviewers with a single review in their history, all in the same product category
- A rating distribution that’s almost entirely fives and ones with nothing in between (real products get threes and fours)
- No negative reviews at all on a product with hundreds of ratings
- Reviews that describe effects in medical language the seller itself can’t legally use
- A “verified purchase” label on a site that has no way to verify purchases
- Review counts that jump suddenly after a period of low activity
None of these is proof by itself. Three or four together on one product page is a pattern.
The trick that works better than reading reviews
For botanicals specifically, there’s a faster and more reliable check: look for a certificate of analysis.
A CoA is a lab report for a specific batch. It lists what was tested (identity, active compound content, heavy metals, microbial contaminants), what the results were, and which lot number it applies to. It’s issued by a third-party lab, not the seller. Unlike a review, it can’t be faked by a bot farm, and unlike a star rating, it actually describes what’s in the bag.
Kratom is the clearest example of why this matters, because it’s a raw imported leaf product with an enormous range of sellers and no standardized labeling. The variability study above is the reason. If you’re shopping for it, a kratom seller that posts a certificate of analysis for every batch is giving you something no review can: a number for mitragynine content, a pass or fail on heavy metals, and a lot number you can match to your bag. Weigh that against a product with 2,000 reviews and no lab data, and the 2,000 reviews lose.
The same logic applies to ashwagandha (withanolide percentage), and mushroom extracts (beta-glucan content). If a vendor can’t produce one, that’s the answer to your question.
A practical order of operations
Here’s how I’d run it for any botanical product in under ten minutes.
- Check for a CoA first. If there isn’t one, stop; nothing below matters.
- Read the one- and two-star reviews before the five-star ones. Complaints are more specific and harder to fake.
- Sort by most recent and look for consistency comments in the past few months.
- Skim the five-star reviews for the gaming signs above. Skip any that describe effects in clinical terms.
- Look for how the company responds to negative reviews. Silence, defensiveness, and a public offer to make it right are three very different signals.
- Check the return policy. A vendor confident in the product will take back an opened bag.
Reviews are a tool for judging the seller. Lab reports are the tool for judging the product. Most people use the first to do the second’s job, and that’s where supplement buying goes wrong.
Before your next order, pull up the product page and look for the lab report before you look at the star rating. If the report’s there, the reviews become useful context. If it isn’t, you already know enough.
