"Lies, Damn Lies & Statistics" – It Was Ever Thus…

What kind of lab is this? Credit: Syd MeadAs a consumer, or as a producer, how badly do you really want to know what is in your wine? Would you stop buying (or making) a wine you really liked if you found that it didn’t “stack up” against others in areas that had nothing to do with aroma, flavor, value, or age-worthiness? As a producer, do you change the way you make a successful wine in reaction to some arbitrary praise for a different style? Who would you trust to make those determinations? Inquiring minds want to know.

In early April Alice Feiring tweeted a link to a press release by a lab proclaiming that their in-house testing of over 450 top-selling white wines in the US revealed that one in three of them have levels of lead or arsenic above the level allowed in drinking water under the Clean Water Act.

Oooh I’m so scared…

In fact I am not scared at all, though that seems to be the intent of the press release. Here’s why.

The action level the EPA lists for lead in drinking water is 15ppb (a part per billion is a microgram per liter in liquid or per kilogram in a solid) and for arsenic it is 10ppb. The low levels allowed for both of these metals in drinking water are based not only on their toxicity but mostly on the volumes likely to be consumed by children, not adults.

By contrast, in a letter to the TTB the director of the Center for Food Safety and Applied Nutrition in the FDA stated:

“The Health Hazard Evaluation Board (HHEB) determined that the daily intake of table wine containing 150ppb or less of lead is likely to pose no more than a minimal risk to the adult population, including women of childbearing age.”

Note that the concentration beneath concern by the HHEB is over ten times the action level for lead in drinking water. The director then goes on in the letter to propose that the action level for lead in table wine should be 300ppb.

There is no mention in the Code of Federal Regulations of a limit for arsenic in table wine. The FDA is currently engaged in rule making which proposes an action level of 10ppb in apple juice — the same level as in the standard for drinking water.

Given that this 10ppb arsenic proposal is based on the fact that children drink a lot of apple juice (as well as water), it would be unsurprising to me if the HHEB was to suggest that a level of arsenic in wine of ten times the drinking water standard would be “likely to pose no more than a minimal risk to the adult population.”

Levels Of Lead & Arsenic In Other Foods

It’s long been known that plants accumulate these metals (and others) from the environment. For example, lead levels in lettuce grown next to highways (where the soil is contaminated with decades of accumulation from burning leaded gasoline) have been measured at up to 1.5ppm (1,500ppb) — over 100 times the drinking water standard.

The FDA has measured levels of arsenic in rice (which tends, more so than other grains, to accumulate the naturally occurring metal from the water it is grown in) — especially brown rice — of up to 700ppb, or over 70 times the drinking water standard.

But that’s not really the point. Concerns over toxicity or toxic effects pivot on the serving size and frequency of ingestion. Even the most determined hypochondriac must recognize that drinking enough wine to reach even mildly toxic levels of lead and/or arsenic is likely to result in negative health consequences due to excessive alcohol consumption that far exceed the damage caused by the exposure to these metals.

Which leads back to the question: what is the point of the press release by this lab, warning of levels of these metals in wine that exceed drinking water standards? Assume that the purpose of the press release was to generate media attention for the new business with a scary headline — the tactic appears to have failed. Today I can find no online references to the “1 in 3 wines tested”” headline, and it appears the lab has even removed the press release from their website (though the original tweet is still up).

Perhaps they don’t have much faith in their methodology. I certainly have questions about their performance. This lab claims that their current accuracy/precision is ±20% — a level of uncertainty that was not good enough for me or for my clients when I managed a wine services laboratory in the late 1990s.

In a Twitter exchange with this lab I asked if they had ISO certification and the answer was “we have started the process.” However, I know from experience that the process can take well over a year, and that ±20% on analytical results won’t be acceptable analytical accuracy/precision to the accrediting agency.

In a further Twitter exchange with the lab, they declined to state where they are getting their startup capital, and would not respond to a request regarding their business model — i.e. how they plan to make money after they burn through their startup funding.

What Is The Agenda Here?

Ignoring the implied — and mistaken — assumption in the press release that table wines should be held to the same standards for these metals as drinking water, there is the question of how the levels of lead and arsenic in wine compare to levels found in other foods. The lab does not say what absolute levels they found in the 450+ wines they analyzed, so it is impossible for an educated and concerned consumer to assess the risk. Without the actual statistics, according to the axiom: perhaps we are left only with lies and damn lies as the other possibilities.

This lab is attempting to establish itself as an arbiter of which wines are “healthier” than others on a range of metrics. Rather than provide actual levels of “unhealthy” components in wines in the context of the range of levels of these same components found in comparable serving sizes of other foods, they claim they are using a proprietary algorithm to weight the levels they measure — however accurately, or inaccurately — by non-transparent and therefore arbitrary criteria in order to generate a one-to-five-star “ranking” for individual wines. While Federal law prohibits wine producers from making health claims about their products, third parties such as this lab are, surprisingly, exempt from this injunction.

Much has been written about mandatory composition/nutrition labels on wines, and I’ve written before about the realities and pitfalls surrounding these proposals. While I do have reservations, I generally support the right of interested consumers to know what they are paying to put in their bodies:

Mrs Winslow's Baby Killer

How would you know this contains morphine, alcohol, soda ash and ammonia?

If the people running this lab were more transparent about their funding, their business model, their weighting algorithm and their overall agenda, it’s likely I would be inclined to support their mission. As it is, I have a lot of unanswered questions, mostly regarding the potential for this lab’s activities to result in both direct and indirect coercion of producers: either a pay-to-play scenario, or a requirement to defend our products against specious or spurious claims of “unhealthiness.”

Since the lab was unwilling to provide information, I forwarded these concerns to the Wine Institute, and apparently the legal and technical teams there share them. Here’s their reply to my inquiry:

Wine Institute’s Technical Advisory Committee has been monitoring [this lab] for several months now to determine their direction, intent and purpose. We agree with your assessment that [this lab] has the potential to leverage their web presence and immunity from TTB therapeutic claims provisions to coerce industry members. At this point, we are actively monitoring their website to see if any of their statements rise to the level of a legally actionable claim for defamation, unfair business practice, or false representation.

In light of this uncertainty, Walter White has good advice for everyone involved in this issue for the time being: Heisenberg As this develops I’m soliciting my producer friends and acquaintances on what they think about a third party ranking their wines on arbitrary scales of “healthiness.” I’m wondering what consumers think of this as well: believable? Helpful? Inquiring minds want to know.

7 thoughts on “"Lies, Damn Lies & Statistics" – It Was Ever Thus…

  1. Ryan

    At least the study has a statistic (albeit one without a lot of meaning). For years, the French media will latch onto a story about alcohol abuse (that distinctly talks about hard liquor and NOT wine) and will plaster images of wine bottles and wine glasses all over their coverage of the report.

    This particular report is a step forward! 🙂

  2. John Kelly Post author

    Ryan – by “report” do you mean this post, or the lab’s press release?

    I consider the press release to be an attempt to generate here the same sort of bad PR the French press seem to take glee in casting on wine. Not sure I see that as a step forward.

    I’m reminded of the study released last year by Pascal Chatonnet’s Excell Lab on pesticide residues in French wine and published in Que Choisir. The actual thrust of the study was to demonstrate Excell’s expertise in helping vignerons use less, and less toxic, pesticides.

    The results demonstrated that no wines tested had any pesticide residues at levels of concern for human health. The lab pitched this as a win, and sought to encourage vignerons to continue using their service to further reduce exposure of the vineyard workers to pesticides.

    But that wasn’t the headline the press picked up on. No, what every single “journalist” plumped was:

    “Pesticides Found In 90% Of French Wine!”

    Fucking brilliant.

  3. Thomas Pellechia


    You should have been an investigative reporter. You picked up on this lab’s press release and then did exactly what consumers can’t and won’t do–you sought to find out who was behind it, what the report meant, how the organization came to its conclusions, and why the press release was issued.

    Whoever regurgitates such “news” without doing due diligence is as guilty of malfeasance as the company that released it.

    I don’t know why, however, you chose not to identify the lab or at least its web address.

    1. John Kelly Post author

      Thomas I didn’t identify them for a number of reasons. Number one is that I don’t want to give them any more air than they are already taking for themselves. They portray themselves as a lab but really they are a media company. The have 40K followers on Twitter but nearly zero retweets of their content, which suggests they invested money to buy followers — a media startup move — rather than to improve the quality of their analytical work.

      Second, the purpose of this post was mostly to start to get the word out to my colleagues that these guys exist. In principle I’m not opposed to a third-party lab buying wines off the shelf and analyzing them to provide consumers with information — one thing this lab claims they can do is help consumers find new wines that match ones they already like by generating analytical groupings (yeah, good luck with that). I think it’s great they want to provide info on sugar levels and calories.

      I’m a lot more concerned about their measurements of allergens and pesticides, given their epic fail on contextualizing their findings on lead and arsenic, and given the media reaction to the Excell Labs report I mentioned above. They have some really moronic statements on their site, for example regarding sulfites, along the lines of: “some people are allergic to sulfites; some allergic reactions produce headaches; therefore sulfites may cause headaches” — pandering to a popular misconception that is not supported by any credible clinical research.

      Because I view this “lab” as a social media company in startup mode, doing something I support in principle, I’d like to give them the opportunity to get their act together and do it right. At the same time I want everyone to be aware that there is the potential for this to turn into just another Yelp-style shakedown racket. The difference is that Yelp employs user-generated content to support their extortion.

      And if anyone really wants to know what the business is they can Google the particulars or scroll through my tweets, or Alice’s.

      1. Murray

        Also – the whole premise they are basing their “research”on may be false. Heavy metals are dangerous. However there are organic and inorganic forms. Arsenic being a case in point – the lead arsenate we used for years (up to say 1970) was dangerous – long lasting in the soil and toxic to just about everything. An inorganic form.

        On the other hand many aquifers are “polluted” with arsenic at “significant levels” and have over centuries proven to be of no harm. This organic form being processed to excretion by the body. Lettuces being an example – no white butterflies bother them, but humans eat them without effect.

        1. John Kelly Post author

          Trivalent arsenate is quite toxic, but it is quickly metabolized in the body to the much less toxic pentavalent methylate and excreted. Prior to the advent of penicillin, Fowlers Solution (trivalent potassium arsenite) was used to treat malaria, syphilis, asthma, chorea, eczema, and psoriasis. In 1910 Paul Erlich introduced Salvarsan (arsphenamine) which for decades was THE treatment for syphilis. Erlich also studied the effectiveness of pentavalent organo-arsenates (such as Atoxyl, arsanilic acid) as chemotheraputic agents. To this day, arsenic trioxide is used to treat acute leukemia.

          I didn’t make the distinction between inorganic and organic arsenic in the original post, because the EPA drinking water standard does not differentiate between the forms.

  4. John Kelly Post author

    Lewis Perdue shared a Google Scholar search on more recent research into the effects of chronic low-level exposure to arsenic.


    Many of these papers suggest there are credible links between chronic exposure and increased morbidity or mortality. I note that the cluster of negative outcomes closely parallels that which have recently been associated with chronic inflammatory disease.

    I remain unfazed.


Leave a Reply

Your email address will not be published. Required fields are marked *