Are Nevada’s Cannabis Testing Labs Counting Cards?

Venturing out of Washington State generally increases my wonder over the diversity and beauty that, in many ways, defines America.  Returning home to Washington State always reminds me of how fortunate I am to live here.

I recently had the opportunity to travel to Nevada, where I was able to present to a number of government personnel the results of some work I had done on their regulated cannabis lab testing data.  The public servants I spoke with all seemed very engaged and passionate about the important role they play in public safety and health.

What impressed me most is how rapidly Nevada escalated the issues suggested by the data and also by how quickly the state acted.  I presented my work on a Thursday afternoon and met with members of a number of different government departments on Friday.  The following Monday, the Department of Taxation published a notice to the labs that put them on notice that the Department was “aware of and investigating potential inflation of THC levels by cannabis laboratories” and that such behavior was not acceptable.  In early October, Governor Sisolak — in part because of “allegations of manipulated lab results” — announced the formation of a multi-agency special task force to root out potential corruption in Nevada’s marijuana marketplace.

What impressed me least about my interactions with the state of Nevada was how difficult it was to get access to the relevant Nevada lab data.   Washington has, for much of it’s 5 years regulating cannabis, shared major portions of the seed-to-sale traceability database freely with the public.  That transparency enabled a much better understanding of the emerging market and, specifically, the fact that bad actors had come to dominate the lab testing industry in Washington and that the vast majority of product available to regulated consumers appeared to suffer not only from inflated values of THC (potency) but also inadequate testing for quality (safety).

In Washington, farms using bad labs can be identified and the stores selling product tested by bad labs can be identified and brands using bad labs for testing can be identified and consumers can be informed.  Good system.  It certainly works for me (or did until Washington chose to use MJ-Freeway as their new traceability provider and the data became dirty and relatively inaccessible).

By contrast, in Nevada it took almost a year of repeated effort for me to get useable lab data from the state.   The data I received had been “blinded” in that, while unique labs and unique growers can be identified for purposes of analysis, there is no indication regarding which lab or which grower corresponds to the numeric code applied to them in the database.   That is unfortunate, because the consumer advocate within me would very much like to call out the growers and retailers that appear to be reinforcing and financially supporting some of the labs whose data are clearly different than those of the others.

I’m writing this post now because I saw an article published in Nevada earlier this week that mentions quite explicitly that, while the state is aware of odd data coming from certain labs, it has made no apparent effort to remove affected product from the cannabis supply chain or to inform consumers regarding which brands (and growers) should be treated with caution.  This article suggests that the state is putting it on the shoulders of ignorant consumers to be “careful when they shop”.

The work I shared with the state of Nevada in September clearly shows some labs are producing data that are qualitatively different than that being produced by others.  Nevada knows not only who these labs are, but where they are and who is responsible for their operations.  Nevada knows which products they have tested and on which store shelves those products currently lie.   Nevada knows how many consumers are purchasing these products each and every day.  Nevada knows how much tax revenue they are making off of the sale of products tainted in this way.

Here are a handful of charts that describe some of the observations I made looking at Nevada’s METRC traceability data spanning the period January 2018 through May 2019. They describe the overall trends in reporting fully decarboxylated THC (THCmax) and overall Quality Assurance failure rates seen during this timeframe.  

The first chart shows the steady increase, over the 6 quarters covered, in average THCmax reported for both Flower and for Concentrates (dabs and cartridges).  

Each of the two largest product categories has shown a consistent increase in THCmax over this time period.   By way of reminder, Nevada’s regulated cannabis industry did not start in January of 2018, that is when they started using METRC as their traceability solution.

The next chart breaks the average quarterly flower THCmax trends down into the values reported by each of the nine busiest labs in the state.

Three of the labs are showing increases in reported THCmax over time, four show relatively constant levels over time and two show not only lower levels but also declines in reported levels over time.

The following chart shows overall monthly flower THCmax levels and QA Failure levels, statewide.

Flower “potency” increases surged quickly early in 2018 and have steadily increased, statewide, since then.  The rate of QA failures shows an odd cyclicity that might imply some sort of seasonality, were it not the case that all of Nevada’s regulated grows are indoor operations.   It looks like a “process out of control”, but that’s just the six sigma black belt in me trying to find a raison d’etre.

The overall QA failure rate for flower during this time was 15.9% statewide.   Here are the failure rates reported by each of the 9 top volume lab licensees statewide.

There are clearly some differences in the results being reported by different labs —- in this case, a three-fold difference between the percentage of flower failing in the “least failing” labs and that in the “most failing” lab.

Now, for some analysis — here is how the top 9 volume labs shake out in overall “Potency” vs “Safety” testing over the past year and a half.

Decent correlation between the two.  Notice how labs that tend to report higher cannabinoid levels tend to also fail fewer samples on the basis of quality.

That means these “high thc labs” find fewer instances of heavy metals, pesticides, microbes, mycotoxins, molds, mildews, yeasts, worms, roaches and rat-turds in the product they test.   The product they test is not unduly wet before packaging.  More of the product they test is reported as “safe”.  It is fit to be thought of as medicine.

In the article I linked near the top of this post, Dana Gentry highlights the inaccurate reporting of potency levels and airs a number of criticisms that can be aimed at regulatory efforts to sanction the labs based on their reported data.  I thought her article was good, but I take issue with a number of points made in it.  

I’ll address those at another time, but I would like to emphasize here that inaccurate reporting of cannabinoids IS a public safety issue.   Most people quickly get how allowing poor quality product to pass QA testing and, hence, make it to market IS A PUBLIC SAFETY ISSUE.   There is little cogent argument against that point.

When consumers are not allowed to know how much “active ingredient” there is in the product they are purchasing, they are less able to effectively moderate their consumption to be appropriate to the task(s) at hand.   Tasks as diverse as as chilling on the couch, coding a new software program, playing ball with the kids or driving down the road on the way to Church.  Inaccurate product labelling dramatically increases the likelihood of mis-alignments between intent and impact on the part of the consumer.  Most people understand that driving after drinking a mug of barley wine is likely more risky than driving after drinking a Bud Light. 

The consumer should be well informed about that which they are consuming.  The regulated and taxed consumer DESERVES to be so informed. 

The final three charts each displays animations of the monthly results reported for Flower by each of the 9 top volume testing lab licensees in the state.   One shows changes in overall Flower QA Failure rates, one shows changes in THCmax reporting, and one shows changes in the combination of these two key measures over time.

I like playing these and putting them on auto-repeat.  Looking at the ghostly tracer bubbles that each series leaves behind.  Looking at how they move across time.  Wondering why some of them seem different.  Why some of them seem to be failing so few samples (please forgive the -1% on the axes of these graphs —- they really stop at 0% but the visualization was better when I gave it a bit of room past zero).

Most of all, if I were a consumer of regulated cannabis in Nevada (or a tourist thinking of going there and possibly trying some of their regulated cannabis), I would like to know which brands had been tested by some of these labs and which ones had not.  Heck, I’d like to know if there are any stores whose product selections differentially reflect product tested by labs that seem to be inflating THC.  From what I saw in Washington, when such data were available, the answer is likely to be yes.    Just as there are growers that rely on labs that inflate THC, there are stores that tend to push such product on ignorant consumers.

Let me know through the comments on this post what you think about the data being spewed out by some of these Nevada cannabis testing labs.   I look forward to the dialogue.

Tomorrow, I intend to put in another public records request to Nevada for their lab data spanning the period Jan 1, 2018 through Nov 30, 2019 with lab names and grower names included.

If you are from Nevada and would like to learn which brands are associated with labs whose pattern of data catches your eye, please support my efforts to have unblinded lab data made available by writing to your state legislators and letting them know you think they should not keep such important safety information from the public.

As a quid pro quo, I’ll promise to reveal some of the growers that seem to have been engaged in “lab shopping” if I’m able to get the unblinded data (they exist … but I only know them as numbers today and that is of little use to anyone other than those regulating Nevada’s cannabis market).

22 comments

  1. Another great article Jim. It has been too long in coming.

    You are correct that consumers deserve and should expect and demand to receive such clear unredacted information. I would encourage any Nevada consumers to support your efforts and daylight the facts for everyone to see.

    1. Thank-you, Don.

      It was nice to get this post out there and it was fascinating to see how many of the problems that were uncovered in Washington’s regulated cannabis testing industry during it’s first few years of operation are also clearly there in Nevada today.

      I expect to have quite a few more posts published between now and the end of the year.

    1. True that, Dana. I actually looked at how reported potency values varied with increasing product testing volume, and it was a pretty clear relationship. Lab-shopping is also apparent in Nevada — also much like it was in Washington last I looked.

      Here is a link to a chart that shows the relationship between testing volume (of the grower) and the average flower THCmax levels that are reported for them: Nevada Potency as Function of Testing Volume

      The world is a wonderful, varied place — but the humans that inhabit it seem to have similar tendencies no matter where one looks.

  2. So basically you’ve found the same thing that’s been happening in WA going on in Vegas. That’s not surprising really when their motivations are greed not clean, safe, quality cannabis. They have trained consumers that are new to cannabis that the higher the numbers the better it is therefore they can charge more for it.

    The budtenders always like to point out high numbers when selling but most can’t tell you from their exp if it’s any good.

    This has been my issue in WA since 502 passed, because to many in the industry will pay to get the results they want & if they are too low won’t even show those. The way stuff is labeled also leaves much to be desired. Everything I have tried from WA stores in the past yr has been crap, no flavor, no terpenes so dry it turns to dust but it has high THC #’s. Some will list the pesticides used but too many won’t. It’s very hard to find good, clean, safe, high quality cannabis that’s hand trimmed at decent price in WA & sounds like Nevada is taking lessons on what not to do from WA.

    1. Yes, Gregg — the pattern of data in Nevada is very similar to what I saw back in 2015 when I initially looked at Washington’s lab data in detail.

      I agree with most of what you say (aside from my belief that one never completely understands the motivations of other people).

      However, I am aware of some fine product that graces the shelves of at least the better retailers in Washington state’s regulated cannabis market. I am also aware of retailers that DO give a damn about their customers and try to curate the products they offer in a way that meets the needs and desires of their customers.

      I’m not a paid shill for any of these companies, but you can’t go wrong with flower from Trail Blazin’ Productions or Washington Bud Company. Similarly, Fairwinds Productions has wonderful topicals and tinctures. Avitas and Heylo are my preferred manufacturers of concentrates. My favorite stores are any of the Uncle Ike’s or Dockside stores. I also like Canna West in West Seattle.

      My advice is to get to know the growers that are out there. Find ones you trust. Give your business to them.

    1. I produced all of the graphs in this article (and comment stream) using data provided by the state of Nevada in response to a public information request.

  3. I really like this statistical approach, but you shouldn’t forget that there are actually good growers out there delivering high THC with less micro fail. In that case you cause bad reputation for a lab and a grower that just has good plant material and GMP

    1. Terry — I have not forgotten that fact for a moment.

      Nor have I ignored the fact that many of the growers that are delivering high THC seem to have enjoyed their increases in THC not “organically” (e.g., slowly and gradually over time as their genetics and processes improve), but discontinuously (e.g., big jumps in reported cannabinoids when a grower switches from one lab to another).

      Also to be very clear Terry. I am not causing bad (or good) reputations for any lab or any grower.

      The data being reported by some labs suggests that the numbers being reported for their clients MAY (in come cases) be inflated and/or inclusive of products that should have failed QA testing that have passed QA testing.

      I’m fine with the data leading to people (even consumers) beginning to question some of their pre-suppositions regarding, for example, the accuracy of the labelling on regulated cannabis in some states.

      For that matter, I’m fine with data doing whatever it wishes to do. If people wish to be seduced by data to a specific world view, then all the power to them. Rest assured the data don’t care. They are inanimate.

    2. I have not forgotten that fact, Terry.

      I suppose it’s also theoretically possible for all of them to gravitate toward a subset of the labs.

      However, when one looks at the data —- particularly the stuff where growers are submitting multiple samples to multiple labs in the same month —- it increasingly seems to be the case that some of the growers are “good at choosing labs”. I don’t know that that necessarily equates to them having better genetics and/or manufacturing processes.

    1. Well, I just think you shouldn’t just come in and stir the pot, before you have any evidence good or bad. You leaving it to the consumer to evaluate your data is from my perspective propaganda. Even the state is tapping in the dark. How about they create a cross reference lab and hire some scientist and figure it out, before releasing some numbers. As always statistics can be twisted.

      1. Terry — I truly don’t understand what you mean by this. First of all — when a pot is in need of stirring, I tend to stir it. Better than letting things burn and, potentially, ruin the pot.

        What makes you suggest that there is a lack of evidence here and why is allowing the consumer (reader) to evaluate the data I present a bad thing?

        Personally, I’ve always believed it to be a good thing to engage people and try to get them to think about patterns in the data, rather than just packaging something up and trying to guide them to see the data the same way I do (or someone else does).

        If you think something has been “twisted” here, by all means be specific.

        As always, I stand behind what I wrote in the post.

        I also always appreciate it when someone points out a viable alternate way to interpret what is being seen or (good Lord forbid!) points out an error I have made in either my data summarizations, analysis, logic or interpretation. I wish I could say that I appreciate your comment this way, but I just don’t understand your point(s).

        1. Jim — You’re posting your arguments with supporting data that lead you to those arguments. Where I’m from that’s not pot stirring and propaganda (wtf?) that’s advocacy with showing your math homework. If any one wants to dispute this effectively, they need to get the data themselves, do the math, and see if their answers and conclusions match or don’t match. Post it and let the consumers evaluate both sets of data and conclusions, and the relative worth of the person’s experience in offering up these analyses as to which is more accurate. That’s how I’d like to see it done. Sniping from the sidelines without doing any work yourself? Worthless.

          Please do more Nevada work. We absolutely need an advocate like yourself down here.

          1. Thank-you, Darin. One hint for anyone doing what you suggest — it is good to remove records in which any cannabinoid percentages reported as having values that are either less than zero or greater than 100. Removing those was the primary modification I did to the data before getting into analysis and summarization.

            I expect to hear back from the state of Nevada by the end of this week regarding my request for updated and unblinded data.

            With luck, they will take the path of transparency in support of their public safety and consumer protection goals.

  4. I went to probably ten different dispensaries in Nevada looking for a flower with lower THC values and higher CBD values. After finding zero options I just so happened to stop at a Hemp store to browse what they were selling. I found a multitude of CBD flower strains there. The market is definitely driven by its THC values. It is the equivalent of a Pharmacy for Ibuprofen and only being able to purchase Hydrocodone. At this point I am having to weigh and mix my own doses to keep from getting completely stoned in the mornings. Also the laws are very vague about growing for personal use and the “rules” seem to conflict.

    1. Travis — I hope your ability to source the products you need improves over time.

      One caution — given what I saw with how some of the labs in Nevada seemed to be reporting higher-than-expected levels of THC in the cannabis flower they tested, I’d recommend that you take the “low” THC values being reported on the Hemp flower with a large grain of salt.

      The same “for-profit” motives that lead a cannabis-test to have higher levels of THC than are actually there would be expected to lead to having hemp flowers having test results that report THC levels that are lower than are actually in the flowers.

      I have not read Nevada’s laws and rules on this but it is my understanding that the primary discriminator between HEMP and CANNABIS is whether the cannabis sativa being tested has less or more than 0.3% THC by weight.

      Your Hemp may actually have more THC than the label says —- and your Cannabis may have less than it’s label says.

      Such a complicated world for consumers.

Leave a Reply

Your email address will not be published. Required fields are marked *