Methodology on Blind Tasting

We have a deep commitment to impartiality, open-mindedness, and fairness. All of our tastings, ratings, and evaluations are done using our proprietary blind tasting methodology and custom built software. Tastings & Beverage Testing Institute will never accept remuneration for a favorable review and we believe in reviews based on objective, benchmark based criteria, not subjectivity and bias. Why do we taste blind? Let us explain.

The Tasting Lab

To ensure consistency and accuracy, Tastings uses a dedicated tasting lab located in our offices in Chicago. The room is specially designed to minimize external factors and maximize our panelists' concentration. We taste in the morning, under the same ideal conditions, which is far better than working out of a suitcase, a trade show, or at a producer's facility. We only taste 25-30 wines, 25-35 beers, or 12-18 spirits in our morning sessions to prevent palate fatigue. Tasting more than these amounts in a single day, as some other competitions and many magazines do, negatively affects the accuracy, quality, and consistency of alcoholic beverage evaluations.
And of course, we always spit.

Our Panels

We conduct weekly invite-only judging panels that include highly experienced, professional tasters. Many of our judges have completed certifications of their own such as SWE, CSW, CSS, MS, Cicerone or others from credible institutions. Our judges are industry buyers and experts such as beverage directors, educators, retailers, restaurateurs or prominent writers. All panelists are rigorously screened and audited and then trained in our proprietary blind tasting methodology. For further information on our judges: Meet our Judges

Our Scoring

A basic tenet of qualitative evaluation is the shorter the scale, the easier it is for tasters to repeat their scores — and repeatability is the essence of accuracy. Our tasters taste in two rounds. We use a simple scale with four bands in the first round:

  • 1 — Not recommended   (less than 80 points)
  • 2 — Of sound commercial quality, though not overly exciting   (80-84)
  • 3 — Shows style and character, yet probably not of the highest merit   (85-89)
  • 4 — Highest quality   (90+)

With this scale, tasters do not concern themselves with the difference between an 88 and an 86, or a 90 and a 91. Products with two or more judges scores of "4" are sent to the "merit round," whose range is as follows:

  • 3 — Very good, but not of the highest merit   (88-89 points)
  • 4 — Truly excellent in style and distinction   (90-92)
  • 5 — Outstanding, though not quite one of the world's finest   (93-95)
  • 6 — Provides a world-class experience   (96-100)

This allows our tasters to consider general quality, without the distraction of adding points. Products breaking the critical 90-point barrier are tasted twice without exception, virtually guaranteeing that they deserve their accolades. Also, after the first round, those with a wide disparity in score are re-tasted. We also re-taste many that scored poorly, to make sure a bad sample was not at fault.

The scores are then culminated onto a modified 100-point scale. The five quality bands below more accurately reflect the quality of products in today's market. It roughly corresponds to a five-star system:

  • 96-100 — Superlative   (Platinum Medal)
  • 90-95 — Exceptional   (Gold Medal)
  • 85-89 — Highly Recommended   (Silver Medal)
  • 80-84 — Recommended   (Bronze Medal)
  • less than 80 — Not Recommended   (No Medal)

To help translate the panels' scores, we use a different statistical method, relying on the mode (the most frequent score) rather than the mean (the average score). This helps reduce the impact of maverick scores; the mode is much closer to what the panelists are really saying. If, for instance, a product in the first round receives three scores of "3," it is placed in the upper center of its band (85-89) and given a final score of 88 points. Should the third score be a 4 or a 2, the product in question would be given an 89 or an 86, respectively. The third score is used to move the total score up or down within the same band. Again, controversial permutations are re-tasted. Those falling in the lowest band (less than 80) are simply listed as "79/NR" (Not Recommended) and no specific score is assigned.

Other Awards and Designations

Best Buy: Wines or spirits which provide uncommon value.
Cellar Selection: This is a wine that we believe will improve significantly with at least three to five years of age.