We have a deep commitment to impartiality, open-mindedness, and fairness. All of our tastings, ratings, and evaluations are done by trained panelists using our proprietary blind tasting methodology and custom-built software. Tastings powered by the Beverage Testing Institute will never accept remuneration for a favorable review; we believe in reviews based on objective, benchmark-based criteria, not subjectivity and bias. We do not require that products adhere to strict style guidelines. Our panels evaluate each product monadically, on its own individual merits. How do we ensure the fairest, most consistent beverage analysis methodology available? Let us explain.
One of our blind tasting panels using BTI's custom software
To ensure consistency and accuracy, the Beverage Testing Institute uses a dedicated tasting lab located in our offices in Chicago. The room is specially designed to minimize external factors and maximize our panelists' concentration. We taste in the morning, under the same ideal conditions—yielding far more consistent results than working out of a suitcase, a trade show, or at a producer's facility. We only taste 25-30 wines, 25-35 beers, or 12-18 spirits in our morning sessions to prevent palate fatigue. Tasting more than these amounts in a single day, as most other competitions and many magazines do, negatively affects the accuracy, quality, and consistency of alcoholic beverage evaluations. And of course, we always use a spittoon.
We conduct daily invite-only judging panels that include highly experienced, professional tasters. Many of our panelists have completed certifications of their own such as SWE, CSW, CSS, MS, Cicerone or others from credible institutions. Our panelists are industry buyers and experts such as beverage directors, educators, retailers, restaurateurs or prominent writers. All panelists are rigorously screened and audited and then trained in our proprietary blind tasting methodology. They follow strict protocols for maintaining the tasting room environment and priming their palate prior to analysis. For further information on our panelists click here: Meet our Judges
All products are tasted in Riedel crystalware
A basic tenet of qualitative evaluation is the shorter the scale, the easier it is for panelists to repeat their scores — and repeatability is the essence of accuracy. Our panelists taste in two rounds. We use a simple scale with four bands in the first round:
With this scale, panelists do not concern themselves with the difference between an 88 and an 86, or a 90 and a 91. Products with two or more judges scores of "4" are sent to the "merit round," whose range is as follows:
This method allows our tasters to consider general quality, without the distraction of adding points. Products breaking the critical 90-point gold medal barrier are tasted twice without exception, virtually guaranteeing that they deserve their accolades.
The scores are then converted to a modified consumer-friendly 100-point scale. The below five quality bands of score more accurately reflects the quality of products in today's market. It roughly corresponds to a five-star system:
To help translate the panels' scores, we use a different statistical method, relying on the mode (the most frequent score) rather than the mean (the average score). This helps reduce the impact of maverick scores; the mode is much closer to what the panelists are really saying. If, for instance, a product in the first round receives three scores of "3," it is placed in the upper center of its band (85-89) and given a final score of 88 points. Should the third score be a 4 or a 2, the product in question would be given an 89 or an 86, respectively. The third score is used to move the total score up or down within the same band. Again, controversial permutations are re-tasted. Those falling in the lowest band (less than 80) are simply listed as "79/NR" (Not Recommended) and no specific score is assigned.
Editorial re-taste session in action
BTI employs a system of checks and balances that makes us unique in the industry. We take a thoughtful, labor-intensive approach to ensure the fairest most consistent score. As we like to say, "More Care = More Fair". To accomplish this, we maintain an archive of all previously reviewed spirits brands—over 6,500 samples and counting. When there is a disparity between a current sample's score and the product's previous BTI score we compare the current sample with our archive sample to verify any differences in taste and/or quality. We re-taste any products that seem flawed or that scored at the lowest end of the spectrum, to make sure a bad sample was not at fault.
Crosschecking disparate scores against archive samples
Best Buy: Wines or spirits which provide uncommon value.
Best of Category: Given to the highest scoring product in its category for a given year.Cellar Selection: This is a wine that we believe will improve significantly with at least three to five years of age.