The wine-loving audience at Rosalia Festival 2018 could rate the candidate wines competing for the Rosé of Rosalia Festival award using either the Vinometer app or through the ‘Vinometer Events’ web platform dedicated to wines evaluation.
We wanted to find the most popular rosé wine at the Festival, more precisely the rosé wine that the Festival audience considered the best. We didn’t just want to know which wine was tasted by most of the guests, was the most popular according to ‘absolute’ criteria or received the highest average rating. If we had asked each guest to give each wine available at the Festival a score, we would have been able to name the number one wine; of course, that was not an option.
So, what could we do to pick the best one?
Some events take the average of the scores awarded to the candidate wines as a basis. This method works well if many guests taste and rate almost every candidate wine. Let’s assume that a rosé wine from Villány received 50 votes with the average score of 4.1, and a wine from Eger received 50 votes with the average score of 4.4, I don’t think it’s too much of a leap to conclude that the latter one is better. But what if there is a third wine that received 2 votes with the average score of 5? The third wine should win? Your intuition is certainly telling you that something is wrong about it, as I dare say that if plus 48 people had rated the third wine, it would have received a completely different average score, as differences in tastes and preferences are significant, two votes thus do not provide a representative picture. (As it would also be unwise to conclude the outcome of elections from your neighbour’s political affinity.) We can say that a wine cannot compete if it receives a certain number of votes below the threshold of exclusion. However, we’ve considered this approach as insufficiently precise and elegant.
Another common method would be asking guests about their favourite (or three favourite) tasted wine(s), and then one could determine which one has been mentioned most often and proclaim that wine to be the Rosé of Rosalia Festival. The main problem with this method is that it would place small wineries at a disadvantage. Since, naturally, fewer guests visit the booth of a winery that is less well-known, even though it produces high-quality wines, than the booth of a large, well-known winery. Thus, the vote would be a popularity contest. This is not what we wanted to know.
Finally, the Vinometer staff chose a more complex mathematical solution. It is based on the following assumptions:
– The average of the scores given by the voters at the Festival represents the ‘average’ rosé. Therefore, for purposes of illustration, if the average of the scores given by more than 1000 voters is 3.9, this rating represents the average (regardless of the fact that voters rate wines using a 1-5 scale).
– So long as a wine has not received any vote, the least faulty assumption is that the given wine is considered as ‘average’. Just think of a blind tasting of wines produced in a wine region unknown to you: If you have tasted ten wines and liked all of them very much, it gives you good reason to assume that you will also like the eleventh wine produced in that wine region. Well, you may not, but more likely, you will. If you have eaten similarly priced mediocre meals in restaurants located closely to one another, you will likely get a mediocre meal in another nearby restaurant. The restaurant may surprise you with an outstanding (or terrible) meal, but, on the basis of your past experience, the meal ordered there will likely be of the same quality as all the others.
– On this basis, the initial score of a wine yet untasted is, let’s say, 3.9.
– If the wine receives a vote, it means new information. Let’s assume that a guest rated the wine with a 5 score. What does that mean? Can it be said that the wine has a 5 score? Sad to say, that is not yet the case, as many votes from guests with different taste preferences are yet to be submitted, so we must rely on the average. The most likely assumption is that the wine is little better than the ‘average’ wine, as it has been rated as excellent on the basis of one vote that constitutes a very small sample size. Now, let’s say, the wine has a 3.95 score.
– The more voters rate a wine, the more certain we can be that the scores given are representative of the real popularity of the wine, and the more we can rely on the average of the scores given to the wine, instead of the average of the scores given to all wines. So, if 40 votes are submitted and the average of the scores given is 4.3, we can say that our estimate of what would be the average of the scores given to a wine after the submission of all votes is between the rating of the ‘average’ wine and the average of the scores awarded to the given wine, more precisely closer to the latter rating; let’s say 4.26.
The above principles were defined on the basis of a pre-determined equation (to math experts: simplified Bayesian estimation) that takes the total number of votes, the average of scores, the total number of votes submitted for a wine, and the average of scores awarded to a wine into account. This method allows less well-known wines to win the vote (less votes may be enough, but the scores need to be very high), but it does not give them an undue advantage (two votes with a 5 score mean a lower corrected score than fifty votes with an average score of 4.8). This method is based on a well-defined methodology that excludes subjective judgments.
You can check what is the outcome of the vote calculated on the basis of the above method and the votes of the audience in the following blog post: The Most Popular Rosé Wines at Rosalia Festival.