Home > Reviews > An Analysis of Review Scales

An Analysis of Review Scales

January 31, 2011 Leave a comment Go to comments

Or: Why I Use the 5-point Review Scale

The most confusing thing in the world when it comes to reading reviews is that there are over 9000 (so to speak) different review “scales” that are used to rate movies, music, games, and many other works on an absolute scale. Some sites use a 10-point scale, while others use 6,5,4, or even 100-point scales. I have even seen some websites like 1up that use a standard academic grading system from A+ to F! This is further confused by the different reviewers’ references for scoring; for example, Final Fantasy is absolutely awful when reviewed like a linear, 3D, first-person shooter, but when reviewed like a role-playing game it’s pretty great. With so much confusion when it comes to reviewing works, what is the “correct” way to review? What is the correct way to read reviews? In this voluntary essay, I will logically analyze the various scales and styles used to review various things, as well as the right way to read these reviews:

The first problem you come across with the various review scales is that the human mind cannot absolutely rate a work due to several factors:

  1. Experience. One person’s experience with a work might be vastly different than someone else. For example, a reviewer of a video game might encounter a game-breaking bug that other reviewers didn’t encounter, so their score might be lower than the average. Or a reviewer of an emotional movie might not like it much due to being annoyed throughout the movie by someone else, therefore losing all emotional tension the film might have presented.
  2. Different types of works require different expectations. For example, a movie that is meant to be scary, reviewed by someone who is easily frightened, might not like the movie since it scared him, even though that was the point of the movie in the first place.
  3. Works are made for different audiences. Don’t review a children’s television show like you would a horror movie or a thriller; rather, rate it as if you were a kid watching it, even if you aren’t.
  4. Most importantly: It’s absolutely impossible, if not extremely challenging, to absolutely rate a work on an extremely large scale. 99 out of 100 is so insignificantly different from 100 that it almost doesn’t matter in the long run.

This is further confused by the differences in meaning between the levels/points on the scale. A 5 out of 5, on one scale, might mean that the work is perfect and free from flaws, while on a more conservative scale it could simply mean that the work was considered very good and highly recommended. Likewise, a 1 out of 5 might mean that a work had no redeeming value whatsoever on one scale, while on another scale, it could mean that the work was just not worth attempting to enjoy, regardless of any redeeming value it might have. Also, most scales evenly space the different ratings out. For example, a 3 out of 5 might mean that a work was mediocre. However, some scales aren’t evenly spaced and use 3 as a general indicator that it was enjoyable, leaving discontent to the ratings 2 and 1.

Then we get to sites such as Metacritic and Rotten Tomatoes, which bring these vastly different (and sometimes inaccurate) review scales and attempt to average them. Generally good works often get 60-ish ratings out of 100, while really good works often get higher than that. Anything below 60-ish (or 6-ish on an out-of-10 scale), in most cases, generally means “don’t even bother trying this”. This is coupled with the fact that there are different reviewers for each work almost always. The extremely large review scales that these sites have leads us to our next issue…

…which is fanboy-ish wars between minor differences in score. Lets say that a website rates a The Legend of Zelda game at 99 out of 100, but then rates a new RPG 100 out of 100. Does that mean that the RPG is better than LoZ? If so, by how much? The problem with figuring that out is that the difference between 99 and 100 is so small that it is almost non-existent, especially considering that both games are drastically different types of games with different expectations.

There are so many things to consider when choosing the right review scale, as I’ve made obvious in this essay. Which, then, is the best? Lets deduce this logically:

  • It is superficial to have an extremely detailed point/percentage system, so the maximum score has to be reasonably low number, such as 4, 5, or 6. The reason I didn’t say 10 or so is that, as noted earlier, everything below a 5 or 6 (“mediocre” or “average”) on that scale means “don’t bother”. Making the review scale small-ish makes your ratings have more impact and makes them less superficially different from each other, eliminating the problem of being able to decide between ratings rather effectively.
  • The medium rating (3, 5, or whatever rating is in the middle of your scale or close to it), cannot mean “mediocre”, as that effectively makes every rating below it not worthwhile. The majority of the ratings (if any), therefore, should be approval-centered. Why? Because if you read that something is mediocre or worse, it should be obvious that it’s not worth trying anyways; why go into explicit detail about the exact level of disapproval?

By now, it should be clear why I use a 5-point system in my reviews. The difference between ratings carries impact, yet there are enough of them to make a clear distinction between my various opinions. Newgrounds.com uses a 6-point system with an average, but the difference between 0, 1, and 2 is rather superficial to me, so I don’t use 0 and just use 1-5 as my point range. Roger Ebert, a popular film critic, uses 0 through 4, with 0 being given to the worst, a half-star being given to the next step up, and so on. While his scale makes sense mostly, it doesn’t carry enough visual impact for me, and seems to focus on negative films compared to positive ones. My scale, similar to X-Play’s system, is a 1 for things that I wouldn’t recommend to anybody, 2 for things that I only generally disliked, 3 for things that I generally liked, 4 for things that were very good but not worth a full recommendation, and 5 for the things that I loved and think are definitely worth a try by anyone.

Reviewing is tough, complicated, and confusing. Writing reviews needs a sharp, keen eye that can spot the problems with a work relative to what it was supposed to be. Even more important than your opinion, though, is the scale you use; make sure it’s accurate enough to express your opinion generally, but don’t make it too large or small. Even if you’re reading a review that uses an inaccurate or confusing scale, be sure to compare it to other reviews and examine the specific criticisms and look for possible bias. I hope that through this essay, you have seen how confusing reviewing is, and how simple, yet informative, it should be.

Advertisements
Categories: Reviews Tags:
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: