In 2014: The Year Video Games Broke My Heart Part 1 I detailed how the quality deficit in Halo: The Master Chief Collection needs to lead to a change in behavior: no more pre-orders and no more day 1 purchases.
In the last few days, two game review sites demonstrated real ethics in game journalism (more on Gamergate’s fake kind later). They've changed review practices partly or wholly in response to game quality levels and the previously controlled conditions reviews were conducted under.
Polygon has adopted a system called Provisional Reviews, which I think is fantastic. In the Halo Master Chief Collection case, this would have meant that as soon as launch day and beyond issues became apparent, Halo MCC would not have been issued a high final of 95 vended out to Metacritic (cmd-F/ctrl-F for Polygon):
The update on Polygon’s Halo MCC review 5 days after launch from an 8 from 9.5 didn’t affect Metacritic at all. With Provisional Reviews, Metacritic would have most likely got the low quality adjusted review score. This will directly affect game studio and publisher bonus structures since studios are often awarded for 9+ Metacritic scoring game.
Eurogamer.net has dropped review scores entirely and will not submit to Metacritic. Most interestingly, they are taking steps to guarantee they are reviewing paying retail customer experiences as well:
We are also changing (or firming up) other areas of our reviews policy, with the intention of ensuring that we always review the same experience that you get when you buy a game. This means that we will only review from final retail versions and online games will be reviewed after they've launched.
Eurogamer will issue first impressions of games on launch day, but also not use their new system to issue a near immutable final recommendation until they can asses the game under real world conditions. Fantastic!
Hopefully other game review sites follow these two sites lead in:
- Don’t issue final scores based on debug build games, only retail copies
- Don’t issue final scores based on controlled network conditions for multiplayer heavy titles, only on
- Don’t issue final scores to Metacritic (if at all) until a games total experience can be judged at launch under real work usage