Photo: Barb Colligon
Earlier this week, PSP reader Paul Baker sent me an email noting Jack McInerney’s high ranking on the latest Castrol Weekly Top 20, in which the young Philadelphia Union striker came in at No. 9 (Josue Martinez came in at No. 3 behind Chris Wondolowski and Robbie Keane). Paul described how he sees little commentary on the Castrol Index, despite the fact that it claims to be be based on objective analytics. Paul wondered how the rankings might compare with the player ratings that PSP does after each game. I link to the Castrol Weekly Top 20 in our news roundups so I decided to find out.
How the Castrol Index works
As “the Official Performance Index of Major League Soccer,” the Castrol Index “uses the latest technology to objectively analyze player performance” by tracking “every move on the field and assesses whether it has a positive or negative impact on a team’s ability to score or concede a goal.” The Castrol Index FAQ page on the league website notes, “A key factor for all areas of performance in the Castrol Index is in which zone on the pitch the action takes place.” So, events such as passes, tackles, interceptions, blocked shots, and so on, are first awarded or deducted points by “Castrol’s team of performance analysts” depending on whether or not they are successful. But a successful pass in the box leading to a shot on goal is given more weight than, say, a possession pass between two defenders their own defensive third.
Leaving aside reasonable questions about the subjective interpretations an analyst might have to make in deciding, in the example of a misplaced or intercepted pass, “how much trouble the mistake is likely to land the team in,” not to mention whether or not (or how) elements of the game such as off-the-ball movement are measured, the Castrol Index is an effort to engage in a meaningful, yet objective, interpretation player performance from basic match statistics.
Castrol Index vs PSP player ratings
While informed by match statistics, the kind of player ratings that PSP has produced for three seasons are at their heart inherently subjective in that that are based on a writer’s viewpoint. They also take into account the kind of intangibles that statistics have difficulty measuring. And, because they are opinion based, they’re usually more fun to read, producing the kind of conversations websites such as PSP value so much in the comments section.
Acknowledging the fact that the Castrol Index and PSP’s player ratings are different animals, how do they compare?
Broadly speaking, I think they compare pretty well. The principal difference to me is the placement of Jack McInerney and Zac MacMath in the first and second spots on the Castrol Index. In PSP player ratings, MacMath comes in at No. 7, two spots above McInerney. But, with a few exceptions, the rankings of the PSP player ratings averages look like a typical Union Starting XI under John Hackworth. Certainly, the high rankings of the Union backline reflect the fact that the Union defense has been the most consistent part of the team (especially since Porfirio Lopez disappeared from the Starting XI). The exceptions—Raymon Gaddis, Antoine Hoppenot, and Chandler Hoffman—have each gotten starts at various points in the season. The Castrol Index rates Gabriel Gomez higher than many Union fans would probably rate him, particularly on recent form. The Index also includes Freddy Adu, Michael Lahoud, and Danny Cruz in the top 11 whereas the rankings of the average of PSP’s player ratings does not.
It is also interesting to note that the difference between Hoffman at No. 11 and Josue Martinez at No. 18 is only .53 points.
So, how to explain some of the differences?
The Castrol Index FAQ page explains that “every move on the field” is assessed based on “whether it has a positive or negative impact on a team’s ability to score or concede a goal.” No particulars are described but it is easy to assume that scoring a goal boosts a player’s numbers. With Adu and Gomez leading in goals scored on a team that has had trouble scoring goals much of the season, they presumably gain points in the Index. But both players, to what I think would be wide agreement among Union fans, have disappointed more than they have scored.
This falling short of expectations is precisely the kind of thing subjective player ratings take into account. Similarly, Gaddis and Hoppenot are likely to rank better in the averages of PSP player ratings because, as rookies, they have exceeded expectations. (And I assure you, Hoppenot’s ratings have dropped sharply as the shockwaves from his explosion on the scene as the Union’s spark plug have diminished through the recent long scoring drought.) That Williams is slightly below Gaddis is probably because his performance suffered early in the season under the tactical experiments of Nowak and also because he played much of the season with an injured foot.
Additionally, the FAQ page says players “who have not played a specified amount of minutes each month (equal to approx 60% of season game time) will have their Castrol Index score divided by that amount of minutes, therefore penalizing players who are either injured, not selected or suspended for a considerable period of time.” PSP’s player ratings make no such claim, which helps in understanding why Gaddis and Hoffman might land higher in PSP’s rankings than in the Castrol Index. In other words, we just thought they generally played better than some of their teammates when they were on the pitch.
In the end, whether one ranking is objective and the other is not does not in itself make one better than the other and both rankings provide good fodder for conversation. Looking at the Castrol Index, does Martinez “deserve” a higher ranking than Hoppenot? Looking at PSP’s rankings, does Hoppenot “deserve” a higher ranking than McInerney (however slim the difference might be)? Comparing the two rankings, do you think Adu has had a better season than, say, Gaddis? And so on.
We have our own favorite players just as, no doubt, you do.
Feel free to discuss below. I’ll do another comparison after the end of the season.