AI Powered Video Assistant Referee with Amazon Rekognition Custom Labels

Original article was published by Olalekan Elesin on Artificial Intelligence on Medium


Ever since the introduction of VARs (Video Assistant Referees) to football, many have expressed their doubts towards the objectives the initiative was supposed to fulfill. When introduced at the last FIFA World Cup in Moscow 2018, a number of football greats including Diego Maradona and Javier Zanetti expressed the improvements the technology would bring to the game they love. According to the FIFA website,

“Technology brings transparency and quality and it provides a positive outcome for teams who decide to attack and take risks.” — Diego Maradona Former Argentine professional footballer and manager.

“I support the use of VAR. I believe it is an element of greater justice for the game and for the teams.” — Javier Zanetti Inter Milan Legend and Record national player of Argentina

FIFA VAR ROOM. Russia 2018. Source: FIFA Website

These comments were made way back in 2018 at the Russia 2018 World Cup. However, ever since the introduction of VAR to English Premier League, EPL, there has been an increasing distrust from fans and sport pundits in the initiative. This is largely due to inconsistent outcomes in with respect to VAR decisions. These outcomes range from penalty decisions, red/yellow card rules, to mention a few.

A recent game between Manchester United and Tottenham Hotspurs which ended 6-1 in favour Tottenham Hotspurs, saw Anthony Martial dismissed for violent conduct against Erik Lamela. In defense of Martial, he was provoked by a shove to his face by Lamela which the referee did not see happen. Lamela was then awarded a yellow card after 2 to 5 minutes of consultation with VAR. In an ideal game, both players should have been sent off.

The example above is only one out of many. The question everyone has been asking is: What if VAR could be more consistent? I would argue that this very difficult to achieve as VAR officials are humans and can attend one game at a time. This variation in VAR personnel, coupled with the fact that VARs have different experience levels is the underlying reason for the inconsistent outcomes in decisions. But wait — can AI help VARs achieve more consistent outcomes by augmenting decisions? Probably, yes. At its core, VARs make decisions based on video data (continuous stream of image frames) in real-time. These decisions are informed by their past experiences as referees or trainings. Sounds like a task for machine learning, right? Enter my experience.