ZBlogW

Managers can’t perform magic, although some people seem to think they can. They’re not David Copperfield or Harry Potter. They can’t work miracles or sprinkle some magical dust to make players know how to play football.

Spending hours on analysis isn’t very useful. It doesn’t put you in better conditions to win the game. The tactics, the schemes, they’re all bull***t. Of course tactics matter, but players win the game. For 45 minutes at a time, players make their own decisions. Football is a continuous sport in which the coach has barely any influence, less than in any other sport.

Advertisement

Success depends on having top players and top players cost money. It’s impossible to achieve results without spending a lot of money. Where you finish in the league depends on the money you’ve spent. It’s a statistical fact. Clubs can change coaches but if the players are the same, results stay the same.

Managers’ influence in the game is much less than people believe. They choose the players but their role is limited. This is a game played by the players. Coaches’ role is less than many of them realise or want to believe.

But don’t take my word for it. Everything you just read came straight from the mouths of Pep Guardiola, Jurgen Klopp, Antonio Conte, Marcelo Bielsa, Massimiliano Allegri, Erik ten Hag, Ernesto Valverde, Juanma Lillo and, of course, Sam Allardyce, who’s up there with all of them.

Football’s main character

If you listen closely to coaches, they’ll often describe their job very differently than you’ve been led to believe.

It’s not that they’re not all accomplished, smart, incredibly hardworking people who know far more and think more deeply about football than the rest of us mortals. It’s just that, when you get down to it, that’s not what wins games.

On TV, on front pages, on the internet, in the stands and down the pub, football belongs to the managers. They bestride the narrow pitch like a Colossus. It’s almost impossible to tell a story about the game that doesn’t cast a coach as the main character, and even if you did, who would want to hear it?

Everything managers say or do makes headlines. When cameras roll or a thousand microphones bloom anywhere near a stadium, they’re usually pointed at a manager or somebody talking about a manager. If a manager takes a job or leaves one, it’s the story of the season. When journalists and fans talk about a match, we describe what the manager “did” on the field. We say the manager won or lost.

Advertisement

But what if we’re thinking about it all wrong?

Wages win games

Do managers even matter? It sounds like a crazy question to ask about a profession of extremely famous people whose employers trip over each other to lavish them with millions. Across decades of quantitative studies, though, it remains a surprisingly open question.

If we could design an experiment to answer the question, we might randomly split clubs into two groups. “The treatment group would get managers,” the economist Stefan Szymanski explains, “and the control group would be run as workers’ collectives.” Only by comparing teams with managers against others without one could we directly measure whether the most visible job in the game actually helps teams win.

Since they can’t turn the football pyramid into a science fair project, researchers have to come at the question in more creative ways. The simplest is the case laid out by Szymanski and Simon Kuper in their book Soccernomics. Averaged over a decade or so, the amount Premier League and Championship clubs spend on wages turns out to be incredibly closely correlated with results, to the extent it looks like money is almost all that matters. Maybe the game really is played on a spreadsheet after all.

The takeaway, in the authors’ view, is that the global market for talent is good at paying players what they’re worth. Clubs that spend more money can afford to hire better players, and good players — not coaches — win games. After all, if wages alone can explain around 90 per cent of the variation in league position, that doesn’t leave much room for managers to matter.

“It is tempting to think they could be replaced by their secretaries, or their chairmen, or by stuffed teddy bears, without the club’s league position changing,” Kuper and Szymanski write.

Advertisement

So is that it, case closed? Managers in the mud? Not quite.

A few years after Soccernomics, two football-loving professors responded in The Numbers Game, which took a different view of the relationship between wages and results. For one thing, they noted, Szymanski’s wage data covered all club employees, managers included. It makes sense that clubs that can afford the best players would hire the best coaches, too.

They also pointed out that on a year-to-year basis, as opposed to the long term, wages only explain about 70 per cent of the variation in league position. That’s still more player power than the hype around managers might make it seem, but if one employee was responsible for nearly a third of a club’s performance each season, that person would be hugely important.

The challenge is figuring out how much of the remaining 30 per cent is actually in coaches’ control. The market for players may be competitive but it’s not perfect, and in any given season, most clubs are bound to be under- or overpaying their squad relative to its talent, accounting for some of the annual fluctuation.

Football also involves an awful lot of luck. “Injuries, dodgy referees, poor form, and any host of other factors cause big swings in performance from year to year,” the Soccernomics authors argued. One reason to prefer long-term averages to single seasons is that they help smooth out statistical randomness.

The difficulty for the great manager theory in all this is that a team’s swings in performance relative to its wage bill are rarely predictable enough to look like the product of skill. “The fluctuations from year to year tend to be random,” Szymanski writes. “So if this variation is due to managers, then they don’t seem capable of very much consistency.”

Sacking the manager doesn’t help

One reason it’s hard to shake the certainty that managers matter is that we see evidence of it every season: some struggling club fires its coach, a new boss comes in and the squad that used to look hopeless suddenly starts winning. Why didn’t they throw the first bum out sooner?

Advertisement

The new manager bounce is real — sort of. Teams really do play better after sacking an underperforming head coach. According to one Dutch study of the English Premier League between 2000 and 2015, clubs that replaced their manager midway through the season saw a “highly significant and positive” bump in performance under the new head coach.

Not only do managers hold the keys to their team’s fortunes, it’s tempting to conclude from the evidence, but anyone can see who needs sacking. A suitable replacement ought to be able to turn the team around. If managers are important to their club’s success, that’s what we would expect and it seems to be backed up by results.

The problem with this way of thinking is that clubs don’t just fire their manager at random. They usually do it after a run of poor form. What we really need to know, if we’re trying to figure out how much blame to pin on the coach, is whether the team might have bounced back for some other reason — player availability, scheduling, just plain luck — even if they hadn’t sacked their manager.

Researchers have used clever methods to dig into that critical counterfactual. The Dutch researchers relied on the difference between results and bookmakers’ odds to compare each coaching change to some other season where the same club hit a similar slump but chose to stand by its manager. Another study of nearly 30 years of English coaching changes controlled for a variety of factors associated with manager sackings (with recent results unsurprisingly leading the list).

When they compared clubs that replaced their coach to ones that didn’t, the studies found the new manager bounce vanished. “Teams experiencing a poor run of results eventually tend to recover whether they change their manager or not,” the researchers explained. The performances under a new manager weren’t any better than what would have been expected if the club had stuck with the old one — if anything, they were slightly worse.

Dozens of similar papers have looked at coaching changes in leagues around the world and the story is broadly the same. Studies that don’t account for struggling teams’ tendency to snap back to normal may see signs of a new manager bounce, but those that do tend to find changing coaches has a negative impact or none at all.

In other words, yes, the new manager bounce is real, but it could have just as easily been a new kitman or new ballboy bounce. Maybe the club should have simply held a press conference to announce a vibe shift. Academics sometimes talk about the “scapegoat theory” of manager sackings, arguing that they’re little more than high-profile PR stunts. Soccernomics compares them to ritual human sacrifice.

Advertisement

If sacking the manager doesn’t actually help, why do clubs keep doing it? Firing head coaches is an expensive hobby. One possibility is that most people, even directors and team owners, just aren’t big on counterfactuals. Or maybe they know what the research says but think they’re better than average at assessing coaching talent, the same way everyone believes they’re a good driver.

Even though it doesn’t make much difference on average, changing coaches could still be a smart move for a side facing relegation if it shakes things up and increases the team’s variance — their range of potential league finishes. What do they have to lose?

Managers still might matter, on that view — it’s just that clubs don’t really know which ones will help or hurt. They’re gambling on the luck of the draw.

The Ranieri conundrum

Other sports have done their own coaching change studies and most have reached similar conclusions: on average, firing a coach seems to have a slight negative effect or none at all. That’s surprising considering how much real power coaches hold in their organisations, which certainly pay them as though they’re important.

One read is that these studies don’t mean coaches don’t matter, only that they matter about as much as the rest of their elite peers. “Coaches in sports are not very different from each other,” a Freakonomics blogger wrote. “It may be true (and more than likely very true) that you are better off with a professional coach than with a random person grabbed from the stands (or no one at all). But it doesn’t appear the choice of professional coach matters much.”

Another common argument is that even if changing coaches makes no difference on average, some could still be a lot better or worse than others. A research paper at the 2019 Sloan Sports Analytics Conference used a novel method to conclude that in sports such as basketball and American football, where changing coaches has little overall effect, coaching might still account for as much as 30 per cent of the variation in teams’ success. That only raises more questions about why teams in those sports seem unable to predict which coach will help their team.

Besides, it’s not clear that we should expect coaches to matter as much in football as in the NBA or NFL, where they can call plays, stop the clock and substitute players at will. Football managers can pick lineups, but that may not matter when it’s clear who the best players are. They can change the formation, but that doesn’t make a difference on average. Given time, some might make their players better, but it’s hard to tell which managers are empirically better at player development. There’s not a whole lot of research to nail down how football managers matter.

Advertisement

One last group of studies tries to measure individual football managers’ skills by comparing their results to expected points based on wages and player availability for their team and opponents.

Most Premier League managers from the period between 2004 and 2009 perform about in line with expectations, supporting the idea that coaches are generally interchangeable. On the other hand, a fair number appear significantly better than average, including some encouraging names such as Alex Ferguson, Arsene Wenger and Jose Mourinho. There are fewer standouts in a Serie A version covering 2011 to 2014, but Antonio Conte manages to beat expectations. Seems plausible.

Does that mean there’s hope for the great manager theory after all?

Maybe, but it’s worth pointing out that the head coach isn’t the only person who might be responsible for a team outperforming its wage bill. Unlike in Herbert Chapman’s days, when the manager would spend his weekdays scouting players and negotiating transfers himself, modern sporting departments have a whole army of professionals — sporting directors, scouts, data scientists and so on — whose full-time job is to find the club the best players for its money. We still sometimes talk as if the manager is responsible for recruitment, but that’s rarely the case at modern clubs, where coaches are unlikely last more than a couple transfer windows anyway.

If we could really measure managers’ skill levels independent of their team, we might expect it to follow them around from job to job. That’s where things get slippery. For example, Rafa Benitez, one of the very top performers in the Premier League study from his years at Liverpool, looks mediocre in the Serie A study at his very next job at Inter. Claudio Ranieri, one of the least promising performers in the Serie A study, would go on to win Premier League Manager of the Year for leading Leicester City to an all-time great triumph over wages as destiny — then get sacked at the edge of the relegation zone a few months later.

Managers may matter on the margins, but who, why and how much are a lot harder to pin down than the stories we tell about them suggest. Arguing about managerial skill is a fun and interesting way to think about tactics, team selection and other details of how football is played. It’s also a distraction from the measurable reality that, much more than coaches, the game belongs to the players.

Listen to Big Sam, who summed up the gist of manager research best: “Where you finish in the league depends on the money you’ve spent. It’s a statistical fact.”

(Top image: iStock/Alex Burstow/Getty Images; design: Sam Richardson)

ncG1vNJzZmismJqutbTLnquim16YvK57k21wbWhmZ3xzfJFsZmltX2WEcLDOZp2op6SXrq24jKaYp5mXmr%2B0ecyaq62domQ%3D