Can you trust your tagging?

Three analysts. One tournament. About ten thousand clicks a match.

That is how the Dutch women's team collected their data at the World Cup in August. Every pass between two players, every build-up type, every circle entry and how it happened, keyed live on a custom keyboard where each international player has her own key. Joost Bitterling, who runs analysis and data at the KNHB, told NOS: "Per wedstrijd komen we op zo'n 10.000 events uit die we verzamelen." Ten thousand events a match. By hand. At the best-resourced hockey programme in the world.

An analyst who codes matches for international teams told me something that sits awkwardly next to that number: collect only the data that changes the probability of a goal being scored. In his words, a pass between two centre backs is low priority. Two serious operations on the same circuit, two opposite answers to the question of how much to collect.

So before we talk about what AI can do with your data, here's the question that comes first: when two different people tagged your last two matches, are the circle-entry numbers even comparable?

The trust problem in tagging is not who does it. It is whether you have decided what you're collecting for, whether everyone uses the same written definition, and whether they have practised. Trained operators agree with each other almost perfectly on simple events. The one hockey study that measured this got the same result. Meanwhile, most published analysis work never checked its own reliability at all, and most never wrote down what a "turnover" is. You don't have to invent those definitions either: there's one below, used on international matches, that you can borrow. Nobody sells an AI that tags a hockey match tactically, whatever the landing page says. What that means for you this week is at the end.

At federation level it's trained analysts, live, one to three of them in the team (NOS). At club level it's mostly the coach. A survey of 538 Irish coaches across 37 sports found 49% did no formal analysis at all, 38% were coach-as-analyst, and 13% had analyst support (Martin et al. 2018). Students and interns are common in the role: a 2024 study of 130 UK and Irish analyst job adverts found academy roles centred on collecting data and facilitating feedback (Worcester, 2024). Nobody has put a number on how many club matches are tagged by a student, an injured player or a parent. My guess is most of them. That's a guess.

Belgium codes live across the senior teams and about ten youth squads, with a 40-minute post-match turnaround and a four-zone tactical model, per the Hudl case study with Emily Calderon, the federation's senior performance analyst. The Netherlands brought a three-person analysis team to the World Cup in August, where most nations bring one or two (NOS).

You are the analyst. Or your assistant is, or the goalkeeper's dad with the iPad. That is not a problem in itself. What matters is the next three sections.

There is no study telling you how much to collect. There is a working rule from the professional end, and it is refreshingly blunt: only collect events that change the probability of a goal being scored, and agree the hypothesis and the method before you collect anything at all. The analyst who gave me that rule reckons it saves a lot of time. He wrote it to me in September and asked to stay anonymous for now.

That is the opposite instinct to ten thousand clicks a match. Both are real choices made by people who know what they're doing. The KNHB collects everything and decides later what matters; the provider decides first and collects less. Neither is wrong. What would be wrong is collecting a lot without ever having said out loud what question you were answering.

Both ways, as above. What they share is that somebody senior made the call deliberately.

Write the question before the match, not after. "Are we getting into the circle from wide or through the middle?" is a question. "Let's tag everything and see" is not, and with one pair of hands it will get you three matches of half-finished data. Part two of this series asks you to pick five numbers and write down why each one matters. This is the same decision, made one step earlier.

A review of 41 rugby analysis studies, published in January 2020, found that of the 22 using performance indicators, only seven defined the variables they measured (Sports Medicine Open). Hughes, Cooper and Nevill looked at 72 notational analysis papers in 2002 and found 70% reported no reliability check whatsoever. So the academic world that trains analysts has, for most of its history, not written down what a circle entry is or checked whether two people count the same number of them.

Here's the catch. "Circle entry" sounds obvious until you ask: does a ball that crosses the line and comes straight back out count? A deflection off a defender? A dribble that touches the line and turns away? Same tagger, different definition, different number. And "turnover", "press", "outlet success" are worse.

So here is what one of those definitions looks like in the wild. Secret Analyst, which codes matches for international teams, has shared the circle entry definition it uses for every match it does:

Notice what it does. It defines an entry by what it produces, not by the ball crossing a line. Run the three questions through it and you get three clean answers. Ball crosses the line and comes straight back out: not an entry, because there's no control, no corner and no chance of a deflection. Deflection off a defender: an entry only if it wins a corner or a stroke. Dribble that touches the line and turns away: not an entry.

I can't tell you that this is the correct definition, and neither can they. It is a choice, and it deliberately throws away the entries that produce nothing. A federation counting line crossings will get a bigger number from the same match. Both numbers are fine. Comparing them is not.

Spain's federation was rolling out standard analysis templates to its clubs in 2023, described by Andrew Wilson, its head of scouting and analysis (Nacsport blog). That is a definition set travelling with the tool. Secret Analyst's definitions are agreed across all its analysts, rarely change, and it was willing to hand one over for this piece. Whether the Dutch and Belgian federations write theirs down and share them is unconfirmed.

Write your definitions down. One line each. Circle entry, turnover, penalty corner won, outlet completed. Tape it to the laptop. It costs ten minutes once and it is the single biggest thing you can do for your numbers.

Better still, borrow rather than invent. The definition above is free and it is the one a professional operation uses on international matches. If your circle entry means what theirs means, your number is at least comparable to something outside your own club.