Accuracy of Prediction Models

[Hogmeister]

Prediction models are now widely used in FPL, and a couple of seasons ago on here I was attempting (and failing!) to assess the accuracy of the FF Hub model. Have just come across this analysis of the accuracy of all the main prediction models over the course of last season. You can find the full detail on X (the artist formerly known as twitter), posted by @midnightmuleFPL.

The results vary depending whether you assess them by absolute difference (-1 treated as +1 in any wee), or signed difference (where I think a -1 in one week would cancel out a +1 in another).

This snapshot reinforces my own expectation - that Solio Analytics would be the most accurate (data sourced from international betting markets, so relatively objective), followed by FPL Review (dedicated model running for several years)… as compared to, for example, FF Hub, where their focus, in my opinion, is on presentation and ease of use.

IMG_2146.jpeg

Personally I found FPL Review a pain to use and have given up on it, these days using a) FF Hub for a quick snapshot of the strength of my squad, GW by GW, and b) Solio for a more detailed assessment of my squad and alternatives.

By the way, I have to agree with Triggerlips about the dishonesty of some content creators when they express opinions on social media, which are more or less 100% based on one of the models, yet they don’t mention that, instead giving the impression that it is their “expert insight”. No issue with pundits using models, why wouldn’t they, but they should be transparent about it, I think.

Trigger often has a go at BigManBakar for that, and a prime example was Ben Crellin’s recent appearance on the FF Hub Youtube channel, where his No Haaland draft was a straight lift from FPL Review, just after it had launched for the season. Indeed, I have been told (by someone who knows him) that Ben Crellin has confessed that in recent years he just follows the model predictions. If so, that’s pretty sad… IMO.

[blahblah]

I did start following Solio because of the "neutral" betting market stuff; but then they started going on about their model and I yawned so big that I fell asleep drowning in their delusional quicksand.

Making a decision on players expected to get 4.6 over 4.4 is wrong on so many levels... Now if they used rolling 4, 5 or 6 GWs using WHOLE fking numbers then I could get interested, by it's a load of hot taking click baiting lemmings trying to make money imho.

And yes, there should be more honesty about which model(s) these "experts" follow and how closely.

Admittedly I don't take it as seriously as some and the only one I regularly watched last season was LetsTalk as he's more older school FISO discussion than the dictatorial "the model says....." of others.

[zipnolan]

I agree that Solio is the best , though I will caveat that by saying that I, too, have found FPL Review unwieldy. It's probably better than I have experienced.

At the risk of being accused of being sucked into delusional quicksand, I would also say that Solio is also great in being customisable and configurable. If you don't agree with the relative weightings of fixtures, team strengths or expected minutes of players, for example, you can just change them yourself. Yes, it has the slight gimmick of having an infrastructure based on betting markets, but this year it has Opta data integrated into it, so those external objective measures are baked into the infrastructure. I really enjoy tweaking on it.

FFHub's tool is fun, but it seems to rely on a lot of manual adjustments that come out of people's heads rather than (sorry, Blah) a "model". So players that we know will start are given projected points of zero, which happens too frequently and is a bit irritating. Obviously there is some objective basis, but I can't really trust its underlying elements. Similarly, historically, FFScout's projections often miss players out, and they don't update as regularly as I would like.

Talking of frequency of updating, one more good thing about Solio (I don't have shares in them, I promise) is how dynamically projections update (often from direct linkages to the betting markets), in real time. On the last day of last season, while deadline streams were foraging around rumours on X, Solio's projected points for Haaland and other City players took a nose-dive well before all the players who were dropped were confirmed on podcasts. I was happy to rely on the changed projections and update my squad with a fair degree of confidence.

[Hogmeister]

@Blah… Yeah, I am a close follower of various “pundits” (the ones I think talk sense), as have said on there before. So have watched literally dozens of youtube videos this pre-season already. I find hearing the different opinions/strategies helpful to formulate my own thinking.

And the models don’t just take a one week snapshot of course - if you do a Solio Solve, you set how may weeks you want the “ideal squad” to be generated for (limited to 5 on the free tier). And if you plug a team into FF Hub, you can assess how many points it is predicted to get in each week going forward. And when picking players to transfer in, it will show the predicted total points over the next three. And of course on either platform you can look at the player prediction tables for however many weeks you want to analyse.

And the others like FF Scout etc will do the same.

Yes, I think there’s a reason that Andy on LetsTalkFPL is the most successful of the youtube pundits - an easy to listen style, professionally produced, and talks sense most of the time. He’ll mention in passing the rating his team gets on FF Hub, but doesn’t major on that. I do like putting Andy alongside someone like Pras, who is more analytical and heavier on chip strategy assessments etc. If you watch those two, you have got most bases covered. IMO

[zipnolan]

PS - Unfortunately I agree that Bakar - who did very well last year, to be fair - very rarely says anything that I haven't heard elsewhere. The other FPL Wire (I know he's not on it any more) people are far more imaginative. I much prefer to hear their takes, even if I may end up being as template as others.

I would defend Ben Crellin, and there's certainly nothing wrong with admitting that he looks at models (we all do it). In the past he has offered solid advice. I can't help thinking that, when it comes to week 1, Haaland will be safely restored to his squad (unless we learn that he is going to play reduced minutes).

[Hogmeister]

@Zip - yes, exactly that. Only thing I might disagree with, is that I wouldn’t describe the Solio foundation on the betting markets being a gimmick, more of an underlying philosophy. In that the multi-million£ betting markets are usually more ITK than anyone else, so their prediction of FPL points derived from that is not only objective, but more accurate.

And yes, the speed of change of a model is a key factor and anything (like FFH and FFS) that depends on a human inputting their opinion every now and again has that challenge. For example, yes I also was able to react to that Solio flag last season about Haaland being dropped. And I’ll often find on FFH that their predicted popints for a player is (IMO) incorrect as they haven’t yet updated their predicted minutes after a recent change (a related transfer or injury etc).

[blahblah]

At the risk of being accused of being sucked into delusional quicksand, I would also say that Solio is also great in being customisable and configurable. If you don't agree with the relative weightings of fixtures, team strengths or expected minutes of players, for example, you can just change them yourself. Yes, it has the slight gimmick of having an infrastructure based on betting markets, but this year it has Opta data integrated into it, so those external objective measures are baked into the infrastructure. I really enjoy tweaking on it.

I did start using their "tool" ("s because I rapidly descended onto swearing at it) but it can't be far off a Degree Module in learning how to use it

As per the gimmick of those Odds. If I took this anymore seriously than my colour coded fixture sheet until my ADHD gets bored; then I'd happily just use odds that were neutral/objective enough to accurately reflect MU not beating Hull 0-6 with at least a Bruno hat-trick. The ghostly fog from Grimsby 12 months could descend on a rocking (whatever their ground is called) - I doubt it will be as dark and wet, but maybe we'll see the magnetic thing again

[blahblah]

My big gripe with all these "Models" is in preseason and until the season starts to make sense.

We're all going on about the uncertainty this summer with so many new managers, (my fave of) virgin managers and Clubs in Europe etc that knocking out estimates on last season's performances is beyond contempt, imho.

And yes I'm pissed at so many jumping on my 3 Cov atb BB in 2. It was my idea and I'm sulking

My aged gut says that last season is very relevant for promoted clubs playing each other until at least Xmas. Both games were 0-0 last season and Hull are worse than the shit they were last season so there's a good chance of them not scoring. At 12.5m for GK plus 2 they save a few quid until I want rid... As much as I'm into the WC impact on Arse, starting Ewijk is a tad optimistic Or 3 Cov Defs at only 12m could be fun

[Hogmeister]

To be fair, I think you can make a reasonable predictions by analysing player performance last year, when putting it alongside change of formations/managers/new transfers etc, plus (with a heavy pinch of salt) preseason performances.

But yes, a fair point - e.g. is three Arsenal at the back with Gabriel and Raya the best approach because of their clean sheet record last year (some models love this), or should you spend the money elsewhere, especially as Saliba is injured, risk of rotation of Mosquera/Calafiori etc, and the player prices having gone up (especially Gabriel). Who really knows?

I am rather hoping that such uncertainty will mean that the GW1 squads are less template than in the last few seasons… but I seem to say that same thing every year!

Having considered at it at very great length (!), my view on chip strategy is
a) I want to get the BB chip out of the way before I wildcard, and so BB1 or BB2 is what I will do…
b) I would like to try (at least) to avoid WC-ing to WC6 and the international break (albeit with the WC on hand if needed for GW4 or whenever disaster strikes)… and
c) therefore BB1 is better as I think it is easier to build a decent longer-term squad with a BB1 squad than a BB2 squad. And it also gets it out of the way, no messing, no risk of injuries affecting the BB2 squad etc.

With the caveat that when Haaland limps off injured in the Charity/Community/Whatever Shield, the strategy will look completely different!

[jimmy ching]

McBurnie, one nil would make me laugh for GW2

[blahblah]

My Arse line is that their 3 best DMs are just back in training, so the Defence, without Saliba and possibly/probably Timber, is unlikely to be what it was for a few GWs.

But then will AV's attack be any good - and yes I FH in 2 to lump on them vs Brentford and I won't forget that for a fair few years.

And Cov at home should be soft?

So yeah they may well keep a CS it two, but it may well be more about the opponents than Arse.

The hardly mentioned thing this season is the early start of Europe, between 3&4, and the number of virgin managers and Clubs treading that path. The problem being that the prime non-Euro sides, Chelski, Spurs and Skunks, aren't exactly stable etc from last season.

[Smurphy Paw]

A couple of points:
All models involve a degree of manual input. This is often around expected minutes, the biggest variable/determinant of expected score, as I have had it explained to me. Get those wrong and otherwise accuracy of their data falls apart.
Scout numbers have been using betting market data for years. The guy who ‘invented’ the approach, and the guy who now keeps the numbers up to date are both in my Compadres 5AS Team.
The guy who invented the approach would probably have issue with some of the misunderstanding or misinformation above (also some accurate insight).
They work hard to get their individual week predicted numbers as good as they can be. But what you get with Scout, Review and (I think) Hub is something of an average score for a player instead of the range of likely outcomes. That means that a lot of players appear to have the same likely score (the 4.4, 4.6 and so on). These are not a specific prediction, they are something of a mid-point.

Like others I like Solio. Their numbers are pretty good and not much different to other models. But where it stands out is picking paths, or different potential routes. Less important the stacking problem of Player A or Player B. More important to maintain flexibility, be able to respond to unexpected events. For others, maybe for them it is more important to lean into risk (imaginary Player C who also has a 4.5ish predicted midpoint, but is likely to get either a dozen or zero). Like going for Cole Palmer for that run of favourable fixtures last year. I still think benefitting from so many penalties was lucky, but they made their luck. They picked a high upside player for the right run of games.

Content Creators shouldn’t be muddled up with any of the above. Perhaps with the exception of JC and the one or two others who develop models and also create content.

[Neath boy]

Who is JC?

[Dr. Giggles]

Who is JC?

Penny.

[Neath boy]

Even more confused now!

[blahblah]

Who is JC?

Jesus H Christ
Julian Cope
John (Cooper) Clarke
Justin C-something
Jimmy Cousin (kinda Undertones bod)
Jeremy Clarkson (not a Brexiteer, apparently)

[blahblah]

Even more confused now!

Enjoy it, it can be all warm and fuzzy

[blahblah]

Who is JC?

Jesus H Christ
Julian Cope
John (Cooper) Clarke
Justin C-something
Jimmy Cousin (kinda Undertones bod)
Jeremy Clarkson (not a Brexiteer, apparently)

John Cale

[thebillfella]

I'm clearly from the dark ages in FPL terms by virtue of the fact that I still play the game using the eye test under my own steam. I've never used any of these tools as I feel it just takes away from the enjoyment of playing the game in it's purest form, but I'm starting to think that sooner or later I'm going to have to adapt just to stand still as everyone else seems to be using them as standard these days?

Sounds like the consensus suggests that Solio is one of the best tools?

And I refuse to use Nazi X so following anyone on there is out for me!

[bloggie]

Who is JC?

Joan Collins.

image_2026-08-12_220941427.png

[blahblah]

Or Jackie

Joan Crawford?

[Smurphy Paw]

JC = @FPL_Spaceman on X, similar on http://jonnyc.bsky.social/. Co-founder of Solio
Content creator because he does their podcast, but also does all their number crunching with Rob T (who is also very good on X).
Apparently won TFF a couple of years ago as well as one of the top FPL players

[zipnolan]

His actual name is Jonny Currie, and he’s a brilliant multi-format player. He won Telegraph two years ago, and he seems to play fantasy in any sport, any country. He also plays Anti-FPL, which is a version that fascinates me.

[blahblah]

Anti FPL?

[zipnolan]

https://antifpl.pythonanywhere.com/antifpl/rules/

[blahblah]

So the idea is to get fl all points without Blankers?

[zipnolan]

I haven't played it, but that seems about right. You need regular and playing players, but ones that rake up almost no points.

[blahblah]

Sounds like the TFF thang where that bod was top 10 overall after quite a few weeks

[Hogmeister]

Just catching up with this (currently away at a University Reunion weekend, with a lot of other very old people!).

Yes great points there by SP and Zip, totally agree that the degree of sophistication in Solio re choosing paths, risk analysis curves, assessing the robustness of your squad, etc has taken everything up a level. That’s a key strength.

They also make a big play about how much you can/should adjust the underlying base data to reflect your own opinions, and provide a range of graphical tools to allow you to do that:
- Think United defence is overrated by the models? Just slide them to a different position on the chart. And the numbers for all their players will be adjusted in any solves you run.
- Particular players you want always/never to have? Just specify that. etc etc
They encourage you to do this, to try make the predictions reflect your own beliefs, rather than just be spoon-fed by the selections the model is churning out.

It really is a very impressive tool.

And yes Johnny C is clearly the brains behind it, the Solio youtube videos with him and Pras are excellent (if you are into the analytical side, that is!). Hadn’t realised he had such an impressive multi-format track record, very interesting.

I think one of the great attractions of this game is you can play it so many ways - eye test from watching Sky/MOTD at one end, to literally never watching a minute of football and instead working the statistics at the other.

I remember posting here a few years ago on that, about how you can choose (if you wish) to get further and further away from the actual football, as you work your way up through progressive layers of increasingly abstract metadata… go to a match… watch TV.. Opta stats… xG/xGA etc.. heat maps… fixture tickers… prediction models as above… pundits analysing the results of those models… AI pulling together an analysis of the most popular picks by that community of pundits (a long, long way from using the eye test!)

Each of us gets to play the way we prefer…

[Hogmeister]

Scout numbers have been using betting market data for years. The guy who ‘invented’ the approach, and the guy who now keeps the numbers up to date are both in my Compadres 5AS Team.

Apologies if I was wrong about that, I thought that one of the big selling points of Solio, especially last year when it was launched, was that it was based on the underlying betting data (multi-million pound syndicates in the Far East etc). And up until that point, the only place such data was used was in RobT’s (invaluable) weekly clean sheet odds.

Hence I described their approach to being different to FF Scout (and others), who I thought were using manual input by a team of expert players, rather than betting market data.

So it sounds like I got that wrong.

So… is there anything at all different/innovative about Solio’s approach to the betting markets, and how they build their underlying numbers, or is that pretty much the same as FF Scout does?