To see less ads Register or Login ----- Daily Fantasy Sports games 18+

Side-game data project / FPL data collection

A Forum for all Fantasy PremierLeague.com (FPL) Links, Mini-Leagues and Side Games (i.e. inter-FISO competitions run by FISO users)
Post Reply
User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

Mav3rick wrote: 22 Sep 2017, 14:05 For anyone thinking that all this data collection looks really hard (it's not really, you just need a little knowledge) then this article was posted on Reddit.

https://medium.com/towards-data-science ... 2598c4b25e

It's a pretty decent beginners guide, I'm thinking there's a lot of guys in this thread who use Google sheets that might find it useful.
 
my BD is in 1 month, but this looks like the best present I ever got in my 20k days :o

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

:mrgreen:

Happy almost birthday MoSe, with time and the right lookups, you should be able to do anything that my site currently does.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

:) I doubt that :)

In the meanwhile,
https://ratemyteam.uk/Fiso/SideGameData/27051/5
had already stopped loading the Region again, and now stopped loading the Manager Name as well.
:?
That can be a bit annoying:
links can be followed individually, and team IDs can be looked up once you know who the manager is,
but with Team Names prone to changes much more than Managers, this item was useful to try and track teams across seasons by looking up last season collected data (where IDs were of course different)

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

For API reference
Mav3rick wrote: 19 Jul 2016, 18:46 Everything so far looks to be available in a nice JSON format:

Team History:
https://fantasy.premierleague.com/drf/e ... d>/history

League data:
https://fantasy.premierleague.com/drf/l ... &ls-page=1

(example)

All player, club and lookup data:
https://fantasy.premierleague.com/drf/bootstrap-static

Current Squad Selection (when logged in only):
https://fantasy.premierleague.com/drf/transfers

Fixtures:
The fixture data is all contained here (online parsed version). You'll need to cross reference the team ids with the lookup data (which has the teams in it).

Gameweek Data:
The gameweek player data can be found at
https://fantasy.premierleague.com/drf/e ... mber>/live
(this is a set of data for all players in FPL)

Then for an individual FPL team, you can get their gameweek selection from
https://fantasy.premierleague.com/drf/e ... weekNumber>
(and cross check the player ids with the live player data if you want to find out individual gameweek data, such as goals scored).
eagle224 wrote: 03 Sep 2016, 21:37 This has been such a helpful thread for getting my python scripts back up and running this season.

Here are a couple more JSON data URL's that folks may find useful:

All player, Club and Lookup data: https://fantasy.premierleague.com/drf/bootstrap-static
Next Gameweek Information: https://fantasy.premierleague.com/drf/bootstrap-dynamic
GameWeek Stats: https://fantasy.premierleague.com/drf/events/

Fixture History which includes players info for YC,RC,Pen,PenSave etc for finished gameweeks: https://fantasy.premierleague.com/drf/fixtures/
Fixtures Info for a gameweek: https://fantasy.premierleague.com/drf/f ... ?event={GW}

Not too sure what this one is for but seems to be a summarised list of fixtures, regarless of the number used at the end of the URL
https://fantasy.premierleague.com/drf/fixtures/1


All Player Stats to Date: https://fantasy.premierleague.com/drf/elements
Individual Player Hitory: https://fantasy.premierleague.com/drf/e ... /{PlayerID}
eagle224 wrote: 03 Sep 2016, 21:58 Just found this information on GitHub - maybe useful for anyone running leagues via coding
and from nickchild's script topic
RomynPG wrote: 20 Sep 2017, 16:57 Re: Greasemonkey script to display team data on FPL league page
Not quite sure what's what in this thread as I don't use the script myself so don't know what data it needs but wrt to latest api when I load

https://fantasy.premierleague.com/a/team/4288/event/5
which is my GW5 squad the following json files are now loaded

1. https://fantasy.premierleague.com/drf/e ... nt/5/picks
2. https://fantasy.premierleague.com/drf/event/5/live
3. https://fantasy.premierleague.com/drf/entry/4288

1. My picks for GW5
2. All GW5 fixtures and elements (players). (Fixtures has "started" & "finished" to reflect fixture state)
3. General squad info

Looking up data in 2 for the 15 players listed in 1 gives the data needed.

You all probably knew that already - but I'm not clear on what the problem is - apart from the api changing - is data missing?

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

MoSe wrote: 23 Sep 2017, 11:12 https://ratemyteam.uk/Fiso/SideGameData/27051/5
...now stopped loading the Manager Name as well.
:?
If I use the GS API loader featured in the article you linked, with

=ImportJSON("https://fantasy.premierleague.com/drf/l ... ings/27051")
I still find there the Manager Name in col AA "Standings Results Player Name"
Maybe FPL moved and/or renamed it? :?

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

MoSe wrote: 23 Sep 2017, 11:12 https://ratemyteam.uk/Fiso/SideGameData/27051/5
...now stopped loading the Manager Name as well.
:?
and now it's there again! :D
(not the Region yet tho)

User avatar
kidzio
Wideboy
Posts: 53
Joined: 06 Sep 2015, 14:42

Re: Side-game data project / FPL data collection

Post by kidzio »

MoSe wrote: 19 Sep 2017, 09:35 thanks Mav, I had personally experienced only 1 case in previous 4 GWs this season with my grid tool runs,
as I said I took the habit to check the imported data in my xls, so it didn't harm me (andybarrel also posted something similar)
and then we had Libero case in FISO Cup GW3 this season, 1st one Taca ever noticed
will report in case I see it happening again, so let's hope no news good news :)

I'm here to report a different odd glitch tho.
I was about to prepare one of my GW scores scatter graph for the Forum League GW5, from data collected yesterday
so I could easily notice 4 odd spikes in the "R" data (Overall Rank, rightmost column)
I reran https://ratemyteam.uk/Fiso/SideGameData/27051/5 right now, and the same errors are still there

#152 R=393,995 instead of 325,473 in its team https://fantasy.premierleague.com/a/entry/1840/history
#322 R=335,681 instead of 704,449 in its team https://fantasy.premierleague.com/a/ent ... 51/history
#507 R=580,734 instead of 1,095,805 in its team https://fantasy.premierleague.com/a/ent ... 13/history
#921 R=1,768,291 instead of 2,301,095 in its team https://fantasy.premierleague.com/a/entry/698/history

they're sorted correctly, all other data appears to be correct, only their OR is incorrect in the League Tool output :?

not an often used item (e.g. in Divisionals I use it as 3rd tiebreaker, so rarely used that I double-check when needed)
but I thought useful reporting it
it's puzzling to me how a tool can fetch 1160 records correctly, and then in only 4 records 1 item comes up wrong :?
corrupt data and inconsistency by FPL across their several /drf/ resources?
always blame FPL ;)
Hi Mav, this problem was brought up by Mose previously. I noticed the problem right now for GW7.
https://ratemyteam.uk/Fiso/SideGameData/1282/7
For number 29, the "R" is wrong. I noticed the error because the "R" is supposed to be in ascending order. This happened to other leagues too after I did a quick check.
Not sure why it only happens to some of the entries.

rohitpotter
Kevin and Perry
Posts: 20
Joined: 17 Jul 2017, 21:51
FS Record: https://fantasy.premierleague.com/a/entry/698/history

Re: Side-game data project / FPL data collection

Post by rohitpotter »

Mav3rick wrote: 23 Sep 2017, 01:38 :mrgreen:

Happy almost birthday MoSe, with time and the right lookups, you should be able to do anything that my site currently does.
HI Mav,

The Grid seems to be down. Can you please take a look?

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: Side-game data project / FPL data collection

Post by Tacalabala »

Sorry Mav, but GRID doesn't seem to be returning correct output for GW9 overall ranking and total - try 1107394 (Jobmeister)

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

The issue Taca found here was actually in the FPL data itself. When I pull back the per-GW performance data, an overall_rank field is supplied. This is different from the same overall_rank (for the GW) supplied by the fpl team's history page (but only in the case of GW9).

I don't know why that particular week is different, but I've applied a bit of a plaster where I load the FPL team history at the end of the current data extraction process and then compare the overall_rank fields for each GW. If they differ, I assume that the one from history is correct and take it's value.

rohitpotter
Kevin and Perry
Posts: 20
Joined: 17 Jul 2017, 21:51
FS Record: https://fantasy.premierleague.com/a/entry/698/history

Re: Side-game data project / FPL data collection

Post by rohitpotter »

Mav3rick wrote: 29 Oct 2017, 11:57 The issue Taca found here was actually in the FPL data itself. When I pull back the per-GW performance data, an overall_rank field is supplied. This is different from the same overall_rank (for the GW) supplied by the fpl team's history page (but only in the case of GW9).

I don't know why that particular week is different, but I've applied a bit of a plaster where I load the FPL team history at the end of the current data extraction process and then compare the overall_rank fields for each GW. If they differ, I assume that the one from history is correct and take it's value.

Hey Mav,

Grid and League Link seems to be down.Can you please take a look?

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: Side-game data project / FPL data collection

Post by Tacalabala »

You might want to wait until the morning or at least later tonight, FPL is updating at the moment

rohitpotter
Kevin and Perry
Posts: 20
Joined: 17 Jul 2017, 21:51
FS Record: https://fantasy.premierleague.com/a/entry/698/history

Re: Side-game data project / FPL data collection

Post by rohitpotter »

rohitpotter wrote: 04 Nov 2017, 21:00
Mav3rick wrote: 29 Oct 2017, 11:57 The issue Taca found here was actually in the FPL data itself. When I pull back the per-GW performance data, an overall_rank field is supplied. This is different from the same overall_rank (for the GW) supplied by the fpl team's history page (but only in the case of GW9).

I don't know why that particular week is different, but I've applied a bit of a plaster where I load the FPL team history at the end of the current data extraction process and then compare the overall_rank fields for each GW. If they differ, I assume that the one from history is correct and take it's value.

Hey Mav,

Grid and League Link seems to be down.Can you please take a look?
Nvm, it's working now. :)

rohitpotter
Kevin and Perry
Posts: 20
Joined: 17 Jul 2017, 21:51
FS Record: https://fantasy.premierleague.com/a/entry/698/history

Re: Side-game data project / FPL data collection

Post by rohitpotter »

Tacalabala wrote: 04 Nov 2017, 21:03 You might want to wait until the morning or at least later tonight, FPL is updating at the moment
Ya probably was updating then. It's working now :)

User avatar
kidzio
Wideboy
Posts: 53
Joined: 06 Sep 2015, 14:42

Re: Side-game data project / FPL data collection

Post by kidzio »

Hi Mav, I don't know if you have the time but if you do, perhaps you can look into the display of the chip "Freehit" which is currently not showing up in the sidegamedata league page. The other chips are showing up but just not the FreeHit. Thanks a lot.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

now the GRID is returning Error for any single ID you insert
I used it tuesday morning to update Divisionals and it worked regularly

EDIT:
and now it's working again :D

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

MoSe wrote: 23 Sep 2017, 11:46 For API reference....
and from nickchild's script topic....
I wanted to get back on that, as sometime before the new season, all the FPL 17/18 fantasy.premierleague.com/drf/ data won't be available anymore
Dunno whether someone already had the idea to store it

this is not for sidegames purpose of course, but we often see someone inquiring for previous seasons data
if not for 6M FPL Teams, it could be done for 650-ish players

elements lists all FPL players, but only reports their season totals for the scoring items
event/<GW>/live reports all players GW detailed scoring items
FPL 1718 event 38 live.png
it's handy as it's "only" 38 files to store, 1 per GW, did it manually and zippd'em together
FPL 1718 drf event 1-38 live.zip
also parsed GW1 data using excel formulas only (the linked script by Kristian Carter is bugged (repeats data in multiple rows) and I can't debug it)
https://docs.google.com/spreadsheets/d/ ... OUuAhGjrtI
alas, in DGW the event/GW api only has one record for a doubler, merging the items from the two matches


another API where to find detailed scoring item data for each player for each GW i.e. single games, would be
element-summary/<playerID>
the ID can be looked up in the complete Elements API, 394 in the pic below is Kane
FPL 1718 element summary 394 kane.gif
Alas, I'd have to manually change that ID in the URL, and store 650 files one by one. That's a bit beyond even my own crazyness.
I wonder if some kind volunteer with coding ability would write a few simple lines like
   for PlayerID from 1 to 647
   append element-summary/PlayerID to AsingleBigFile
(actually, only the "history" section would do, not the previous 5, but that would require opening each file...?)
this way we would have a single data source "by game by player" which anyone could use by his skills and purpose
for instance (as blah pointed me out) to produce something like Bramernic site for TFF: https://fantasysports.org.uk/tff2017-2018/points.php
You do not have the required permissions to view the files attached to this post.

User avatar
RomynPG
Dumbledore
Posts: 9639
Joined: 13 Oct 2005, 18:32
FS Record: FPL 134 overall 06/07 ... never close since

Re: Side-game data project / FPL data collection

Post by RomynPG »

Darn it - anticipating the new season wouldn't kick off until after the World Cup I'd pegged some time over this weekend to grab all the data MoSe mentioned above along with detailed squad data for all known FISOers.

:roll:

User avatar
nickchild
FISOhead
Posts: 648
Joined: 27 Jul 2006, 23:51
FS Record: 180th Overall FPL and X-League Winner 2008/09
Contact:

Re: Side-game data project / FPL data collection

Post by nickchild »

Does anyone know the new API URL for team data? https://fantasy.premierleague.com/drf/e ... 98/event/1 doesn't work anymore... I will continue to dig around and see what I can find

User avatar
kidzio
Wideboy
Posts: 53
Joined: 06 Sep 2015, 14:42

Re: Side-game data project / FPL data collection

Post by kidzio »

nickchild wrote: 15 Aug 2018, 09:29 Does anyone know the new API URL for team data? https://fantasy.premierleague.com/drf/e ... 98/event/1 doesn't work anymore... I will continue to dig around and see what I can find
Mav3rick might be able to help. His sidegame data page is still working.

thesilkworm
Grumpy Old Man
Posts: 1554
Joined: 01 Oct 2009, 17:59

Re: Side-game data project / FPL data collection

Post by thesilkworm »

nickchild wrote: 15 Aug 2018, 09:29 Does anyone know the new API URL for team data? https://fantasy.premierleague.com/drf/e ... 98/event/1 doesn't work anymore... I will continue to dig around and see what I can find
Are either of these what you're looking for?

https://fantasy.premierleague.com/drf/entry/440198
https://fantasy.premierleague.com/drf/e ... nt/1/picks

First one is general info about team and second one is GW1 picks.

User avatar
nickchild
FISOhead
Posts: 648
Joined: 27 Jul 2006, 23:51
FS Record: 180th Overall FPL and X-League Winner 2008/09
Contact:

Re: Side-game data project / FPL data collection

Post by nickchild »

Perfect! Just what I was after. Was missing the /picks part. Thanks

bspittles
Dumbledore
Posts: 7607
Joined: 13 Oct 2005, 18:26
Location: In hiding

Re: Side-game data project / FPL data collection

Post by bspittles »

Looking at the JSON from https://fantasy.premierleague.com/drf/bootstrap-static, does anyone know which element indicates whether a player has left the EPL? Is there such an indicator?

I thought is was STATUS but that's just an indication whether a player can play the next game or not.

Thanks in advance.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

umm..
https://fantasy.premierleague.com/drf/elements/

you'd have to check the "news" field?
I understand that's not a toggle or a fixed-values variable, but a text field to parse...
DRF Morata Status.gif
You do not have the required permissions to view the files attached to this post.

bspittles
Dumbledore
Posts: 7607
Joined: 13 Oct 2005, 18:26
Location: In hiding

Re: Side-game data project / FPL data collection

Post by bspittles »

Thanks MoSe. I eventually found it (sorry for not updating the thread). The STATUS element can be a for available, u for unavailable, i for injured or s for suspended.

Unavailable indicates that they aren't playing in the EPL at present.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

OK fine then,
I honestly thought that 'unavailable' might have indicated other circumstances too, when not injured or banned.

Say,
uneligible because loan agreements with oppo,
family leave (it escapes me atm the correct english term when a player has mourning, or when becomes father...),
on NT duty for players whose Continental Championships clash with PL matches... (e.g. Son this season)

User avatar
Talkie Toaster
Grumpy Old Man
Posts: 4427
Joined: 11 Jul 2017, 15:43

Re: Side-game data project / FPL data collection

Post by Talkie Toaster »

It looks like it can also be a 'n' when a player is not available because of a clash with a parent club (see Puncheon for GW32 against Palace) or a 'd' for doubtful.

User avatar
kidzio
Wideboy
Posts: 53
Joined: 06 Sep 2015, 14:42

Re: Side-game data project / FPL data collection

Post by kidzio »

Hi, I noticed the website to grab the fpl data is no longer available. Would the grid by Mav still be available? Or the league one?


Sent from my iPhone using Tapatalk

User avatar
eagle224
Grumpy Old Man
Posts: 2427
Joined: 10 Nov 2005, 22:14
Location: England
FS Record: Veteran

Re: Side-game data project / FPL data collection

Post by eagle224 »

kidzio wrote: 28 Jun 2019, 19:23 Hi, I noticed the website to grab the fpl data is no longer available. Would the grid by Mav still be available? Or the league one?


Sent from my iPhone using Tapatalk
The main API web addresses have changed this season. It probably won't take long for punters to work it all out.

Last season: https://fantasy.premierleague.com/drf/bootstrap-static/
This season: https://fantasy.premierleague.com/api/bootstrap-static/

The website that grabs the data will need updating and will probably start working again once everything has been figured out.

User avatar
Talkie Toaster
Grumpy Old Man
Posts: 4427
Joined: 11 Jul 2017, 15:43

Re: Side-game data project / FPL data collection

Post by Talkie Toaster »

Yeah, It's mostly just the /drf/ has changed to /api/ but there are some other changes as well, such as

fantasy.premierleague.com/drf/leagues-classic-standings/nnnn

is now

fantasy.premierleague.com/api/leagues-classic/nnnn/standings

There could be changes to the JSON data structures as well which may not be identifiable until after the game has started.

The bootstrap-static data has some changes to its structure for starters, although from a quick look it's mostly just stuff that wasn't being used being removed.

View Latest: 1 Day View Your posts
Post Reply

Return to “FPL Links, Mini-Leagues & Side Games”