To see less ads Register or Login ----- Daily Fantasy Sports games 18+

Side-game data project / FPL data collection

A Forum for all Fantasy PremierLeague.com (FPL) Links, Mini-Leagues and Side Games (i.e. inter-FISO competitions run by FISO users)
Post Reply
User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Side-game data project / FPL data collection

Post by Mav3rick »

Tacalabala (in the 'When does the game start' thread) wrote:This has got me worried, as if they have massively changed the way the data is accessed/displayed, it will make sidegames near impossible to run. Hopefully I'm just worrying for nothing!
Me too, although it depends which data you're looking at. As far as I can tell, it's still ISM running FPL, so that suggests to me that the JSON data would still be used (although obviously the form might change), especially so if it's being re-designed for better mobile capabilities.

It's going to wreck my RMT site if the JSON for an individual FPL team can't be accessed anymore, and if the individual player JSON files are lost then I'd expect many auto imports will fail, possibly affecting the price change sites too.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: When does the new FPL season start?

Post by MoSe »

Mav3rick wrote:
Tacalabala wrote:This has got me worried, as if they have massively changed the way the data is accessed/displayed, it will make sidegames near impossible to run. Hopefully I'm just worrying for nothing!
Me too, although it depends which data you're looking at. As far as I can tell, it's still ISM running FPL, so that suggests to me that the JSON data would still be used (although obviously the form might change), especially so if it's being re-designed for better mobile capabilities.

It's going to wreck my RMT site if the JSON for an individual FPL team can't be accessed anymore, and if the individual player JSON files are lost then I'd expect many auto imports will fail, possibly affecting the price change sites too.
I'll just need the Gameweek History table to be referrable as a table and imported wholly as a web query into excel
Can't do JS and didn't need it, the very basic level of xls import was enough
Failing that, I'd have to relinquish the Divisionals management, or hire some dev to provide me with the weekly score, hits and transfers data for each team involved (129 last season)

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: When does the new FPL season start?

Post by Tacalabala »

Mav3rick wrote:
Tacalabala wrote:This has got me worried, as if they have massively changed the way the data is accessed/displayed, it will make sidegames near impossible to run. Hopefully I'm just worrying for nothing!
Me too, although it depends which data you're looking at. As far as I can tell, it's still ISM running FPL, so that suggests to me that the JSON data would still be used (although obviously the form might change), especially so if it's being re-designed for better mobile capabilities.

It's going to wreck my RMT site if the JSON for an individual FPL team can't be accessed anymore, and if the individual player JSON files are lost then I'd expect many auto imports will fail, possibly affecting the price change sites too.
Yeah, it's the way that the Euro one was designed, deliberately it seems to stop data capture. We'll see, for all we know they might make it easier!

User avatar
RomynPG
Dumbledore
Posts: 9639
Joined: 13 Oct 2005, 18:32
FS Record: FPL 134 overall 06/07 ... never close since

Re: When does the new FPL season start?

Post by RomynPG »

As I've probably been data collecting in various forms from FPL for 10-ish years now I probably use the most primitive method - I parse the html in VBA :o

I used to do lots - players, fixtures, squads etc - enough to fill in some holes when D+ took up the Crack-the-Code stuff a while ago and support a whole load of games and nerdy things. Now I only do varoius league pages for the 5AS.

Regardless of whether anything changes this year maybe it's time to pool resources and put something together that collects all the data we all use and make it available via an API of some sort.

Making that side of things easier may encourage more side-games - I know I'd like to bring back a game that needs full squad info that I don't collect any more.

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: When does the new FPL season start?

Post by Tacalabala »

RomynPG wrote:As I've probably been data collecting in various forms from FPL for 10-ish years now I probably use the most primitive method - I parse the html in VBA :o

I used to do lots - players, fixtures, squads etc - enough to fill in some holes when D+ took up the Crack-the-Code stuff a while ago and support a whole load of games and nerdy things. Now I only do varoius league pages for the 5AS.

Regardless of whether anything changes this year maybe it's time to pool resources and put something together that collects all the data we all use and make it available via an API of some sort.

Making that side of things easier may encourage more side-games - I know I'd like to bring back a game that needs full squad info that I don't collect any more.
Agreed, though I'll hold my hands up and admit I can't code for toffees haha

User avatar
Mystery
FISO Knight
Posts: 13816
Joined: 13 Oct 2005, 18:33
Location: Just about here
FS Record: 116th in TFF 05/06
Contact:

Re: When does the new FPL season start?

Post by Mystery »

Happy to help if I can although pretty much all the work I do is in Google Sheets.

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: When does the new FPL season start?

Post by Mav3rick »

RomynPG wrote: Regardless of whether anything changes this year maybe it's time to pool resources and put something together that collects all the data we all use and make it available via an API of some sort.

Making that side of things easier may encourage more side-games - I know I'd like to bring back a game that needs full squad info that I don't collect any more.
If it's something people want then I'm happy to help. Based on last year's structures, my RMT site was pulling back (for any given FPL team id and GW number) things like total score, total goals scored, total yellow cards, total red cards, penalties missed, total BPs etc. If that still broadly works post-redesign, then I'd be happy to make it available to everyone via a simple API hosted on my web server.

Assuming that the additional data requirements aren't too numerous, I could add in extra bits of data as needed by us here if we can agree what we need.

Probably needs a new thread for further discussion (mods - can we get a thread split please?), and we may need to wait until GW1 is underway before we can be sure that all the data needed is available, but I'll do what I can if there's popular demand.

User avatar
RomynPG
Dumbledore
Posts: 9639
Joined: 13 Oct 2005, 18:32
FS Record: FPL 134 overall 06/07 ... never close since

Re: When does the new FPL season start?

Post by RomynPG »

Mav3rick wrote:
RomynPG wrote: Regardless of whether anything changes this year maybe it's time to pool resources and put something together that collects all the data we all use and make it available via an API of some sort.

Making that side of things easier may encourage more side-games - I know I'd like to bring back a game that needs full squad info that I don't collect any more.
If it's something people want then I'm happy to help. Based on last year's structures, my RMT site was pulling back (for any given FPL team id and GW number) things like total score, total goals scored, total yellow cards, total red cards, penalties missed, total BPs etc. If that still broadly works post-redesign, then I'd be happy to make it available to everyone via a simple API hosted on my web server.

Assuming that the additional data requirements aren't too numerous, I could add in extra bits of data as needed by us here if we can agree what we need.

Probably needs a new thread for further discussion (mods - can we get a thread split please?), and we may need to wait until GW1 is underway before we can be sure that all the data needed is available, but I'll do what I can if there's popular demand.
Sounds like there's some interest - nice one.

Good plan for a thread split - maybe call it "Sidegame data project"? ... leave it here for a bit - to hopefully be seen by general data collectors - and then move to sidegames later?... or whatever.

I have some thoughts I'll put together.

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

I'll quickly describe what I already have (well, had last season):

Gameweek Details for a given team
Input: FPL Team ID, Gameweek Number
Output: A gameweek summary for the starting XI (or XV in case of BB) that contains the total GW score and the total number of GoalsScored, GoalsConceded, MinutesPlayed, Saves, YellowCards, Cleansheets, Assists, BonusPoints, OwnGoals, PenaltiesSaved, PenaltiesMissed, RedCards for all the starting players, including auto-subs. I don't store any data, it's just parsing data on the FPL site and returning it.

I'll need to try to get that working again for my own site once the new FPL site launches, if I manage to do that then I could easily expose the data via a WebApi. Any interested clients would need to be able to parse some web friendly content types, such as JSON or XML so it wouldn't be suitable for everyone.

I think sidegames care most about GW data and events, in many cases comparing the events of one team to events of another. Unfortunately, GW data is the least programatically accessible data on the old FPL site at least. I don't know how much a more dedicated API would help side game maintainers if they run their games through Excel currently using table imports (I really don't know much at all about Excels abilities here). I was able to auto run automated (simple) sidegames on my site via the single method above, but that obviously involved other code auto-processing the game weeks as they passed:

Twenty20
Heptathlon
World Cup

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: Side-game data project / FPL data collection

Post by Tacalabala »

FISO DATA 2015-16.zip
Ideally, we'd have something like this online for all registered FISOers. Obviously this is all dependent on .iqy queries.
You do not have the required permissions to view the files attached to this post.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

I'm positive now that I can NOT manage my sidegames on my own anymore
(unless I individually check and import each team scores :o )

the problem lies in the .iqy queries and in the browser version on the PC I'm using

as far as I can tell, you can't pick the browser used by the .iqy, excel it's an MS Office program, and it uses MS IExplorer,
couldn't find any way to customise it to Chrome, regardless I made Chrome the predefined browser on my PC

Bigger problem, I only have the work PC, don't have one at home.
And on the work PC I have to do with the company certified SW
And we're stuck to IE 8, can't upgrade it (it's indeed to circumvent that that I installed Chrome on my own)

Don't know wheter updating to the latest IE version would solve it,
but IE8 it's not able to correctly display the green banner and its buttons, so I can't navigate to the team page and the GW history page
it lets me login, but then even pasting the /a/team/my or the /a/event/xxx/history link into the url line, it always redirects me to the /a/#home page
(after an avalanche of pulselive and ims script errors)

That's in IE alone, leaving alone setting up a web query in excel, where the available MS browser version problems of course propagate.
Mind, I have no problem with explorer from the Android, but there in the android excel I don't have iqy to manage data imports...
___________

As I said, I only need a very limited data set from each player to run the FISO Divisionals (GP, TC, and TM)
Tacalabala's sheet would be overkill ....if his IE version allows him to keep it updated I could piggyback it :oops:
of course it would be ..."humiliating" ;) to depend on someone else's work to run and manage a sidegame

an auto-updating website as Mav3rick proposed would be optimal as it would only require the initial setup, and no maintenance,
but it would of course be useful only if it can present data from ALL the teams in a sidegame (on in FISO) in one view,
otherwise it would be the same as checking them manually and individually on the FPL site

another option, if Mav3 can't present a data table for a given League, or List of teams,
it could also work as he suggested as a data-fetch page for a single team, as long as the "Input: FPL Team ID, Gameweek Number" can be embedded in the URL, and not having to insert it manually in a field in the page
that way I'd set up and customise the individual web queries to query Mav's site for each single player, as I did so far with FPL

it's either that, or forsaking running the sidegame (hoping someone else takes on), or checking data manually every GW for each team (also error-prone)...
...or buying a PC with Win10 & IE11 (provided someon confirms FPL correctyl runs there) for the sole purpose of runinng an FPL sidegame :o :lol:

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

btw, i've checked Mav's RMt site, excellent work, how could've I missed it in two seasons here?

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

MoSe wrote:Bigger problem, I only have the work PC, don't have one at home.
And on the work PC I have to do with the company certified SW
And we're stuck to IE 8, can't upgrade it (it's indeed to circumvent that that I installed Chrome on my own)
btw, dropped a hint to my boss, and he said IE11 has been tested on some pilot users
it worked fine, except for some company applications (don't know whether the ones out unit is using)
I doubt it could get "certified" before september, nothing happens in august in italy :(

User avatar
Aldershot Rejects
Dumbledore
Posts: 9585
Joined: 03 Aug 2011, 17:15
Location: Kent
FS Record: 5th Metro (2010-11); 146 - Sky (2015-16); 218 - Sky (2014-15); 386 - Sky (2020-21); 636 - FPL (2017-18); last 16 Sky Cup (2018-19)

Re: Side-game data project / FPL data collection

Post by Aldershot Rejects »

As a non-IT person, thus thread is complete gobbledygook, but it just makes me more thankful for the work you guys do on sidegames .

Thank you and good luck in whatever you decide :)

User avatar
jpk
Dumbledore
Posts: 5216
Joined: 13 Oct 2005, 18:42
Contact:

Re: Side-game data project / FPL data collection

Post by jpk »

just my 2pence worth .... rather than use webqueries from excel - have you tried VBA and XMLHTTP e.g.

Code: Select all

Dim xmlhttp As Object
Dim response As String
Set xmlhttp = CreateObject("Microsoft.XMLHTTP")
xmlhttp.Open "GET", "https://www.someadrress.com/" + Trim(Str(some variable)), False
xmlhttp.setRequestHeader "Connection", "close"
xmlhttp.send
If xmlhttp.readyState = 4 Then
       response = xmlhttp.responseText
End If 

then all the html is in the response variable and you can parse that in code.

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

IF ONLY had I ever learned VBA.....

RomynPG does the VBA html parsing, but as Taca had put it "I can't code for toffees haha" :oops:
  • last time I coded something it was in C++ over than 20 years ago
    I know VBA shouldn't be rocket science, but I lost too many neurons in the last 2 decades ;)
webqueries from excel were a handy and readymade built-in tool to use for us non-coders

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

MoSe wrote:webqueries from excel were a handy and readymade built-in tool to use for us non-coders
In general the site redesign has worked out better than expected for me as they have simplified all the data access that I have so far been able to see (an actual gameweek score page will be the big test, but I can't see that until GW1 is in progress).

Everything so far looks to be available in a nice JSON format:

Team History:
https://fantasy.premierleague.com/drf/e ... d>/history

League data:
https://fantasy.premierleague.com/drf/l ... &ls-page=1

(example)

All player, club and lookup data:
https://fantasy.premierleague.com/drf/bootstrap-static

Current Squad Selection (when logged in only):
https://fantasy.premierleague.com/drf/transfers

Fixtures:
The fixture data is all contained here (online parsed version). You'll need to cross reference the team ids with the lookup data (which has the teams in it).

Gameweek Data:
The gameweek player data can be found at
https://fantasy.premierleague.com/drf/e ... mber>/live
(this is a set of data for all players in FPL)

Then for an individual FPL team, you can get their gameweek selection from
https://fantasy.premierleague.com/drf/e ... weekNumber>
(and cross check the player ids with the live player data if you want to find out individual gameweek data, such as goals scored).

Even the login process is a (tiny) bit simpler.

The way things are working is a bit like a single page app, where the HTML loads and then triggers a request to get the JSON from the URLs above, and the page is then populated by javascript. I don't know how your web queries work in terms of the rendering engine, but if the javascript isn't executed then the Excel "load from this url" feature might stop working, as at least some of the data is being loaded dynamically and isn't rendered in the HTML.

Again, we'll need to wait until GW1 is underway before we can see the full league data structures, but if your Excel can parse the data returned from the "League data" url above then you should be laughing, especially as it should change less than the HTML in future seasons.
Last edited by Mav3rick on 15 Aug 2016, 17:32, edited 5 times in total.

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

MoSe wrote: ... it would of course be useful only if it can present data from ALL the teams in a sidegame (on in FISO) in one view, otherwise it would be the same as checking them manually and individually on the FPL site
I can't say for definite until we see the GW1 in progress pages (and associated data) but I think it will be very possible for me to take an FPL league id and GW number, and then to do all the parsing and cross checking to provide the GW event data I mentioned above for all participants in a league.

That's assuming everything on the GW page in the new FPL site works like the others, but I think you'd be able to request a URL that includes the LeagueId and GW number and get back a list of JSON or XML data.

Input: https://somesite/<FplLeagueId>/<GameweekNumber>
Output: A list of FPL team ids (and team/manager name) with GoalsScored, GoalsConceded, MinutesPlayed, Saves, YellowCards, Cleansheets, Assists, BonusPoints, OwnGoals, PenaltiesSaved, PenaltiesMissed, RedCards for all the starting players, including auto-subs for each of those team ids.

You'll be able to get something similar with maybe less finesse from the FPL json links above.

If you need CSVs or HTML or similar then there's probably some sort of tool/content type provider we can use to do a conversion but I'd struggle to replicate (and test/maintain) something as complex as the multi-sheet excel list currently in use.

User avatar
SuperGrover
Grumpy Old Man
Posts: 1540
Joined: 12 Sep 2012, 15:38
Location: Chicago, USA

Re: When does the new FPL season start?

Post by SuperGrover »

Mav3rick wrote:
RomynPG wrote: Regardless of whether anything changes this year maybe it's time to pool resources and put something together that collects all the data we all use and make it available via an API of some sort.

Making that side of things easier may encourage more side-games - I know I'd like to bring back a game that needs full squad info that I don't collect any more.
If it's something people want then I'm happy to help. Based on last year's structures, my RMT site was pulling back (for any given FPL team id and GW number) things like total score, total goals scored, total yellow cards, total red cards, penalties missed, total BPs etc. If that still broadly works post-redesign, then I'd be happy to make it available to everyone via a simple API hosted on my web server.

Assuming that the additional data requirements aren't too numerous, I could add in extra bits of data as needed by us here if we can agree what we need.
I have something similar written in Java (last language I used as a developer, so easier for me than learning PHP or something more appropriate). I could aslo add additional fields if needed or at least help bounce ideas around for a design of any solution that does so.

I am a bit late to the thread though; do we know for certain if the JSON feed has been eliminated?

EDIT: I see the player JSON data is still readily available at https://fantasy.premierleague.com/drf/elements/. Looks to me like the format has changed but is now simpler (used to have a map of value in English to a code, which then was later utilized as the key in the key/value pair). I don't see any issue making this work again.

EDIT X2: Missed previous post by Mav3rick which explains all the available data in the new site. Looks great to me.

User avatar
SuperGrover
Grumpy Old Man
Posts: 1540
Joined: 12 Sep 2012, 15:38
Location: Chicago, USA

Re: Side-game data project / FPL data collection

Post by SuperGrover »

Mav3rick wrote:If you need CSVs or HTML or similar then there's probably some sort of tool/content type provider we can use to do a conversion but I'd struggle to replicate (and test/maintain) something as complex as the multi-sheet excel list currently in use.
If Excel is absolutely required I can assist (although probably not for a couple months due to a new home purchase). I already spit out multi-tabbed, well formatted Excels using the Apache POI - Java API for Microsoft Documents (https://poi.apache.org/index.html). I am able to produce multi-formats (numbers rounded to decimals, bolding, italics, ec.), can adjust cell formatting (horizontal/vertical alignment, cell joins/splits), and even column/row groupings. I should be able to produce the aforementioned spreadsheet from an input file (JSON preferably) without too much issue (although not sure how well the solution would scale to excessively large Excel documents).

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

Sounds like we have options then. I missed that "elements" file SG, didn't see it being used by the site, but it looks like another useful data source.

They've really made is easier for us!

I like the idea of your excel outputs, we definitely have the ability here to present the data now, we just need to see if the side game runners can use a single format, and if their existing import routines are still working.

At the very least, I feel confident we can help MoSE.
Last edited by Mav3rick on 20 Jul 2016, 17:47, edited 1 time in total.

User avatar
DrBunker
FISO Knight
Posts: 19050
Joined: 02 Jan 2007, 11:13
Location: Proud to be owned by DMWSL613 Limited
FS Record: FISODAS season 21 Premiership champion

Re: Side-game data project / FPL data collection

Post by DrBunker »

Mav3rick wrote:They've really made is easier for us!
The idea that they'd deliberately make it harder for game developers always seemed confusing to me. I'm glad to hear it sounds like our sidegames shouldn't be affected negatively.

thesilkworm
Grumpy Old Man
Posts: 1554
Joined: 01 Oct 2009, 17:59

Re: Side-game data project / FPL data collection

Post by thesilkworm »

I'm just curious, what do the people who are contributing to this thread do for a living? Do you use these skills in your work? Obviously feel free to ignore this question :)

User avatar
Mav3rick
FISO Jedi Knight
Posts: 20858
Joined: 20 Jul 2009, 20:35
FS Record: FPL: 1082, 1201, 1800, 10203

The stats are dark and full of errors.

Re: Side-game data project / FPL data collection

Post by Mav3rick »

thesilkworm wrote:I'm just curious, what do the people who are contributing to this thread do for a living? Do you use these skills in your work? Obviously feel free to ignore this question :)
Yep, it's my day job (which rather irritatingly eats into my FPL time).
MoSe wrote:webqueries from excel were a handy and readymade built-in tool to use for us non-coders
Ok MoSe, I had a couple of hours spare after lunch, so I put this together as a starting point, the URL contains the FPL League Id and the gameweek number, I get all the participants in the league, get their GW scores and return everything in a nice simple table:

Here's an example using the FISO league.

The gameweek data is randomised for now obviously, I'll need to return real data for the listed fields for my own site, but I could add other columns with relative ease I think. It's all static and simple HTML so Excel should be fine, if anybody wanted the actual JSON then I can expose that too. The data (which is all public info) is cached for 10 minutes but not persisted anywhere other than the FPL website.

Format of the URL is simply
https://ratemyteam.uk/Fiso/SideGameData ... /<GWNumber>

Is that useful to side game admins?

User avatar
MoSe
Dumbledore
Posts: 9550
Joined: 10 Sep 2014, 12:25
Location: next door S.Siro stadium
FS Record: FISODAS CUP Winner Season 25
FISO H2H Winner: 15/16 Div2 - 16/17 Div1
FISO Mirror: 16/17 PL Winner

Re: Side-game data project / FPL data collection

Post by MoSe »

Mav3rick wrote:Ok MoSe, I had a couple of hours spare after lunch, so I put this together as a starting point,
....
It's all static and simple HTML so Excel should be fine,
I'm moved that you took the effort! :D
I found out that excel can import from the web in HTML or XML, but not from the /drf csv text for instance

In can thus confirm now that I could easily import the whole Pending Entrants table values neatly arranged in their rows and columns
I was even puzzled at first that I imported different values from the page I had displayed in my browser, then I realised that the query imported the latest randomised values!

now I'm beginning to realise tho the actual and handier requirements specific to the Divisionals sidegame
for instance, could I find all the needed data in a single table for all the sidegame teams, I could even just Copy it from the web page and Paste it into Excel, forsaking the need to set up and albeit simple query to be updated :mrgreen:

or, as modifying the url in a webquery amounts as redefining it,
in order to link it instead to a variable value I'd just insert in a cell, I'd have to make it parametric,
but as I got it, it would require the gw in the url to be in the format ?GW=1 instead of /1
https://ratemyteam.uk/Fiso/SideGameData/31821?GW=1 a feature I'm not familiar with (yet ;) )

As the reason for all this is basically my being stuck to an obsolete IE version, quite a personal limitation,
I'd be happy to present you via PM my actual requirements to properly run my sidegame, should you allow me to continue abusing of your patience :oops:
maybe the next week?

Of course the project and endeavour to collect and provide a comprehensive and more accessible data source and collection for ANY sidegame runner, goes beyond my single personal case and predicament
Maybe Taca and RomynPG could better guide and discuss such wider project

User avatar
RomynPG
Dumbledore
Posts: 9639
Joined: 13 Oct 2005, 18:32
FS Record: FPL 134 overall 06/07 ... never close since

Re: Side-game data project / FPL data collection

Post by RomynPG »

That's really useful Mav - thx.

What's especially useful is the fact that all the data is so easily available in JSON.

Not sure about login etc - do you need to be logged in to FPL to access from a external server for example?

My main aim, and something that's been building for a few years now, is that I want to move everything I do wrt sidegames onto the web - in particular I want to do that with the 5AS H2H. At present, and for historical reasons, my sidegames are run on an 11 year old PC running XP and Office 2000 - grabbing raw html and parsing it and storing it in a main lookup table. On top of this fairly generic import are all the extra worksheets that, after initial season config, allow the H2H to be run with a couple of clicks a week. It was done this way so Krispy could bring his awesome game to life by allowing the admin of what is an extremely complex game relatively straightforward.

That method is well past it's sell-by date and I want to get in a position where I can pass-on the H2H admin when I finally get bored of it. I also want to bring back another game that needs a level of data I don't now collect - full player stats - and I'm not redoing it Access again.

I won't be able to get this in place for this season - at least as far as the H2H goes - so I'm looking at getting something together as the season progresses. My first thoughts are to collect and store whatever data is available - fixtures/results, player stats (some frequency), squad stats and anything else that could feasibly be used for a sidegame. (

(It's also more practical that way, permanent storage, when you want to add additional data to fixtures or players to use. Example - adding referee to fixtures and birthday to players/refs allowed for Pointless Sweepstake - The Birthday Problem - which was fun :D )

I'd also like to move my fixture/results grid online - allowing me to add user defined team rankings etc.

I currently work with LAMP and JSON/AJAX stuff so am comfortable with what I'd need to do there. The data already in JSON makes the whole thing pretty easy - I hope.

... and if I was doing it for myself ...then why not just open that data up a bit - in the same manner as your examples - for output via JSON or just plain csv (as alot of data would be tabular).

That's how I got here - looking forward to where I'm headed.

User avatar
DrBunker
FISO Knight
Posts: 19050
Joined: 02 Jan 2007, 11:13
Location: Proud to be owned by DMWSL613 Limited
FS Record: FISODAS season 21 Premiership champion

Re: Side-game data project / FPL data collection

Post by DrBunker »

RomynPG wrote:I want to get in a position where I can pass-on the H2H admin when I finally get bored of it.
Let's hope this project works then as there aren't many scarier sentences on FISO than that!

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: Side-game data project / FPL data collection

Post by Tacalabala »

I can see what has changed - you can't click on squad names to access player history, at least not at this stage anyway. This is a massive pain in the arse for me as now I'm essentially relying on people to provide me with the URLS themselves.

User avatar
Tacalabala
FISO Knight
Posts: 19010
Joined: 07 Sep 2008, 01:03

Re: Side-game data project / FPL data collection

Post by Tacalabala »

To: support@mail.fantasy.premierleague.com
Subject: Other player's history pages
Date: Wed, 20 Jul 2016 22:13:03 +0100

Good evening,

Firstly, may I take the opportunity to congratulate you on a visually stunning update of the game.

Please can you resolve a problem that I appear to be having with the new set-up? I have previously been able to access other player's history pages by clicking on squad/usernames on league pages, but this is not possible at this time. Is this only a temporary restriction which will be reinstated in the coming days?

Kind regards
Will let you know what the response is - they really have to bring this back in, crazy to stop this.

User avatar
Stemania
FISO Jedi Knight
Posts: 20448
Joined: 27 Aug 2006, 11:54
Location: On the Iron Throne of xG, the seat of The Crown Prince of the Stat Perverts. Or if not, in the STC!
FS Record: Best: TFF 321st. FPL 129th. FFS Career HoF Peak 2nd (Live 1st). Ability since lost.

Re: Side-game data project / FPL data collection

Post by Stemania »

Are you sure it isn't always like this before the first update since it isn't practical to see what team anyone has picked yet? The list of teams under each league aren't strictly speaking actually in he league yet, just to be added after the first update I think. :?

View Latest: 1 Day View Your posts
Post Reply

Return to “FPL Links, Mini-Leagues & Side Games”