Issue
I’m currently working on a project where we estimate the expected daily fantasy points for each NBA player and then build an optimized lineup which can plugged into various daily fantasy basketball tournaments.
In order to report metrics on our results for an entire season, we are trying to get historic fantasy tournament results (how many points did it take get first place, how many points did it take to win the minimum reward money, etc.). I’m trying to pull data from this table Fantasy Cruncher's website: https://www.fantasycruncher.com/contest-links/NBA/2021-03-10. However, the table is not included in the html source code. Anyone have advice on how I could scrape data from that website?
We are using Python 3 for this project, and so far have tried BeautifulSoup, Pandas, and couple other libraries that were mentioned in posts I found online.
Note: I'm open to suggestions on alternative websites to find this data, however it's not something that's widely available. Even Fanduel's website has a similar set up where the data you see on the page does not show up in the html source code.
Solution
You can use this example how to load the data from the page:
import requests
import pandas as pd
url = (
"https://www.fantasycruncher.com/funcs/tournament-analyzer/get-contests.php"
)
data = {
"sites[]": [
"draftkings",
"draftkings_pickem",
"draftkings_showdown",
"fanduel",
"fanduel_single",
"fanduel_super",
"fantasydraft",
"yahoo",
"superdraft",
],
"leagues[]": "NBA",
"periods[]": "2021-03-10",
}
data = requests.post(url, data=data).json()
df = pd.json_normalize(data)
print(df)
df.to_csv("data.csv", index=False)
Prints:
id site league slate site_id name period max_entries max_entrants cost prizepool places_paid total_entrants winning_score mincash_score startdate winning_payout mincash_payout DateTime Title game_cnt winner_cnt winner has_lineups
0 675539213 fanduel NBA 55339 55339-244619717 $25K Tue NBA Piggy Bank Shot (150 Entries Max) 2021-03-10 150 598802 0.05 25000 180000 392082 366.20 279.50 1615424400 1000.00 0.10 2021-03-10 20:00:00 Main 2 0 hornethurricane25 1
1 675539237 fanduel NBA 55339 55339-244619991 $1M Tue NBA Comeback Shot ($200K to 1st) 2021-03-10 150 297619 4.00 1000000 78971 250535 366.20 284.80 1615424400 200000.00 7.00 2021-03-10 20:00:00 Main 2 0 calieternal85 1
2 675539211 fanduel NBA 55339 55339-244619739 $25K Tue NBA Fadeaway ($0.25 to Enter, 20 Entr... 2021-03-10 20 119760 0.25 25000 35937 80362 359.00 275.10 1615424400 500.00 0.50 2021-03-10 20:00:00 Main 2 0 ketchdenny,bbgj278990 1
3 675532537 draftkings NBA 46045 104452190 NBA $1M Return From Break [$250K to 1st] 2021-03-10 150 117647 10.00 1000000 24790 98331 330.50 261.75 1615424400 250000.00 15.00 2021-03-10 20:00:00 Late Swap 2 2 blankco216,markvikings 1
4 678166877 fanduel_single NBA 55354 55354-244682782 $3K Wed NBA Piggy Bank Shot (150 Entries Max) 2021-03-10 150 71856 0.05 3000 21550 37734 285.76 231.50 1615426200 100.00 0.10 2021-03-10 20:30:00 SA @ DAL 1 0 bearcats594,cloverbandit,ceegee,richdeese,drs1... 0
5 675532493 draftkings NBA 46045 104452184 NBA $50K And-One [20 Entry Max] 2021-03-10 20 59453 1.00 50000 15950 59453 330.75 259.75 1615424400 2500.00 1.50 2021-03-10 20:00:00 Late Swap 2 1 Bradsh86 1
6 678166786 fanduel_single NBA 55353 55353-244682520 $2K Wed NBA Piggy Bank Shot (150 Entries Max) 2021-03-10 150 47904 0.05 2000 14375 32370 308.42 255.85 1615424400 100.00 0.10 2021-03-10 20:00:00 WAS @ MEM 1 0 joseph36,fadeaway2324,splashvinnie,jdarich 0
7 675532511 draftkings NBA 46045 104450982 NBA $20K mini-MAX [150 Max Entry] 2021-03-10 150 47562 0.50 20000 9972 47562 328.75 264.75 1615424400 2000.00 1.00 2021-03-10 20:00:00 Late Swap 2 1 TashaB 1
...
and creates data.csv
(Screenshot from LibreOffice):
Answered By - Andrej Kesely
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.