Issue
I'm trying to get the data out of a bus schedule. It should update in real time. The page I want to do this with is https://fahrplan.oebb.at/bin/stboard.exe/dn?L=...vs_liveticker&evaId=493034&boardType=dep&productsFilter=1011111111011&dirInput=&tickerID=dep&start=yes&eqstops=false&showJourneys=12&additionalTime=0. I tried downloading the page's html source every 30 seconds and accessing the table from there but the downloaded file only includes the javascript which generates the table, not the table itself. Is there a way to either run the script locally and save the data to a file or to get the server to run the script? Thank you in advance!
Solution
You need a bit more gymnastics before getting to some usable info, given the binary response. First, you can get the api endpoint scrapeed via XHR by inspecting the Dev Tools - Network tab in Chrome browser. The following code will return some meaningful data:
import requests
import pandas as pd
import json
url = 'https://fahrplan.oebb.at/bin/stboard.exe/dn?L=...vs_liveticker&evaId=493034&boardType=dep&time=10:05&productsFilter=1011111111011&additionalTime=0&disableEquivs=yes&maxJourneys=50&outputMode=tickerDataOnly&start=yes&selectDate=today'
r = requests.get(url)
json_obj = json.loads(r.text.split('journeysObj = ')[1])
df = pd.json_normalize(json_obj['journey'])
print(df)
This will return:
| id | ti | da | pr | st | lastStop | ati | tr | trChg | rt | rta | rt.status | rt.dlm | rt.dlt | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 479996818 | 10:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 10:39 | False | 0 | False | nan | nan | nan | |
| 1 | 479996819 | 10:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 10:54 | False | 0 | False | nan | nan | nan | |
| 2 | 479996819 | 10:28 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 10:54 | False | 0 | False | nan | nan | nan | |
| 3 | 4799968110 | 10:31 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 11:09 | False | 0 | False | nan | nan | nan | |
| 4 | 4799968110 | 10:43 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 11:09 | False | 0 | False | nan | nan | nan | |
| 5 | 4799968111 | 10:46 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 11:24 | False | 0 | False | nan | nan | nan | |
| 6 | 4799968111 | 10:58 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 11:24 | False | 0 | False | nan | nan | nan | |
| 7 | 437788811 | 11:01 | 16.08.2022 | Bus 646 | Wels Hbf via Krenglbach | Wels Hbf (Busterminal) | 11:09 | False | nan | False | nan | 4 | 11:05 | |
| 8 | 4799968112 | 11:01 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 11:39 | False | 0 | False | nan | nan | nan | |
| 9 | 4799968112 | 11:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 11:39 | False | 0 | False | nan | nan | nan | |
| 10 | 4799968113 | 11:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 11:54 | False | 0 | False | nan | nan | nan | |
| 11 | 4799968113 | 11:28 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 11:54 | False | 0 | False | nan | nan | nan | |
| 12 | 4799968114 | 11:31 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 12:09 | False | 0 | False | nan | nan | nan | |
| 13 | 437838812 | 11:40 | 16.08.2022 | Bus 646 | Aistersheim | Aistersheim Kriegerdenkmal | 12:19 | False | 0 | False | nan | nan | nan | |
| 14 | 4799968114 | 11:43 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 12:09 | False | 0 | False | nan | nan | nan | |
| 15 | 4799968115 | 11:46 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 12:24 | False | 0 | False | nan | nan | nan | |
| 16 | 4799968115 | 11:58 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 12:24 | False | 0 | False | nan | nan | nan | |
| 17 | 4799968116 | 12:01 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 12:39 | False | 0 | False | nan | nan | nan | |
| 18 | 4799968116 | 12:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 12:39 | False | 0 | False | nan | nan | nan | |
| 19 | 4799968117 | 12:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 12:54 | False | 0 | False | nan | nan | nan | |
| 20 | 4799968117 | 12:28 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 12:54 | False | 0 | False | nan | nan | nan | |
| 21 | 4799968118 | 12:31 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 13:09 | False | 0 | False | nan | nan | nan | |
| 22 | 4799968118 | 12:43 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 13:09 | False | 0 | False | nan | nan | nan | |
| 23 | 4799968119 | 12:46 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 13:24 | False | 0 | False | nan | nan | nan | |
| 24 | 4799968119 | 12:58 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 13:24 | False | 0 | False | nan | nan | nan | |
| 25 | 437811810 | 13:00 | 16.08.2022 | Bus 646 | Aistersheim | Aistersheim Kriegerdenkmal | 13:39 | False | 0 | False | nan | nan | nan | |
| 26 | 437788812 | 13:01 | 16.08.2022 | Bus 646 | Wels Hbf via Krenglbach | Wels Hbf (Busterminal) | 13:09 | False | 0 | False | nan | nan | nan | |
| 27 | 4799968120 | 13:01 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 13:39 | False | 0 | False | nan | nan | nan | |
| 28 | 4799968120 | 13:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 13:39 | False | 0 | False | nan | nan | nan | |
| 29 | 4799968121 | 13:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 13:54 | False | 0 | False | nan | nan | nan | |
| 30 | 4799968121 | 13:28 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 13:54 | False | 0 | False | nan | nan | nan | |
| 31 | 4799968122 | 13:31 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 14:09 | False | 0 | False | nan | nan | nan | |
| 32 | 437866810 | 13:40 | 16.08.2022 | Bus 646 | Pichl b.Wels | Pichl b.Wels Ortsmitte | 14:03 | False | 0 | False | nan | nan | nan | |
| 33 | 4799968122 | 13:43 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 14:09 | False | 0 | False | nan | nan | nan | |
| 34 | 4799968123 | 13:46 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 14:24 | False | 0 | False | nan | nan | nan | |
| 35 | 4799968123 | 13:58 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 14:24 | False | 0 | False | nan | nan | nan | |
| 36 | 4799968124 | 14:01 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 14:39 | False | 0 | False | nan | nan | nan | |
| 37 | 4799968124 | 14:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 14:39 | False | 0 | False | nan | nan | nan | |
| 38 | 4799968125 | 14:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 14:54 | False | 0 | False | nan | nan | nan | |
| 39 | 437853810 | 14:20 | 16.08.2022 | Bus 646 | Haag am Hausruck | Haag am Hausruck Bahnhofstraße | 15:00 | False | 0 | False | nan | nan | nan | |
| 40 | 4799968125 | 14:28 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 14:54 | False | 0 | False | nan | nan | nan | |
| 41 | 437802811 | 14:31 | 16.08.2022 | Bus 646 | Wels Hbf via Krenglbach | Wels Hbf (Busterminal) | 14:39 | False | 0 | False | nan | nan | nan | |
| 42 | 4799968126 | 14:31 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 15:09 | False | 0 | False | nan | nan | nan | |
| 43 | 4799968126 | 14:43 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 15:09 | False | 0 | False | nan | nan | nan | |
| 44 | 4799968127 | 14:46 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 15:24 | False | 0 | False | nan | nan | nan | |
| 45 | 4799968127 | 14:58 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 15:24 | False | 0 | False | nan | nan | nan | |
| 46 | 437988810 | 15:00 | 16.08.2022 | Bus 646 | Meggenhofen | Meggenhofen Pendlerparkplatz | 15:34 | False | 0 | False | nan | nan | nan | |
| 47 | 4799968128 | 15:01 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 15:39 | False | 0 | False | nan | nan | nan | |
| 48 | 4799968128 | 15:13 | 16.08.2022 | Bus 2 | Wels Ghegastraße | Wels Machstraße | 15:39 | False | 0 | False | nan | nan | nan | |
| 49 | 4799968129 | 15:16 | 16.08.2022 | Bus 2 | Flemingstraße | Wels Machstraße | 15:54 | False | 0 | False | nan | nan | nan |
Answered By - platipus_on_fire
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.