Answer the question
In order to leave comments, you need to log in
How to parse dynamic text in python?
I am writing a small exchange rate parser in python and the requests and beautifulsoup4 libraries.
Here is the site I want to parse.
Here is the parse code for the tr I need
def parse():
URL = 'http://sky.bank/uk/currency'
HEADERS = {
'user-agent': 'my user agent'
}
response = requests.get(URL, headers=HEADERS, verify=False)
soup = BeautifulSoup(response.content, 'html.parser')
items = soup.findAll('tr')
print(items[1])
print(items[2])
print(items[3])
Answer the question
In order to leave comments, you need to log in
They are updated by the script for sure. And the script probably makes a background request to the backend of the site.
From here:
a) Find out what kind of request it is and learn how to execute it yourself using requests. Bonus - it's probably something human-readable, like JSON, not HTML.
b) Use a full-fledged browser that can execute scripts, such as selenium. Cumbersome and slow, but it allows you to successfully mow for the average user.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question