K
K
kopelev20002019-01-31 21:45:39
Python
kopelev2000, 2019-01-31 21:45:39

How to go through all the pages on the site. Page switching js script?

Please tell me how to go through all the pages on the site.
Pages are switched by js script:
5c53421e9ff5a380743023.png
No ideas at all... .

Answer the question

In order to leave comments, you need to log in

3 answer(s)
O
Oksana ..., 2019-01-31
@kopelev2000

Just like manually: in the layout, look for these numbers and from the number 2 (since the first one is already open) click () on the object with selenium. Wait 3-5 seconds for the page to fully load and parse the information. Then go to 3 and again wait, parse; then on 4 ... and so on until the pages run out. And do not forget to specify the condition “if the element is not found, then ...”
And also, judging by the search string, the page number is displayed there as p = 1. That is, you can also give driver.get() the address, increasing each time by one.

A
Alexander, 2019-02-01
@NeiroNx

Well, selenium is certainly good - but it could have been done like this:
https://www.udemy.com/api-2.0/discovery-units/all_...
(this link may be slightly different for you - you need to look in network activity)
p =1 and page_size=12 (set this one to 100500 and pull everything at once)
and the answer is already in JSON

V
Vadim Shatalov, 2019-01-31
@netpastor

selenium

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question