Answer the question
In order to leave comments, you need to log in
How to get the full body of a page?
Guys. I am learning parsing. Everything seems to be clear, but some nuances appeared during the training. Let's say there is a website (NOT ADVERTISING) . By default, 50 products are available for viewing in the catalog, but there is also a filter in which you can specify (show all or just "All"). So when parsing, I get 50 products, but I would like all the products. The idea of curl to pull the bodies of all pages has already arisen. Is there a more elegant way to solve this problem? Let's say somehow programmatically click on "All".
Thanks in advance!
Answer the question
In order to leave comments, you need to log in
Open the site in a browser, launch the development tools (F12), go to the "Network" section, click on the "show all" page, look at the request with all the parameters in the logs.
PS See the simasettings request cookie, where the fragment
changes tos:16:"isInfinityScroll";i:1
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question