W
W
WebDeveloper20162016-10-28 15:31:47
Parsing
WebDeveloper2016, 2016-10-28 15:31:47

How to organize parsing?

Parsing a web page using CsQuery. The problem is that the content that I need to parse appears only when the button is clicked. It is generated by JS I checked. Those. it doesn't exist initially. And I can't parse him. :( How to be? Can I somehow simulate a click? I download the page simply through the WebClient.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
I
Ilya, 2016-10-28
@glebovgin

Use CasperJS for example.

R
Rou1997, 2016-10-29
@Rou1997

The code can be requested via ajax from the server and then placed in the desired block with a javascript. (I suspect it is)

So they tell you about it!
AJAX requests are HTTP requests, so study HTTP and learn how to sniff it, because in DevTools it is also a sniffer, but not very convenient, better Fiddler, Wireshark or Charles, not only to conveniently intercept that request, but also to simulate it and verify that it is correct.
Selenium and Phantom.js will only make things easier in some cases, but without the basics (HTTP) you'll still be hopeful.
There are people who are simply not allowed to become a reverse engineer, it says "master the sniffer", and they read "sit still, master nothing and keep hammering nails with a microscope", I hope this is not about you, but if so, then just do something - to others, to each his own.
It may be worth simulating all requests sequentially, starting with a GET request to the page itself (html), a normal sniffer will allow you to do this, and it also has a query builder for tests.

S
sim3x, 2016-10-28
@sim3x

Opened the page
Opened the developer tools
Clicked
Looked into the networking tab
Parse the link that is called after the click

K
kykyryky, 2016-10-28
@kykyryky

Selenium как вариант. Для имитации кликов самое то. И считывать данные с элементов просто.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question