H
H
HexUserHex2020-05-18 00:23:49
selenium
HexUserHex, 2020-05-18 00:23:49

Problem with parsing (slenium)?

I appeal to those who often write parsers, there is one resource from which I try to get data, I tried with bs4 but without success ... today I decided to try with selenium (I use its fork because you can’t change headers in the original ..) although I tried the original and it's the same.
In general, I always get the same blocking message, tell me what to do?

You have been blocked.....


How I try:
#!/usr/bin/env python3
from seleniumwire import webdriver

driver = webdriver.Firefox()
url='https://leboncoin.fr'

driver.header_overrides = [(url, {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36"})]
driver.get(url)

for request in driver.requests:
    if request.response:
        #print(request.path, request.response.status_code, request.response.headers['Content-Type'])
        print(str(request.response))

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question