A
A
activebomb2019-07-26 16:26:09
Python
activebomb, 2019-07-26 16:26:09

How to parse a link if there is data-ipb="noparse"?

import requests
from bs4 import BeautifulSoup as bs

headers = {'accept': '*/*',
           'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.0.2; en-us; Galaxy Nexus Build/ICL53F) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}
base_url = 'https://www.biznet.ru/topic305118.html/page-2#entry2183295'

def bf_parse(base_url, headers):
    session = requests.session()
    request = session.get(base_url, headers=headers)
    if request.status_code == 200:
        soup = bs(request.content, 'html.parser')
        div = soup.find_all('div', attrs={'class': "topic_reply"})
        print(div)
        
bf_parse(base_url, headers)

I tried (d.find('a') for d in soup.find_all('div', attrs={'class': "topic_reply"})), it didn't work, you need to pick up all the links (href) that are located around here (for some reason, I sin that nothing works for me because of the data-ipb="no-parse" line.5d3aff4a85c92111618635.png

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question