Answer the question
In order to leave comments, you need to log in
How to force selenium to load the part of the page that is loaded after the scrollbar is scrolled?
Let's say I need to parse a page: page .
Selenium only gives out 18 olympiads out of 251. How can I make it load the page completely?
The code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from bs4 import BeautifulSoup
ser = Service(r'C:\Users\DNS\Desktop\Python\Test\Проект\Driver\chromedriver.exe')
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--normal')
browser = webdriver.Chrome(service=ser, options=options)
browser.get('https://olimpiada.ru/activities?type=any&class=9&period_date=&period=year&perechen=on')
requiredHtml = browser.page_source
soup = BeautifulSoup(requiredHtml, 'lxml')
open("Htm.html", "w", encoding="utf-16").write(str(soup))
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question