N
N
Nek1o2019-07-13 17:48:32
Python
Nek1o, 2019-07-13 17:48:32

Why is scrolling stuck in a div block?

Using Selenium, I'm trying to scroll a div block with Instagram community followers to the very end (there are more than 4 thousand posts), but at 2432 people the scrolling freezes, the next batch of users starts loading endlessly. Tried through Chrome and Firefox, tried installing different versions of webdriver, but it didn't help. I write in Python.
Scrolling is done using these two lines in a loop:
scr1 = browser.find_element_by_xpath('html/body/div/div[@role="dialog"]/div[2]')
browser.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", scr1)
I also tried to keep the div from being cluttered with too many users, remove the top entries (li tags) and scroll further, but it didn't help. It is worth saying that scrolling sometimes freezes earlier, but usually on user 2432. It won't be able to load further.
I don't think it's a Selenium issue, as I've tried manually scrolling through this freeze stage, but it still freezes.
Perhaps this is an Instagram limitation?
5d29ed322a0db838574895.png

Answer the question

In order to leave comments, you need to log in

2 answer(s)
I
Ivan Yakushenko, 2019-07-13
@Nek1o

Status Code: 429
5d29f1634d4f9205270232.jpeg

M
Michael, 2019-08-20
@moonz

I also tried to make such an implementation, even with the help of JS I scrolled to the end, waited for it to appear, etc. But this is a bad implementation, besides, you can paint over the browser altogether)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question