M
M
Max2020-01-30 15:53:07
Browsers
Max, 2020-01-30 15:53:07

What are lightweight web scrapers that mimic user actions and execute js?

Hello!

There is a task to launch as many sessions as possible in a short time and for a certain time - 5-10 minutes, log in and perform certain actions (press buttons, fill out forms, collect information).

Before that, I used puppeteer (including headless), but it turned out to asynchronously launch only about 10 chrome windows without problems, then the program started to lag. I tried to split it into processes and threads, the result is not much better and instead of the program, the computer began to lag.

Ideally run 30-200 sessions, are there any solutions for a similar task? Not necessarily on nodejs.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
V
Vladimir Korotenko, 2020-01-31
@maximrabotaet

There is apache jmeter
big question what you need for interactive.
If you know Java, then you can generally write your own module

M
Mikhail Osher, 2020-01-30
@miraage

Observations running 2 million headless sessions

A
Alexander Taratin, 2020-01-31
@Taraflex

zombie.js.org

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question