D
D
DimaSensation2019-03-06 13:33:39
Python
DimaSensation, 2019-03-06 13:33:39

What to do if Splash stops responding and takes up a lot of space in docker?

Used by scrapy via scrapyd and splash in docker.
Has anyone experienced the following two situations?
1) Periodically, splash stops responding to requests from scrapy and no errors are thrown into the log. With what it can be connected?
2) splash in docker takes up space on the drive with each new render and does not give it back. As a result, the container size can grow to hundreds of gigabytes. Are there other methods besides stopping the container and then deleting it.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
P
Pavel Zamyatin, 2019-03-06
@corw

My splash has a reasonable size, but has a tendency to eat up RAM. I had to limit 8Gb, the flight is normal. Sometimes, just in case, I reload the container after large projects.

A
alternativshik, 2019-03-06
@alternativshik

splash often crashes, it's even written in its documentation.
Write an issue on their github and wait for answers from the developers.
PS it did not grow for me and just did not hang, it just restarted under load.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question