Answer the question
In order to leave comments, you need to log in
Where is it better to store page snapshots and how to load the robot?
Good afternoon, there is a task to store the full unloading of the page (since a lot of data is loaded via ajax (in the form of an endless scroll), and the search engine needs to show this whole thing).
Unloading I think to do once a week according to the schedule.
There were two questions
1. to store simply key value? Key - page, value - full content
2. Uploading with more than 2400 pictures will come out, how will the robot behave with it? Maybe it's worth paginating this case, as it was done before?
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question