How to download 12 million Instagram photos?

Question

API

Hello.
The task is to download photos from 12 million profiles, about 50 photos from each.
The API is probably not worth considering, + I heard that it is problematic to get access to it.
Questions are simple:
-How to do?
-What instrument?
- Has anyone had any experience?
-What are the restrictions on the API, and on http requests?
I would appreciate any advice and experience.

Answer 1

API for Instagram is problematic to get. their native API (which was before they were bought by Facebook) is declared deprecated and in fact almost does not work, and the new one, through Facebook, requires a lot of confirmations, and it is unlikely that Facebook will be allowed to massively pull data through the API.
However, in the case of Instagram, this is generally not necessary. the site is written in React (and, unlike Facebook's meaty code, it is written very cleanly and modernly), so in fact the process of collecting data in their case is extremely simple:
When querying the GraphQL API, one thing needs to be taken into account. Now Instagram requires a "signature" of requests, it's done like this:
The variable varspassed to JSON.stringify()is the content of the field variablesfrom the GraphQL query.
Instagram itself does not ban for constant requests, but if you start getting impudent, it turns on throttling, starting to give HTTP 429 for some time (up to 5-10 minutes), after which everything moves on. Thus, parallelization + proxies solve the problem quite effectively.

Answer 2

Scrapinghub
or steam yourself with a proxy, the
restrictions will be on the frequency of requests and on the number, about 5000 per IP, but look better https://toster.ru/user/opium - it's in the Insta dock