How to programmatically scrape the Internet?

Question

Programming

I really want to know how a program is implemented that can easily download pictures or collect any information specified in the program. In what language should such programs be written (from the linux console)?

Answer 1

No difference. Which one you know better. All universal programming languages ​​allow you to create such a program.
If the task is simple and quite specific, then it is quite possible to get by with ready-made curl or wget programs and you don’t need to write any program at all, except perhaps a simple script that runs curl or wget (and this script is not necessary, you can run curl / wget simply from the command line lines).
And the program is needed if you need to read a page from the site, analyze its content and proceed to download other files from the same site using the links that are contained on the page downloaded before.
Although in the simplest cases, recursive downloading can also be solved with simple utilities (maybe the same wget and curl can also do this).

Answer 2

C ++ on sockets ... finally the norms, and you investigate the protection