How to write a parser for all files on a site?

Question

C++ / C#

How to write a parser for all files on a site? What would be the search for both html pages and css, js and pictures in the folders that they are on the site, and then save the files to a local disk in the same hierarchy as on the site? It makes no sense to use ready-made programs like WebCopy. I want to sort out this issue. I'm not asking you to write a program. Does anyone have examples or sources? I will be grateful.

Answer 1

Download Sitemap from the site and go through it downloading everything.