Y
Y
Yura2018-04-08 03:12:25
Parsing
Yura, 2018-04-08 03:12:25

Is there a character brute-force search engine for site pages?

Hello, in order to manage the site, I need to select links to all the pages that the site has. The problem is that there are pages to which there are no links and I cannot collect them with standard utilities.
Question: is there a character enumerator for searching site pages?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
X
xmoonlight, 2018-04-08
@xmoonlight

The problem is that there are pages to which there are no links and I cannot collect them with standard utilities.
Then how do you know the pages are there if they don't have links?!

X
xDimus, 2018-04-08
@xDimus

How do you want to iterate them?
If so [aaaa.html, aaab.html, aaac.html...] ​​then it's easy, define the character set, length and go - only the site is unlikely to like such a number of requests.
I would first look at the information in sitemap.xml, robots.txt and what pages have already been found / indexed by search engines.

R
rPman, 2018-04-08
@rPman

Once upon a time, I worked with a project to analyze the Internet for links to different file storages, all sorts of rapidshares, and so on, for this, requests were made to the search engine according to a list of keywords, pages were loaded using links, and the content on these links was analyzed.
In fact, you have no other way to get links to the content of the site.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question