S
S
Sergey Karbivnichy2020-05-04 16:19:44
Software and Internet Services
Sergey Karbivnichy, 2020-05-04 16:19:44

How to check many sites for availability?

There is a list of sites (over 15k). We need a program that will check each site for availability, and return information for each site (or rather, what each site returns), for example: response code (200,404, etc.), or that there is no such site. (Something like proxy checker, but for sites)
I found the Xenu program, but it checks the entire site, but I only need the main page (or, more simply, to weed out idle sites)
5eb016493f12d233105867.png

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Denis Sechin, 2020-05-04
@tamogavk

The for loop and in it curl the request

D
Dimonchik, 2020-05-04
@dimonchik2013

in XENU there is a list of URLs
https://documentation.help/Xenu-Link-Sleuth/urllis...
I don't remember - can I specify the depth there? take zero or 1
well and so classic:
https://stackoverflow.com/questions/6136022/script...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question