How to do a quick search in the daily zip folder for gigabyte log files?

Question

Data archiving

There is a folder with daily archives, each of which is in its pure form up to 1GB. Total, for 5 years, let it be 1.5Tb.
I want to find a way (for sure someone does it), how to find the necessary information by a word or formula.
What ideas do you have at the moment - install software for indexing the entire folder (but I don’t know how much the base of this index will weigh) such as GOOGLE desktop, YANDEX desktop, dtSearch and others, you also need to study how they differ (speed, search in archives, work with an index, index size, memory consumption - important).
Of the other options (simpler, but apparently longer), I consider CLI utilities, ready-made scripts that allow you to simply search for the desired text in large files without filling the system memory with the size of the file being opened and without taking up hard disk resources. That is, all work must be done within the allocated amount of RAM (512MB, for example) without fidgeting the hard disk in vain, that is, not reading and writing to it at the same time. For example, in portions, get 100-200MB to search for data, save results, and so on.
There is another option - maybe there are utilities that convert log files (of an arbitrary type) into a readable form, clearing the logs of garbage (large amounts of unnecessary binary data may be present), or, for example, converting to a database, according to a given algorithm, and then enjoy SQL queries for any search.
Maybe I'm generally moving in the wrong direction and there are ready-made products (free of course) that make it even easier.
PS. Logs are generated by closed software, i.e. Initially, it is not possible to release them in an arbitrary structure.

Answer 1

Not only universal search engines.
And there are more specialized systems designed specifically for analyzing logs.
As online (for some money, this is understandable, since the volumes are rather big).
So are those that can be downloaded and installed locally. Free and paid.
https://www.splunk.com/en_us/products/splunk-light.html
etc.
there are a lot of them.