R
R
Romses Panagiotis2021-04-11 17:21:05
Amazon Web Services
Romses Panagiotis, 2021-04-11 17:21:05

Is there a utility to view the list of files in a ZIP on an S3 bucket?

We need some utility that would allow us to browse the directory of files inside a ZIP file residing on an S3 bucket without downloading it locally.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
E
Eugene, 2021-04-11
@romesses

example:
https://stackoverflow.com/questions/41789176/how-t...
In short, the algorithm:
1. find out the size of the zip file
2. read a few bytes from the zip file at a known address in order to find out the location and size of the listing files in a zip file
3. read a few bytes from a zip file with a listing of files
4. create our own empty zip file by inserting the received bytes there at the desired addresses.
functions of working with zip we issue a list of files
PS script was not written by me, but a couple of years ago I also solved a similar problem.
The "save" here is reading part of the file, which allows S3 and knowledge of the zip format
In the comments to the answer on SO, it is written about the zip size limit of 4GB - I did not encounter it and did not solve the task of bypassing the problem.

I
Ivan Shumov, 2021-04-11
@inoise

Technically not feasible

R
rPman, 2021-04-11
@rPman

for ubuntu, you can connect s3 as a regular directory using the fuse s3fs utility and work with the archive with any archiver

R
Roman Mirilaczvili, 2021-04-14
@2ord

There is a tool, remotezip . Suitable for getting a list and partial extraction of files via HTTP. I don’t know about the utility for S3, it seems that you can modify it if necessary for private access to the bucket.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question