How to create a zip archive from a folder on S3 in python within AWS lambda?

Question

Python

The bucket contains a folder with files (conditionally input). It stores some files with a total size of over 500 MB.
How to create an archive next to a bucket folder within the AWS lambda constraint?
Ps I heard about streaming recording, but I can't figure it out yet...

Answer 1

No way. Generally.

Answer 2

It rests on the capabilities of Python and AWS-Lambda. As far as I remember, there are memory limits for the lambda and a time limit for 1 handler session. But I think that this problem can be solved. Compression libraries must be streamable. That is, without accumulating the result in memory. And the compression ratio must be set to store so that it is as fast as possible.
In general, within the framework of this forum, it is almost impossible to discuss. we need sorts and at least 10 experiments. Try. I think this is solvable.

Answer 3

You can create an object in the lambda's RAM (up to three gigabytes is available there) and zip it. Right in my memory. And from there, put it on s3 already with a zip. I once did something like this.