How to approach storage of heavy resources in an executable file?

H

horror_x2012-06-01 12:48:06

Programming

horror_x, 2012-06-01 12:48:06

There was a need to store large resources in single executable files. In particular, MSVC is used in conjunction with Qt. The standard Qt and Win API resource schemes are not suitable, since it is strongly undesirable to load resources into memory - ideally, they should be read in parts from the executable file.
An example is the self-extracting archives that 7-zip and WinRar create. I would like to achieve the same effect at the build stage, without resorting to post-processing of executable files and parsing their structure during execution.

Reply

Answer the question

In order to leave comments, you need to log in

4 answer(s)

H

horror_x, 2012-06-04
@horror_x

A great solution was suggested by amirul :

I really dislike tens of megabyte executable files, but since you have already decided to use them anyway, and the only decision left to make is, in fact, how, then use the resources and don’t fool around. Windows just uses mapping to load executable files with page demand paging at the time of access.

Those. the standard Windows scheme is still suitable for these purposes.
Another suitable solution was suggested by leotsarev and 1099511627776 - to add data to the end of the executable file, and then the ways to manipulate them depend only on the imagination.

M

Michael, 2012-06-01
@1099511627776

Well, and you tried according to the RARJPEG principle - to glue the resources with the executable file.
And then, in the process of loading, by tags in the file, find reurs and work with them

L

Leonid Tsarev, 2012-06-02
@leotsarev

An example is the self-extracting archives that 7-zip and WinRar create.
They work like this: at the end of the PE file, after everything, the actual archive is appended at the end. The program opens its own EXE for reading and looks for the archive header without bothering to parse the PE file itself.

M

Michael, 2012-06-01
@1099511627776

It would be nice for you to find, for example, how WoW works with its resources, there are resources of a hoo Gigabyte, and it doesn’t keep everything in memory.
As options:
- make your own archive format,
- shove some kind of database into BLOBs and then read them from there
- make zip - with coef. compression 0 to quickly unpack and load into memory