How to implement backward compatibility of user data in the program?

Question

Programming

A desktop application in c# is being developed, in which the user will be able to create and maintain their own data project. The development is iterative, with each new version, new functionality appears. Accordingly, the amount of data in the project grows, the format of the saved project also changes (more precisely, it is supplemented).
It is required to implement support for projects of old versions of the program, in particular:
1) Loading of all previous formats of the user project.
2) Saving the current project to the project of the old format.
Actually, the question is: how best to implement backward compatibility of formats, where can I read about it and are there ready-made solutions?

Answer 1

If the data format is only supplemented, then the solution is elementary. Let's say you have 2 classes in your project that implement reading and saving data. With each version, you just have new data and the old ones do not exactly change. Add (if not already done) in each file the version number. In the program, with each innovation, you simply make a couple more read / write classes for the new version. Then just watch the version when opening the file and use the required class for reading.
Another option is more stupid, fix the read / write class so that it ignores data unknown to it. Those. if you open a file from a newer version in an older version of the program, it will simply ignore data unknown to it.
In practice, I saw a very interesting implementation. There was a very clever arrangement of reading. True, the project was in Java.
There was a file read\write class, pseudocode:

class CReader{
public CReader(URL file);
void readData(){
   someStructs;
}
void writeData(){
   someStructs;
}
... другие методы
}

That was the first version of the program, then a new version comes out in which some new data and structures appeared, but the old ones did not change. Pseudocode:

class CReader1 extends CReader{
@Override
void readData(){
   super(); //Выполнить родительский метод
   someNewStructs;
}
@Override
void writeData(){
   super(); //Выполнить родительский метод
   someNewStructs;
}
}

Those. the principle is that eventually all new data that is entered with a new version of the program is always written at the end of the file. The file opens perfectly in older versions of the program and without any errors, just if they want to save the project in the old version and there is data that was not in the old versions, then a warning is displayed when saving the file about the partial loss of information. The solution is stupidly simple and ingenious at the same time.

Answer 2

Keep the migration set from older versions to the latest. When saving an old project, make sure to convert to the latest format. Same thing when opening a project.

Answer 3

In such cases, you write your own VersionUpdater, which rebuilds the data storage structure, i.e. if we want to add a new field to the database, then we need to save the old information and add new information to it (for example, transfer to a new table with the added field). This approach will allow us to correctly process old data and add new ones right in the code, and in VersionUpdater we complete everything correctly.

Answer 4

The easiest way to do this is in XML. Then we can easily add new tags and ignore what is not included in the XML.
The binary file must be done in the manner of XML - in the form of a hierarchical structure of short streams. The most difficult thing is that this binary file does not have to run back and forth, like on a race track. To do this, at each of the levels of the hierarchy, there is one of two.
1. Request a certain sequence of codes, and then ask: do you have this code? Is there this one? Something like (for simplicity, I write as in C ++) ...

BlockReader blk;
int order1[] = { opHeader, opSettings, dirData };
BlockOrder order(blk, order1);

order.require(opHeader);
// считать заголовок
author = blk.readString();

if (order.get(opSettings) {
  // считать настройки
}

order.require(dirData);
blk.enterDir();
  // считать данные таким же образом — там может быть свой BlockOrder
blk.leaveDir();

Yes, why are we asking for block order twice?
First time, that's what. Let's imagine an outdated file that does not have a settings block. We read the header, we see a directory with data instead of the settings block, and immediately the question is: is there something missing or something extra?
The second time is for clarity (the code comments on itself) and protection against errors when the order declared at the beginning does not match the real one.
2. Just read the blocks one by one and interpret. Usually it happens in any collections.

while (blk.getBlock()) {
  switch (blk.opcode) {
  case dirTiledLevel:
     // считать плиточный уровень
  case dirGraphicLevel:
     // считать уровень с фоном — цельной картинкой
  }
}

It is forbidden to mix ordered with collections.
You can also make a bit in the header of each block: Essential. The old version, having stumbled upon such a block and not reading a single byte (or having stumbled upon a directory and not entering), displays an error: the version is clearly newer, it is impossible to read. This is important when there are cross-references in the file.
To write such files, you have to accumulate a block in memory, and then dump it in one sitting (after all, you need to write down the length of the block). However, there is also a version for structures that know their length in advance - then it is unnecessary to accumulate in memory.
Further. In the header, you can save the fields "SavedWithVersion", "MinRequiredVersion", "MinFullySupportingVersion".
We add a new block (or a new field to an existing block) - we raise MinFullySupportingVersion to the current one. We change the structure in such a way that we break compatibility - we raise MinRequiredVersion.
The fact that the version is slightly outdated can also be caught by indirect signs - some block was not completely read, some directory was not included. A block/directory can have a Compatibility flag - you don't need to check them. And vice versa - if an important block is marked as Compatibility, the version is also outdated. Of course, the Essential and Compatibility flags cannot be seen together.
How to do this if the save format is a database (for example, SQLite), I do not know.
But saving to an outdated format, when data structures have gone far ahead, will have to be adjusted by hand. Is it possible, in principle, how to convert to the old format, etc.