Say I have a MD trajectory that contains a million snapshots (i.e. a million atomic structures). Each snapshot contains some calculated information (either by DFT or some force field) such as energy, forces and potentially other properties. What software can efficiently store these atomic structures into a database file without losing any relevant information? Ideally, one should always be able to reproduce each atomic structure from the database. I'm particularly interested in database programs that can handle huge amount of structural data without taking too much memory space.
One of the more recent options for the xyz file format is bqbtool by Martin Brehm, which significantly compresses the trajectory files by factor of 20~30 using a lossless compression scheme. One of the main advantages of this compression scheme is that one can use the associated code travis to calculate basic properties such as radial and spatial distribution, structure factor, correlation functions,vibrational spectra and much more directly from the converted bqb file.
You can read more about bqbtool at https://brehm-research.de/bqb.php