Say I have a MD trajectory that contains a million snapshots (i.e. a million atomic structures). Each snapshot contains some calculated information (either by DFT or some force field) such as energy, forces and potentially other properties. What software can efficiently store these atomic structures into a database file without losing any relevant information? Ideally, one should always be able to reproduce each atomic structure from the database. I'm particularly interested in database programs that can handle huge amount of structural data without taking too much memory space.
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ +1 but you'll likely be limited by the size of the data itself, which has nothing to do with the sofrware. $\endgroup$– Nike DattaniJul 14 at 17:24
-
$\begingroup$ Why not write the frames into any standard compressed trajectory format (like XTC or DCD), and then the per-frame information as a comma- or space-separated values file in plain text? $\endgroup$– Shern Ren TeeJul 14 at 21:47
-
$\begingroup$ MDAnalysis can iterate over trajectory files loading one frame into memory at a time (I'm sure other Python MD packages do too), and you can cache previous searches as a list of frames together with metadata to identify the search. $\endgroup$– Shern Ren TeeJul 14 at 21:49
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
One of the more recent options for the xyz file format is bqbtool by Martin Brehm, which significantly compresses the trajectory files by factor of 20~30 using a lossless compression scheme. One of the main advantages of this compression scheme is that one can use the associated code travis to calculate basic properties such as radial and spatial distribution, structure factor, correlation functions,vibrational spectra and much more directly from the converted bqb file.
You can read more about bqbtool at https://brehm-research.de/bqb.php
