What would be the preferred format for a geometry database?

Question

Introduction (analogy to basis set database files)

Many programs have a database of basis sets stored in a file such as the following (so that users can just specify in their input file something like H:STO-2G to get the STO-2G basis set for the H atom):

H:STO-2G
# STO-2G Minimal Basis (2 functions/AO)

0.1309756377D+01 0.2331359749D+00 

0.4301284983D+00 
0.6789135305D+00 

HE:STO-2G
# STO-2G Minimal Basis (2 functions/AO)

0.2432879285D+01 0.4330512863D+00 

0.4301284983D+00 
0.6789135305D+00 

A proposed format for a geometry database file

My group has made something similar for geometries: a database with about 62,000 geometries, and interfaces to PySCF, OpenMolcas, Psi4, CFOUR, Dalton, and MRCC (so far) so that users can just put something like:

geometry = C33H53N5O9:GW5000.v2

in the program's input file to use v2 (version 2) of the GW5000 geometry of the $\ce{C33H53N5O9}$ molecule, rather than copying-and-pasting the 100-line XYZ file with the risk of human error and making the input file maybe 10x longer than it needs to be!

The current format for the geometry database file is such as that each entry looks as follows:

NAME = C33H53N5O9:GW5000.v0
# Number of atoms: 100
# Common name: Unknown
# InChI=1S/C33H53N5O9/c1-20(47-29(2,3)4)22(23(39)35-31(7,8)25(41)37-33(11,12)27(43)45-13)34-24(40)30(5,6)36-26(42)32(9,10)38-28(44)46-19-21-17-15-14-16-18-21/h14-18,20,22H,19H2,1-13H3,(H,34,40)(H,35,39)(H,36,42)(H,37,41)(H,38,44)/t20-,22+/m1/s1

# SMILES : COC(=O)C(NC(=O)C(NC(=O)[C@H]([C@H](OC(C)(C)C)C)NC(=O)C(NC(=O)C(NC(=O)OCc1ccccc1)(C)C)(C)C)(C)C)(C)C

# SMARTS: Unknown

# Reference code: QIXLEO
# CAS number: Unknown
# Source paper: https://doi.org/10.1038/s41597-020-0385-y (Stuke et al.)
# Source data host: https://doi.org/10.14459/2019mp1507656
# Source download link: https://dataserv.ub.tum.de/index.php/s/m1507656 , Filename: df_5k.json
# Source method: def2-TZVPP/PBE+TS-vdW
# Properties: Number of electrons = 123, Spin multiplicity = Singlet, Electronic state = Ground, Charge = 0, Point group = Unknown
# Notes: IR spectroscopy has been done in (2010) Dehghany M, et al. 
# Date first added:   2023-02-04 14:14:48 (by Walter Kohn Jr.)
# Date modified:      2023-04-05 12:01:18 (by Walter Kohn Jr.)
# Date modified:      2023-04-06 19:03:26 (by Nike Dattani)
# Date last modified: 2023-04-07 03:01:58 (by Nike Dattani)

H      37.13229674      36.52932952      30.63502894 
H      37.87447083      34.45373990      31.61484874 
H      38.29175516      34.28685128      29.88425431 
H      39.43188162      36.70033091      26.00124438 
C      36.88232957      40.90658315      31.61111721 
...

Parts of the proposed format that "need" to be kept

Since every line that starts with the # character is a "comment", we really only need the NAME and the XYZ lines, but the following information in the "comments" is extremely useful:

• Number of atoms allows us to quickly find all 30-atom molecules (for example)
• Common name allows us to quickly find all stored geometries for "butane" (for example)
• Source paper and Source data host and Source download link/Filename are crucial for "reproducibility" and crediting the papers from which certain geometries originate.
• Source method is "crucial" for reproducibility and it tells us whether it's an experimental geometry from, for example, microwave spectroscopy (quite reliable) or a theoretical geometry from STO-2G/Hartree-Fock (very reliable).

Parts of the proposed format that are more questionable

• InChI,SMILES,SMARTS,Reference code, and CAS number seem useful, since so many different molecules can have the same chemical formula, and might not have a "common name". However, it seems that there's so many molecular identifiers, for example here you can see that there's an "EINECS" number, a "Beilstein" number, an "RN" number, an "MDL number" several "DBID" numbers, and much more that I could include here, including different "types" of SMILES formats. It seems that at least one specifier in addition to the chemical formula and "common name" must be provided, but which one?
• The Date added and Date modified comments aren't strictly necessary because the revision history would ideally be visible from GitHub, but it doesn't seem harmful to include this information here (it's not like we have to choose which information to include form dozens of options, as was the case in the previous bullet point). However, maybe we only need the first Date added and the last Date modified?

Parts of the proposed format that I would like to keep, but am not sure how to include them in the best way

• Some of the Properties are quite important (e.g. charge, spin, and electronic state), since they tell the user which keywords to use when doing an electronic structure calculation, for example. This information could even go in the NAME variable, which can really be anything (it doesn't need to be the chemical formula), and you might even think this would be better than the format I currently have, so I welcome feedback on that. One reason why I included the "number of electrons" was so that I can easily get a rough idea of how hard a calculation with this molecule would be, but this isn't strictly necessary. The "point group" was included for similar reasons, for example if I wanted to do a study on only linear molecules.

What format do you think would be best?

If you think any part of my proposed format can be improved (for example, removing unnecessary information, or adding more useful information, or making a change like C33H53N5O9:GW5000.v2 to C33H53N5O9_GW5000.(v2)), please paste it in a code block!

Answer 1

This answer is now a community wiki.

Regarding the parameters in the database file, here are my views:

• The first Date added and the last Date modified are sufficient since the rest of the file versioning can be obtained from git.

• I think it is good for the properties such as the charge, spin, and electronic state to not be included in the name. It is good for them to be on a separate line since I see them as metadata. If we need them as query parameters to search for some particular characteristics of molecules, then perhaps it is good to have it in the NAME itself.

---

Regarding the XYZ information, it would be good to prepend the XYZ coordinates information with a string like : # XYZ Begin and add a line at the end of the XYZ info like : # XYZ End

For example:

s = '''NAME = C10H15NO2S2:GW5000.v0
# Number of atoms: 30
# Common name: Unknown
# InChI=1S/C10H15NO2S2/c1-5-8-7(3)15-10(14)11(8)6(2)9(12)13-4/h6H,5H2,1-4H3/t6-/m1/s1
# SMILES : COC(=O)[C@H](n1c(=S)sc(c1CC)C)C
# Smarts: Unknown
# Reference code: FOVWES
# CAS: Unknown
# Source paper: https://doi.org/10.1038/s41597-020-0385-y (Stuke et al.)
# Source data host: https://doi.org/10.14459/2019mp1507656
# Source download link: https://dataserv.ub.tum.de/index.php/s/m1507656 , Filename: df_5k.json
# Source method: PBE_TS-vdW
# Properties: Number of electrons = 46, Spin multiplicity = Singlet, Electronic state = Ground, Charge = 0, Point group = Unknown
# Date added: 2023-02-05  14:14:48 (Vandan Revanur)
# Date modified: 2023-04-09 11:35:40.422937 (Vandan Revanur)
# XYZ Begin
30

C      37.56022072      46.42557279      47.12244195 
N      38.41486173      47.34224375      47.69527868 
C      38.40387895      48.62793964      47.12650724 
C      37.56283274      48.72707549      46.05929103 
S      37.27624844      44.86270782      47.60509887 
C      39.26642677      46.96082591      48.83037942 
C      38.45126980      46.41549852      50.01494371 
O      38.86895847      45.59870651      50.80522393 
O      37.28597863      47.09279152      50.14396759 
C      36.43011242      46.62401114      51.20431940 
H      39.69355182      47.89896576      49.21150415 
H      35.56314436      47.28997420      51.19573739 
H      36.94702454      46.66765744      52.17066399 
H      36.12213864      45.59052324      51.00135426 
H      36.21596648      50.16390411      45.18266298 
H      37.56492846      49.69470384      44.13523550 
C      37.28353465      49.90044176      45.17838095 
C      39.25272587      49.73859624      47.67187517 
H      37.84704948      50.78361100      45.50151663 
H      39.19453702      49.75075573      48.77118150 
H      38.79609693      50.68564642      47.35631688 
C      40.41180910      46.03181425      48.43355973 
H      41.24233846      48.80775624      47.54934542 
H      40.77986133      49.73738376      46.11606557 
H      40.02347830      45.07764438      48.06231905 
H      41.04052914      45.83337078      49.30836666 
H      41.02130010      46.50140795      47.65197806 
C      40.71890084      49.71107794      47.21170776 
H      41.26362809      50.57895454      47.60606918 
# XYZ End'''

With this one can access the XYZ information of any molecule with the name like this:

molecule_name = 'C10H15NO2S2:GW5000.v0'
p = re.compile(f'(NAME = {molecule_name})(.*?)# XYZ Begin(.*?)# XYZ End', flags=re.DOTALL)
result = p.search(s)

result.group(3) would output:

>> '\n30\n\nC      37.56022072      46.42557279      47.12244195 \nN      38.41486173      47.34224375      47.69527868 \nC      38.40387895      48.62793964      47.12650724 \nC      37.56283274      48.72707549      46.05929103 \nS      37.27624844      44.86270782      47.60509887 \nC      39.26642677      46.96082591      48.83037942 \nC      38.45126980      46.41549852      50.01494371 \nO      38.86895847      45.59870651      50.80522393 \nO      37.28597863      47.09279152      50.14396759 \nC      36.43011242      46.62401114      51.20431940 \nH      39.69355182      47.89896576      49.21150415 \nH      35.56314436      47.28997420      51.19573739 \nH      36.94702454      46.66765744      52.17066399 \nH      36.12213864      45.59052324      51.00135426 \nH      36.21596648      50.16390411      45.18266298 \nH      37.56492846      49.69470384      44.13523550 \nC      37.28353465      49.90044176      45.17838095 \nC      39.25272587      49.73859624      47.67187517 \nH      37.84704948      50.78361100      45.50151663 \nH      39.19453702      49.75075573      48.77118150 \nH      38.79609693      50.68564642      47.35631688 \nC      40.41180910      46.03181425      48.43355973 \nH      41.24233846      48.80775624      47.54934542 \nH      40.77986133      49.73738376      46.11606557 \nH      40.02347830      45.07764438      48.06231905 \nH      41.04052914      45.83337078      49.30836666 \nH      41.02130010      46.50140795      47.65197806 \nC      40.71890084      49.71107794      47.21170776 \nH      41.26362809      50.57895454      47.60606918 \n'

This XYZ string can be further used downstream with libraries such as rdkit like:

from rdkit import Chem
Chem.MolFromXYZBlock(xyz_string)

If we decide to not have XYZ Begin and XYZ End, we can still retrieve the XYZ info using the following regex, which retrieves the XYZ info between the line next after the date modified until the next NAME line:

molecule_name = 'C10H15NO2S2:GW5000.v0'
p = re.compile(f'(NAME = {molecule_name})(.*?)Date modified:(.*?)\)(.*?)NAME', flags=re.DOTALL)
result = p.search(s)
result.group(4)
>> '\n\nC      37.56022072      46.42557279      47.12244195 \nN      38.41486173      47.34224375      47.69527868 \nC      38.40387895      48.62793964      47.12650724 \nC      37.56283274      48.72707549      46.05929103 \nS      37.27624844      44.86270782      47.60509887 \nC      39.26642677      46.96082591      48.83037942 \nC      38.45126980      46.41549852      50.01494371 \nO      38.86895847      45.59870651      50.80522393 \nO      37.28597863      47.09279152      50.14396759 \nC      36.43011242      46.62401114      51.20431940 \nH      39.69355182      47.89896576      49.21150415 \nH      35.56314436      47.28997420      51.19573739 \nH      36.94702454      46.66765744      52.17066399 \nH      36.12213864      45.59052324      51.00135426 \nH      36.21596648      50.16390411      45.18266298 \nH      37.56492846      49.69470384      44.13523550 \nC      37.28353465      49.90044176      45.17838095 \nC      39.25272587      49.73859624      47.67187517 \nH      37.84704948      50.78361100      45.50151663 \nH      39.19453702      49.75075573      48.77118150 \nH      38.79609693      50.68564642      47.35631688 \nC      40.41180910      46.03181425      48.43355973 \nH      41.24233846      48.80775624      47.54934542 \nH      40.77986133      49.73738376      46.11606557 \nH      40.02347830      45.07764438      48.06231905 \nH      41.04052914      45.83337078      49.30836666 \nH      41.02130010      46.50140795      47.65197806 \nC      40.71890084      49.71107794      47.21170776 \nH      41.26362809      50.57895454      47.60606918 \n\n'

Answer 2

Several efforts exist for a database / repository of computational chemistry files. I'm most familiar with the molecular-based examples.

You asked about a unique identifier. As far as a unique identifier, I would strongly suggest either the InChI or InChIKey which are unambiguous, developed by IUPAC, and support reasonable normalization and standardization.

Probably the most successful repository is ioChem-BD which uses CML as the internal format. You can also run the software locally too since the service is open source.

The MolSSI team has QCFractal / QCArchive which seems to be mostly used for their effort at running quantum chemical calculations for force field design. (Their internal representation is evidently QCSchema.)

The OpenChemistry project has offered Chemical JSON as an open community interchange format (i.e., "better than molden") and it's been adopted by Avogadro and cclib. There were two efforts at a server / repository:

• OpenChemVault
• MongoChem

At the moment, I'm hoping we can push for a newer effort, including a Google Summer of Code project idea for this.

If you (or others) are interested in mentoring or providing input, please send me an e-mail.

Answer 3

It is not really clear from the title and the long description what the issue is. To me, it sounds like you just want the .xyz data but in a clearly separable form so that one does not have to parse the file separately. It also sounds like you want to be able to add additional information like the SMILES string, CAS number, etc, and the question appears to be whether there is a portable way to do this?

This type of data is most easily handled by

1. a database format, or
2. a Python-type dictionary (C++ map, etc)

There already exist several data formats for exchanging structured files, like JSON and XML. I would recommend JSON, with which you can define data formats and for which there are interfaces for several languages like:

• https://github.com/json-c/json-c (C)
• https://github.com/nlohmann/json (C++)
• https://github.com/jacobwilliams/json-fortran (Fortran)
• https://docs.python.org/3/library/json.html (Python).

JSON is also used as the internal storage format of the Basis Set Exchange, whose web interface is available at http://basissetexchange.org. If you need to store basis set data, you should either tag the versioned basis set from the Basis Set Exchange, or explicitly include the basis in the JSON format used by the Basis Set Exchange, which can then be printed out by BSE functionality.