7
$\begingroup$

Introduction (analogy to basis set database files)

Many programs have a database of basis sets stored in a file such as the following (so that users can just specify in their input file something like H:STO-2G to get the STO-2G basis set for the H atom):

H:STO-2G
# STO-2G Minimal Basis (2 functions/AO)

0.1309756377D+01 0.2331359749D+00 

0.4301284983D+00 
0.6789135305D+00 

HE:STO-2G
# STO-2G Minimal Basis (2 functions/AO)

0.2432879285D+01 0.4330512863D+00 

0.4301284983D+00 
0.6789135305D+00 

A proposed format for a geometry database file

My group has made something similar for geometries: a database with about 62,000 geometries, and interfaces to PySCF, OpenMolcas, Psi4, CFOUR, Dalton, and MRCC (so far) so that users can just put something like:

geometry = C33H53N5O9:GW5000.v2

in the program's input file to use v2 (version 2) of the GW5000 geometry of the $\ce{C33H53N5O9}$ molecule, rather than copying-and-pasting the 100-line XYZ file with the risk of human error and making the input file maybe 10x longer than it needs to be!

The current format for the geometry database file is such as that each entry looks as follows:

NAME = C33H53N5O9:GW5000.v0
# Number of atoms: 100
# Common name: Unknown
# InChI=1S/C33H53N5O9/c1-20(47-29(2,3)4)22(23(39)35-31(7,8)25(41)37-33(11,12)27(43)45-13)34-24(40)30(5,6)36-26(42)32(9,10)38-28(44)46-19-21-17-15-14-16-18-21/h14-18,20,22H,19H2,1-13H3,(H,34,40)(H,35,39)(H,36,42)(H,37,41)(H,38,44)/t20-,22+/m1/s1

# SMILES : COC(=O)C(NC(=O)C(NC(=O)[C@H]([C@H](OC(C)(C)C)C)NC(=O)C(NC(=O)C(NC(=O)OCc1ccccc1)(C)C)(C)C)(C)C)(C)C

# SMARTS: Unknown

# Reference code: QIXLEO
# CAS number: Unknown
# Source paper: https://doi.org/10.1038/s41597-020-0385-y (Stuke et al.)
# Source data host: https://doi.org/10.14459/2019mp1507656
# Source download link: https://dataserv.ub.tum.de/index.php/s/m1507656 , Filename: df_5k.json
# Source method: def2-TZVPP/PBE+TS-vdW
# Properties: Number of electrons = 123, Spin multiplicity = Singlet, Electronic state = Ground, Charge = 0, Point group = Unknown
# Notes: IR spectroscopy has been done in (2010) Dehghany M, et al. 
# Date first added:   2023-02-04 14:14:48 (by Walter Kohn Jr.)
# Date modified:      2023-04-05 12:01:18 (by Walter Kohn Jr.)
# Date modified:      2023-04-06 19:03:26 (by Nike Dattani)
# Date last modified: 2023-04-07 03:01:58 (by Nike Dattani)

H      37.13229674      36.52932952      30.63502894 
H      37.87447083      34.45373990      31.61484874 
H      38.29175516      34.28685128      29.88425431 
H      39.43188162      36.70033091      26.00124438 
C      36.88232957      40.90658315      31.61111721 
...

Parts of the proposed format that "need" to be kept

Since every line that starts with the # character is a "comment", we really only need the NAME and the XYZ lines, but the following information in the "comments" is extremely useful:

  • Number of atoms allows us to quickly find all 30-atom molecules (for example)
  • Common name allows us to quickly find all stored geometries for "butane" (for example)
  • Source paper and Source data host and Source download link/Filename are crucial for "reproducibility" and crediting the papers from which certain geometries originate.
  • Source method is "crucial" for reproducibility and it tells us whether it's an experimental geometry from, for example, microwave spectroscopy (quite reliable) or a theoretical geometry from STO-2G/Hartree-Fock (very reliable).

Parts of the proposed format that are more questionable

  • InChI,SMILES,SMARTS,Reference code, and CAS number seem useful, since so many different molecules can have the same chemical formula, and might not have a "common name". However, it seems that there's so many molecular identifiers, for example here you can see that there's an "EINECS" number, a "Beilstein" number, an "RN" number, an "MDL number" several "DBID" numbers, and much more that I could include here, including different "types" of SMILES formats. It seems that at least one specifier in addition to the chemical formula and "common name" must be provided, but which one?
  • The Date added and Date modified comments aren't strictly necessary because the revision history would ideally be visible from GitHub, but it doesn't seem harmful to include this information here (it's not like we have to choose which information to include form dozens of options, as was the case in the previous bullet point). However, maybe we only need the first Date added and the last Date modified?

Parts of the proposed format that I would like to keep, but am not sure how to include them in the best way

  • Some of the Properties are quite important (e.g. charge, spin, and electronic state), since they tell the user which keywords to use when doing an electronic structure calculation, for example. This information could even go in the NAME variable, which can really be anything (it doesn't need to be the chemical formula), and you might even think this would be better than the format I currently have, so I welcome feedback on that. One reason why I included the "number of electrons" was so that I can easily get a rough idea of how hard a calculation with this molecule would be, but this isn't strictly necessary. The "point group" was included for similar reasons, for example if I wanted to do a study on only linear molecules.

What format do you think would be best?

If you think any part of my proposed format can be improved (for example, removing unnecessary information, or adding more useful information, or making a change like C33H53N5O9:GW5000.v2 to C33H53N5O9_GW5000.(v2)), please paste it in a code block!

$\endgroup$
6
  • 1
    $\begingroup$ I don't see why the basis set stuff is relevant? Besides, basis sets are a horrible example since they come in dozens of formats and have data incompatibilities. Geometries, however, are trivial: use xyz format and every program will be able to use them. $\endgroup$ Apr 9, 2023 at 17:45
  • 1
    $\begingroup$ @SusiLehtola I think the idea here is about more than the geometry (though this isn't as trivial as you make it seem, considering the various programs that also expect or would like to be able to pass information about connectivity or defining internal coordinates). The meta data that might be packaged along with it, which can be important for e.g. searching for a particular molecule and/or a geometry produced with a particular method. $\endgroup$
    – Tyberius
    Apr 10, 2023 at 0:20
  • 1
    $\begingroup$ Nike, if you haven't come across these before, two relevant projects to look at might be ChemicalJSON and QCSchema, which try to standardize these sort of input/output formats. $\endgroup$
    – Tyberius
    Apr 10, 2023 at 0:29
  • $\begingroup$ I've been talking about the need for comp. chem. repositories for a while. We're hoping to start a Google Summer of Code project this year, so input is welcome. I'm personally vested in ChemicalJSON in part because we want it as an open community effort. $\endgroup$ Apr 10, 2023 at 18:16
  • $\begingroup$ Thanks for those suggestions @Tyberius (and Geoff)! I'll be looking into all of these later in this week! $\endgroup$ Apr 11, 2023 at 3:22

3 Answers 3

4
$\begingroup$

This answer is now a community wiki.

Regarding the parameters in the database file, here are my views:

  • The first Date added and the last Date modified are sufficient since the rest of the file versioning can be obtained from git.

  • I think it is good for the properties such as the charge, spin, and electronic state to not be included in the name. It is good for them to be on a separate line since I see them as metadata. If we need them as query parameters to search for some particular characteristics of molecules, then perhaps it is good to have it in the NAME itself.


Regarding the XYZ information, it would be good to prepend the XYZ coordinates information with a string like : # XYZ Begin and add a line at the end of the XYZ info like : # XYZ End

For example:

s = '''NAME = C10H15NO2S2:GW5000.v0
# Number of atoms: 30
# Common name: Unknown
# InChI=1S/C10H15NO2S2/c1-5-8-7(3)15-10(14)11(8)6(2)9(12)13-4/h6H,5H2,1-4H3/t6-/m1/s1
# SMILES : COC(=O)[C@H](n1c(=S)sc(c1CC)C)C
# Smarts: Unknown
# Reference code: FOVWES
# CAS: Unknown
# Source paper: https://doi.org/10.1038/s41597-020-0385-y (Stuke et al.)
# Source data host: https://doi.org/10.14459/2019mp1507656
# Source download link: https://dataserv.ub.tum.de/index.php/s/m1507656 , Filename: df_5k.json
# Source method: PBE_TS-vdW
# Properties: Number of electrons = 46, Spin multiplicity = Singlet, Electronic state = Ground, Charge = 0, Point group = Unknown
# Date added: 2023-02-05  14:14:48 (Vandan Revanur)
# Date modified: 2023-04-09 11:35:40.422937 (Vandan Revanur)
# XYZ Begin
30

C      37.56022072      46.42557279      47.12244195 
N      38.41486173      47.34224375      47.69527868 
C      38.40387895      48.62793964      47.12650724 
C      37.56283274      48.72707549      46.05929103 
S      37.27624844      44.86270782      47.60509887 
C      39.26642677      46.96082591      48.83037942 
C      38.45126980      46.41549852      50.01494371 
O      38.86895847      45.59870651      50.80522393 
O      37.28597863      47.09279152      50.14396759 
C      36.43011242      46.62401114      51.20431940 
H      39.69355182      47.89896576      49.21150415 
H      35.56314436      47.28997420      51.19573739 
H      36.94702454      46.66765744      52.17066399 
H      36.12213864      45.59052324      51.00135426 
H      36.21596648      50.16390411      45.18266298 
H      37.56492846      49.69470384      44.13523550 
C      37.28353465      49.90044176      45.17838095 
C      39.25272587      49.73859624      47.67187517 
H      37.84704948      50.78361100      45.50151663 
H      39.19453702      49.75075573      48.77118150 
H      38.79609693      50.68564642      47.35631688 
C      40.41180910      46.03181425      48.43355973 
H      41.24233846      48.80775624      47.54934542 
H      40.77986133      49.73738376      46.11606557 
H      40.02347830      45.07764438      48.06231905 
H      41.04052914      45.83337078      49.30836666 
H      41.02130010      46.50140795      47.65197806 
C      40.71890084      49.71107794      47.21170776 
H      41.26362809      50.57895454      47.60606918 
# XYZ End'''

With this one can access the XYZ information of any molecule with the name like this:

molecule_name = 'C10H15NO2S2:GW5000.v0'
p = re.compile(f'(NAME = {molecule_name})(.*?)# XYZ Begin(.*?)# XYZ End', flags=re.DOTALL)
result = p.search(s)

result.group(3) would output:

>> '\n30\n\nC      37.56022072      46.42557279      47.12244195 \nN      38.41486173      47.34224375      47.69527868 \nC      38.40387895      48.62793964      47.12650724 \nC      37.56283274      48.72707549      46.05929103 \nS      37.27624844      44.86270782      47.60509887 \nC      39.26642677      46.96082591      48.83037942 \nC      38.45126980      46.41549852      50.01494371 \nO      38.86895847      45.59870651      50.80522393 \nO      37.28597863      47.09279152      50.14396759 \nC      36.43011242      46.62401114      51.20431940 \nH      39.69355182      47.89896576      49.21150415 \nH      35.56314436      47.28997420      51.19573739 \nH      36.94702454      46.66765744      52.17066399 \nH      36.12213864      45.59052324      51.00135426 \nH      36.21596648      50.16390411      45.18266298 \nH      37.56492846      49.69470384      44.13523550 \nC      37.28353465      49.90044176      45.17838095 \nC      39.25272587      49.73859624      47.67187517 \nH      37.84704948      50.78361100      45.50151663 \nH      39.19453702      49.75075573      48.77118150 \nH      38.79609693      50.68564642      47.35631688 \nC      40.41180910      46.03181425      48.43355973 \nH      41.24233846      48.80775624      47.54934542 \nH      40.77986133      49.73738376      46.11606557 \nH      40.02347830      45.07764438      48.06231905 \nH      41.04052914      45.83337078      49.30836666 \nH      41.02130010      46.50140795      47.65197806 \nC      40.71890084      49.71107794      47.21170776 \nH      41.26362809      50.57895454      47.60606918 \n'

This XYZ string can be further used downstream with libraries such as rdkit like:

from rdkit import Chem
Chem.MolFromXYZBlock(xyz_string)

If we decide to not have XYZ Begin and XYZ End, we can still retrieve the XYZ info using the following regex, which retrieves the XYZ info between the line next after the date modified until the next NAME line:

molecule_name = 'C10H15NO2S2:GW5000.v0'
p = re.compile(f'(NAME = {molecule_name})(.*?)Date modified:(.*?)\)(.*?)NAME', flags=re.DOTALL)
result = p.search(s)
result.group(4)
>> '\n\nC      37.56022072      46.42557279      47.12244195 \nN      38.41486173      47.34224375      47.69527868 \nC      38.40387895      48.62793964      47.12650724 \nC      37.56283274      48.72707549      46.05929103 \nS      37.27624844      44.86270782      47.60509887 \nC      39.26642677      46.96082591      48.83037942 \nC      38.45126980      46.41549852      50.01494371 \nO      38.86895847      45.59870651      50.80522393 \nO      37.28597863      47.09279152      50.14396759 \nC      36.43011242      46.62401114      51.20431940 \nH      39.69355182      47.89896576      49.21150415 \nH      35.56314436      47.28997420      51.19573739 \nH      36.94702454      46.66765744      52.17066399 \nH      36.12213864      45.59052324      51.00135426 \nH      36.21596648      50.16390411      45.18266298 \nH      37.56492846      49.69470384      44.13523550 \nC      37.28353465      49.90044176      45.17838095 \nC      39.25272587      49.73859624      47.67187517 \nH      37.84704948      50.78361100      45.50151663 \nH      39.19453702      49.75075573      48.77118150 \nH      38.79609693      50.68564642      47.35631688 \nC      40.41180910      46.03181425      48.43355973 \nH      41.24233846      48.80775624      47.54934542 \nH      40.77986133      49.73738376      46.11606557 \nH      40.02347830      45.07764438      48.06231905 \nH      41.04052914      45.83337078      49.30836666 \nH      41.02130010      46.50140795      47.65197806 \nC      40.71890084      49.71107794      47.21170776 \nH      41.26362809      50.57895454      47.60606918 \n\n'
$\endgroup$
2
  • $\begingroup$ +1 for some great contributions here. (1) The reason why the properties might be useful in the NAME is because a user might want to calculate the energies for the triplet states (for example) of some molecules: Either triplet can be parsed from the NAME parameter, or from the Properties comment, or something else. I'm not sure which is best. (2) Why is the XYZ Begin and XYZ End needed? Please see my example of the format for basis sets. This format is readable by (for example) CFOUR, ACES, and MRCC. It doesn't have BASIS Begin or BASIS End around the relevant data. $\endgroup$ Apr 9, 2023 at 16:10
  • $\begingroup$ Requiring XYZ Begin and XYZ End introduces one more possible source of error, for example if someone forgets to add these, or if they are typed incorrectly. Also, I wonder about your thoughts on including "InChI,SMILES,SMARTS,Reference code, and CAS number". Do we only need one of them? Which one? As for the modification dates: members of the open source community may feel more incentive to contribute if they are recognized for the modifications that they make, so I don't mind if there's more lines between the "Date first added" and "Date last modified", only for significant changes. $\endgroup$ Apr 9, 2023 at 16:21
5
$\begingroup$

Several efforts exist for a database / repository of computational chemistry files. I'm most familiar with the molecular-based examples.

You asked about a unique identifier. As far as a unique identifier, I would strongly suggest either the InChI or InChIKey which are unambiguous, developed by IUPAC, and support reasonable normalization and standardization.

Probably the most successful repository is ioChem-BD which uses CML as the internal format. You can also run the software locally too since the service is open source.

The MolSSI team has QCFractal / QCArchive which seems to be mostly used for their effort at running quantum chemical calculations for force field design. (Their internal representation is evidently QCSchema.)

The OpenChemistry project has offered Chemical JSON as an open community interchange format (i.e., "better than molden") and it's been adopted by Avogadro and cclib. There were two efforts at a server / repository:

At the moment, I'm hoping we can push for a newer effort, including a Google Summer of Code project idea for this.

If you (or others) are interested in mentoring or providing input, please send me an e-mail.

$\endgroup$
2
  • $\begingroup$ +1 I also got your email and replied there too. I'll look into all of these towards the end of this week! I like the InChI identifier for all of the reasons that you mentioned, but what I don't like is that it's so long. I like the much shorter InChI key, but it has a non-zero collision probability, so two different molecules can have the same InChI Key. Other identifiers such the CAS, EC, ICSC, RTECS, UN, NCI, etc. numbers, seem to not be as unique (I see 10 different types of CAS numbers for methane on PubChem) though. $\endgroup$ Apr 11, 2023 at 3:21
  • $\begingroup$ While it's possible to have collision in an InChI key (as with any hash) in real life practice, I don't think it's a major worry for most people .. unless you're generating millions of unique molecules. $\endgroup$ Apr 11, 2023 at 19:21
1
$\begingroup$

It is not really clear from the title and the long description what the issue is. To me, it sounds like you just want the .xyz data but in a clearly separable form so that one does not have to parse the file separately. It also sounds like you want to be able to add additional information like the SMILES string, CAS number, etc, and the question appears to be whether there is a portable way to do this?

This type of data is most easily handled by

  1. a database format, or
  2. a Python-type dictionary (C++ map, etc)

There already exist several data formats for exchanging structured files, like JSON and XML. I would recommend JSON, with which you can define data formats and for which there are interfaces for several languages like:

JSON is also used as the internal storage format of the Basis Set Exchange, whose web interface is available at http://basissetexchange.org. If you need to store basis set data, you should either tag the versioned basis set from the Basis Set Exchange, or explicitly include the basis in the JSON format used by the Basis Set Exchange, which can then be printed out by BSE functionality.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .