7
$\begingroup$

I have some directories, named ‘abc-000, abc-001, abc-002, …… abc-100’. The path of the directory can be, for example, \home\abc-000 and so on.

Inside each directory, there is a file named ‘OUTCAR’. The ‘OUTCAR’ file consists of many lines including the following:

enter image description here It's noted that, this paragraph appears 5 times with same headings (but with different values) in the same 'OUTCAR' file, and we need to extract data from the final (last) paragraph only.

Now, the 9 elements of the stress matrix can be represented by the following symbols: enter image description here

I need to use a python or bash code that will read each of those ‘OUTCAR’ files, and then it will save the data from those ‘OUTCAR’ files in one single text or excel file (named: stress.txt or stress.xlsx) in the following way:

abc-000    ii  ij   ik  ji  jj   jk   ki  kj   kk   ForceTotal    StressTotal

abc-001    ii  ij   ik  ji  jj   jk   ki  kj   kk   ForceTotal    StressTotal
…………………………………………………….…………………………………………………………………………………………….……………………………………………………
…………………………………………………….…………………………………………………………………………………………….……………………………………………………
abc-100    ii  ij   ik  ji  jj   jk   ki  kj   kk   ForceTotal    StressTotal

As I am new in coding, can anyone please suggest me how to do it? Any kind of help is greatly appreciated.

View and/or download link of the OUTCAR file: click here

$\endgroup$
10
  • 3
    $\begingroup$ Kindly share your OUTCAR for reference $\endgroup$ Commented Nov 10, 2022 at 12:05
  • 1
    $\begingroup$ +1, but in addition to supplying the input and output files as suggested by Pranav, could you please copy-and-paste all text into code blocks rather than showing screenshots? The text in screenshots is not searchable, and also not visible at all to people using screen readers or with devices that don't load images. $\endgroup$ Commented Nov 10, 2022 at 16:06
  • $\begingroup$ Thanks @Pranavkumar for your response. The OUTCAR file is shared at the end. Please check it. $\endgroup$
    – Sak
    Commented Nov 10, 2022 at 19:01
  • $\begingroup$ Thanks @NikeDattani for your suggestion. The OUTCAR file is given at the end. It's large, so it's link is shared. The stress matrix can be found from there. $\endgroup$
    – Sak
    Commented Nov 10, 2022 at 19:02
  • 1
    $\begingroup$ @Sak I believe Nike is suggesting you make a folder for your question, which you would call 9943 since that is the ID number for this question $\endgroup$
    – Tyberius
    Commented Nov 11, 2022 at 17:02

3 Answers 3

9
$\begingroup$

I have condensed the bash script by using arrays.

for x in 'abc-*/';
do
read a[{1..9}] <<<$(cat ${x%?} | grep "stress matrix" -A6 | tail -6 | head -3 | xargs)
read b[{1,2}] <<<$(cat ${x%?} | grep "stress matrix" -A6 | tail -2 | awk '{print $6}'| xargs)
echo ${x%?} ${a[*]} ${b[*]} >>test.csv
done

test.csv shows the same results

abc-000 -467.71245 24.34138 7.71803 24.34138 -314.11534 -3.45455 7.71803 -3.45455 -371.79956 0.039198 676.007623
$\endgroup$
7
$\begingroup$

This is a basic bash script that can be used. One can make this code better too

for fold in `ls -d test-*/`; 
do
line_no=`grep -n 'stress matrix after NEB' OUTCAR|tail -n 1|cut -d ':' -f1`
sig11=`sed -n "$((${line_no}+1))p" OUTCAR|awk '{print $1}'`
sig12=`sed -n "$((${line_no}+1))p" OUTCAR|awk '{print $2}'`
sig13=`sed -n "$((${line_no}+1))p" OUTCAR|awk '{print $3}'`
sig21=`sed -n "$((${line_no}+2))p" OUTCAR|awk '{print $1}'`
sig22=`sed -n "$((${line_no}+2))p" OUTCAR|awk '{print $2}'`
sig23=`sed -n "$((${line_no}+2))p" OUTCAR|awk '{print $3}'`
sig31=`sed -n "$((${line_no}+3))p" OUTCAR|awk '{print $1}'`
sig32=`sed -n "$((${line_no}+3))p" OUTCAR|awk '{print $2}'`
sig33=`sed -n "$((${line_no}+3))p" OUTCAR|awk '{print $3}'`
total_force=`grep 'FORCE total' OUTCAR|tail -n 1|awk '{print $6}'`
total_stress=`grep 'Stress total' OUTCAR|tail -n 1|awk '{print $6}'`

echo ${fold%?} ${sig11} ${sig12} ${sig13} ${sig21} ${sig22} ${sig23} ${sig31} ${sig32} ${sig33} ${total_force} ${total_stress}>> result.csv
done

results.csv

test-000 -467.71245 24.34138 7.71803 24.34138 -314.11534 -3.45455 7.71803 -3.45455 -371.79956 0.039198 676.007623
$\endgroup$
1
  • $\begingroup$ Thank you very much @Pranav. $\endgroup$
    – Sak
    Commented Nov 11, 2022 at 12:13
5
$\begingroup$

Python3 alternative

You can save this to a .py file. I wrote and tested it under Python 3.10.6:

'''
Seeks a folder and all subfolders it contains for files named 'OUTCAR', extracting data given by regexp pattern. Saves
data to CSV spreadsheet. Assuming this program is saved to a file named outcar2csv.py, is invoked from the linux command
line like this:

python3 outcar2csv.py data_folder_path yourfile

Where data_folder_path is the path to the folder to be recursively searched for files named 'OUTCAR', and yourfile is
the name of the csv file to be created to save the extracted data. This file is created in the same place where
outcar2csv.py is located.
'''

import os
import re
import mmap
import csv
import time
import sys

def process_file(file, tags, pattern):
    '''
    Function seeks regexp pattern in given file, returning data from the last match found as a
    dictionary of tag : data.
    '''
    with open(file, 'r') as f:
        m = mmap.mmap(f.fileno(), 0, access = mmap.ACCESS_READ)
        last_match = pattern.findall(m)[-1]
        return {tag : data.decode() for tag, data in zip(tags, last_match)}

def row_saver(destination_file, tags):
    '''
    Generator that saves data one row a time in a CSV spreadsheet, where each row is
    passed as a dictionary compatible with DictWriter object from Python csv module.
    '''
    with open(destination_file, 'w', newline = '') as csvfile:
        fieldnames = ['folder']
        fieldnames.extend(tags)
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames, dialect='unix')
        writer.writeheader()
        while True:
            writer.writerow((yield))

def main():
    top_folder, destination_file = sys.argv[1:]
    
    t0 = time.time()

    pattern = re.compile(b"""
    \s stress \s matrix \s after \s NEB \s project \s \(eV\)
    \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+)
    \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+)
    \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+) \s+ (-?\d+\.\d+)
    \s+ FORCES: \s max \s atom, \s RMS .*
    \s+ FORCE \s total \s and \s by \s dimension \s+ (-?\d+\.\d+) .*
    \s+ Stress \s total \s and \s by \s dimension \s+ (-?\d+\.\d+) .*
    """, re.VERBOSE)

    tags = ['ii', 'ij', 'ik',
            'ji', 'jj', 'jk',
            'ki', 'kj', 'kk',
            'ForceTotal', 'StressTotal']

    saver = row_saver(destination_file, tags)
    saver.send(None)

    how_many = 0
    for dirpath, dirnames, filenames in os.walk(top_folder):
        for name in filenames:
            if name == 'OUTCAR':
                how_many += 1
                file = os.path.join(dirpath, name)
                row = {'folder': os.path.split(dirpath)[-1]}
                row.update(process_file(file, tags, pattern))
                saver.send(row)

    print(how_many, 'files processed in', round(time.time() - t0, 3), 'seconds.')

if __name__ == '__main__':
    main()

I created some mock data by modification of the OUTCAR file given, for testing it. The script can be used as exemplified below:

user@machine:~$ ls Downloads/mattermodelling
mockdata  outcar2csv.py
user@machine:~$ cd Downloads/mattermodelling
user@machine:~/Downloads/mattermodelling$ ls mockdata/
abc-000  abc-002  abc-004  abc-006  abc-008  abc-010  abc-012  abc-014  abc-016  abc-018
abc-001  abc-003  abc-005  abc-007  abc-009  abc-011  abc-013  abc-015  abc-017  abc-019
user@machine:~/Downloads/mattermodelling$ python3 outcar2csv.py ~/Downloads/mattermodelling/mockdata/ spreadsheet.csv
20 files processed in 0.377 seconds.
user@machine:~/Downloads/mattermodelling$ ls
mockdata  outcar2csv.py  spreadsheet.csv
user@machine:~/Downloads/mattermodelling$ cat spreadsheet.csv 
"folder","ii","ij","ik","ji","jj","jk","ki","kj","kk","ForceTotal","StressTotal"
"abc-007","-862.37367","50.67924","2.91086","76.73190","-612.12661","-4.15208","7.69324","-6.63256","-243.32872","3.217959","318.071499"
"abc-008","-491.44190","89.30117","6.58274","19.24851","-816.34700","-6.96243","1.26080","-2.61234","-743.36987","5.135049","572.092093"
"abc-016","-387.31784","93.66937","6.30587","52.19730","-605.28808","-2.26975","3.81523","-4.72937","-675.31002","5.550989","057.487611"
"abc-014","-809.19049","19.72927","3.31798","06.07626","-868.67673","-9.00880","8.43166","-1.77182","-047.79519","9.173462","527.617069"
"abc-001","-101.10447","65.32618","0.96934","10.07195","-343.88294","-0.12684","1.07403","-6.83529","-223.11711","1.132396","726.216061"
"abc-005","-458.47658","91.04737","9.53896","51.30511","-523.23447","-1.07006","0.63547","-9.55906","-425.59611","0.189480","709.950391"
"abc-009","-009.53367","71.75084","4.66395","11.19572","-466.63143","-2.06748","8.65482","-2.91662","-765.26647","5.337915","378.528833"
"abc-011","-089.14072","33.43878","6.52117","05.15536","-329.99575","-9.70646","9.67780","-6.04529","-632.04250","0.986787","539.275902"
"abc-003","-226.22536","85.58539","5.83980","27.83603","-170.95152","-5.91629","6.88458","-9.64086","-432.43266","9.142282","639.281772"
"abc-004","-426.08425","31.80692","1.78031","24.45488","-018.67193","-7.11674","3.13194","-9.94171","-751.95429","7.610343","967.272264"
"abc-015","-676.72436","21.50989","1.36015","04.46427","-937.33530","-4.04833","8.32037","-1.46997","-212.69520","8.042924","133.224298"
"abc-019","-230.94008","12.00516","7.90733","29.98929","-777.89241","-0.80954","5.72769","-6.88261","-671.07960","6.707047","524.685876"
"abc-000","-688.85233","72.06209","0.19069","50.47013","-548.46840","-7.30824","5.13827","-0.69694","-401.06181","7.933544","001.298579"
"abc-018","-656.39700","23.52044","3.80865","31.69298","-436.68556","-9.85304","2.57257","-0.40098","-006.07856","8.912860","482.683707"
"abc-006","-885.79980","39.81359","5.79319","30.19686","-484.66123","-2.12213","4.80344","-5.07017","-203.31291","0.387975","415.977214"
"abc-010","-533.00892","01.85543","5.31156","92.66351","-685.26155","-2.79869","5.84854","-3.74227","-366.89325","8.122444","167.212737"
"abc-002","-885.29241","08.40123","3.89631","30.16682","-058.96081","-2.43187","0.28793","-9.48565","-317.18443","5.285536","932.580443"
"abc-013","-574.77203","38.14837","6.74880","98.88899","-500.02623","-5.83326","9.87984","-3.23035","-892.95830","3.727332","414.234494"
"abc-012","-663.11922","47.40477","5.91988","23.31203","-854.46445","-2.66353","5.09891","-4.63409","-334.80885","3.028020","121.844300"
"abc-017","-012.63384","88.15725","9.74615","60.62091","-916.43703","-0.54265","5.07418","-9.11345","-438.85608","5.319353","406.512338"
user@machine:~/Downloads/mattermodelling$
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .