Member Avatar for Member #1089176

Hi,
I am very new to programming but am hoping it will be able to speed a few things up for me. However I am struggling to work out how to write a Python3 script that does the following and am wondering if you could help me out.

I have a list of text files that are the output of a computational chemistry code.
At some point in each, (after a varying number of iteration steps) there will be the lines:

**** Optimisation achieved ****

Final energy = -348.67740315 eV
Final Gnorm = 0.00037832

After some data, the file goes on to have the lines:

Total number of defects = 1

Total charge on defect = -4.00

Defect centre is at 1.0000 0.0000 0.0000 Frac

The file goes on and at some point later on (again after varying number of iteration steps) there will be the lines:

**** Optimisation achieved ****

Final defect energy = 64.41932012
Final defect Gnorm = 0.00000283

N.B. If optimisation is not achieved the energies are still prited but are not of any interest to me. **
**N.B. The numbers here are taken from an example file (and are not the only numeric values within the file)

I know how to open/read each file within the directory. I also know how to make and write to a new file. My problem, however is this:

How do I find and print the lines 'Final Energy =...' and 'Charge on defect...' and 'Final defect energy=...' but only provided optimisation is achieved?

Hoping you can help.
Thanks

  • 2 Contributors
  • 5 Replies
  • 341 Views
  • 2 Days Discussion Span
  • Latest Post Latest Post by Gribouillis

Dani AI

Generated

Short summary and approach: the file contains two important optimisation blocks and a defect-summary section. A reliable solution is a small stateful text scan rather than a blind global search: detect the "Optimisation achieved" marker, then only within a short window after that marker capture the nearby Final energy or Final defect energy lines. Capture Total charge on defect when seen, but only report it for a file that has an optimisation block (or tie it to the nearest optimisation block depending on the required semantics). was correct to suggest regular expressions; that plus a simple state variable gives a robust Python 3 solution.

Suggested workflow (no code shown):

  • Open files in text mode with an explicit encoding and iterate line-by-line (streaming keeps memory use low).
  • When a line matches an optimisation marker (case-insensitive), set an "in-optimised-block" flag and a short lookahead counter (e.g. 5–12 lines).
  • While the counter > 0, test each following line for the three target patterns and record any numeric values; decrement the counter each step. Reset the flag when the window ends or a new optimisation marker appears.
  • After the full file is scanned, decide whether to output values (for example: output only if the corresponding value was captured under an optimisation block; use "NA" otherwise). For multiple matches, prefer the last captured value per file unless older values matter.

Useful regex patterns (use with re.IGNORECASE and convert captures to float):

optim_re      = r'^\*+\s*optimis\w*\s+achieved\s*\*+$'
energy_re     = r'^Final(?:\s+defect)?\s+energy\s*=\s*([+-]?\d+(?:\.\d+)?)'
charge_re     = r'^Total\s+charge\s+on\s+defect\s*=\s*([+-]?\d+(?:\.\d+)?)'

Troubleshooting tips: watch Python 2 vs Python 3 differences (no ifilter, open files in text mode), normalize whitespace and units (strip trailing eV), handle both British/American spelling variants by matching the root optimis, and decide whether to take first or last occurrence when files contain multiple optimisation blocks. Output to CSV with columns (filename, bulk_energy, charge, defect_energy, status) for easy post-processing.

Recommended Answers

All 5 Replies

Member Avatar for Gribouillis

I would start with regular expressions and itertools

import itertools
import re
wanted = (
     'Final energy =',
     'Total charge on defect =',
     'Final defect energy =',
     '**** Optimisation achieved ****',
)
regex = '^(?:{0})'.format('|'.join(re.escape(s) for s in wanted))
regex = re.compile(regex)
with open('filename.txt', 'rb') as lines:
    lines = itertools.ifilter(regex.match, lines)
    for line in lines:
        print(line)
Edited by Gribouillis
Member Avatar for Member #1089176

hi , apologies but I don't understand some of the syntax on the code you have shown. Would it be possible to talk me through it? Thanks

Member Avatar for Gribouillis

Which parts of the code don't you understand ?

Member Avatar for Member #1089176

line 9

Member Avatar for Gribouillis

This line uses the string format() method to build a regular expression. For example

>>> import re
>>> wanted = ('cat', 'dog', 'parrot')
>>> regex = '^(?:{0})'.format('|'.join(re.escape(s) for s in wanted))
>>> regex
'^(?:cat|dog|parrot)'

This regex is able to tell if a string starts with any of the words cat, dog and parrot. Read this for a tutorial on regular expressions and this for the format() method.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.