Member Avatar for erezz

Hi,
I have a log file contain 28 lines,
1-I need to grep the line with the size next to that i need to grep the name from the line
Example:
Size element
1. 906K ykms2
2. 1.2M ykps1
2-if one or more of the lines are missing I need to have that printed
Example:
There are 26 file in the directory the missing files are:
1.ykas4
2.hfns1
3-I need to find out if the size of the files is smaller than the size seen below,
Example:
1.The size of the file from ykms2 is less that 906K
2.The size of the file from ykps1 is less that 1.2M

At the end I need all of the info to be printed to a log file.

Thanks in advance,
Erez

Here are the 28 lines:

-rw-r--r-- 1 101 102 906K Jul 5 02:30 2012187014201-20120705-13907-MS_Rel_17.sp2_1.88-ykms2-1fac1fae.tar.gz
-rw-r--r-- 1 101 102 1.2M Jul 5 02:30 2012187013702-20120705-20272-PS_Rel_17.sp2_1.88-ykps1-1fac19ae.tar.gz
-rw-r--r-- 1 101 102 946K Jul 5 02:30 2012187013201-20120705-11904-MS_Rel_17.sp2_1.88-ykms1-1fac1eae.tar.gz
-rw-r--r-- 1 101 102 25M Jul 5 02:30 2012187011702-20120705-18520-NS_Rel_17.sp1_1.316-ykns1-1fac14ae.tar.gz
-rw-r--r-- 1 101 102 156M Jul 5 02:30 2012187011601-20120705-11608-AS_Rel_17.sp2_1.88-ykas4-1fac05ae.tar.gz
-rw-r--r-- 1 101 102 159M Jul 5 02:30 2012187011202-20120705-25902-AS_Rel_17.sp2_1.88-ykas3-1fac04ae.tar.gz
-rw-r--r-- 1 101 102 160M Jul 5 02:30 2012187010801-20120705-642-AS_Rel_17.sp2_1.88-ykas2-1fac03ae.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 02:30 2012187010201-20120705-12683-XSP_Rel_17.sp2_1.88-ykxspprov1-1fac3cae.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 02:30 2012187014201-20120705-26189-XSP_Rel_17.sp2_1.88-ykxsp2-1fac83ae.tar.gz
-rw-r--r-- 1 101 102 26M Jul 5 02:40 2012187012301-20120705-10767-NS_Rel_17.sp1_1.316-hfns1-1fac14e2.tar.gz
-rw-r--r-- 1 101 102 173M Jul 5 02:40 2012187010402-20120705-15021-AS_Rel_17.sp2_1.88-ykas1-1fac02ae.tar.gz
-rw-r--r-- 1 101 102 158M Jul 5 02:41 2012187012301-20120705-24809-AS_Rel_17.sp2_1.88-hfas2-1fac03e2.tar.gz
-rw-r--r-- 1 101 102 170M Jul 5 02:41 2012187011901-20120705-27946-AS_Rel_17.sp2_1.88-hfas1-1fac02e2.tar.gz
-rw-r--r-- 1 101 102 156M Jul 5 02:41 2012187013201-20120705-4493-AS_Rel_17.sp2_1.88-hfas4-1fac05e2.tar.gz
-rw-r--r-- 1 101 102 157M Jul 5 02:41 2012187012702-20120705-20072-AS_Rel_17.sp2_1.88-hfas3-1fac04e2.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 02:42 2012187013201-20120705-9659-XSP_Rel_17.sp2_1.88-hfxsp1-1fac82e2.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 02:42 2012187011202-20120705-29856-XSP_Rel_17.sp2_1.88-hfxspprov1-1fac3ce2.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 02:45 2012187012202-20120705-4906-XSP_Rel_17.sp2_1.88-ykxsp1-1fac82ae.tar.gz
-rw-r--r-- 1 101 102 900K Jul 5 02:50 2012187013601-20120705-26512-MS_Rel_17.sp2_1.88-hfms1-1fac1ee2.tar.gz
-rw-r--r-- 1 101 102 52M Jul 5 03:00 2012187013401-20120705-23502-EMS_Rel_17.sp1_1.316-ykems1-1fac4bae.tar.gz
-rw-r--r-- 1 101 102 52M Jul 5 03:19 2012187014301-20120705-19418-EMS_Rel_17.sp1_1.316-hfems1-1fac4be2.tar.gz
-rw-r--r-- 1 101 102 746K Jul 5 03:19 2012187014601-20120705-32723-MS_Rel_17.sp2_1.88-hfms2-1fac1fe2.tar.gz
-rw-r--r-- 1 101 102 18M Jul 5 03:19 2012187014701-20120705-31649-CDS_Rel_17.0_1.458-ykcds1-1fac41ae.tar.gz
-rw-r--r-- 1 101 102 13M Jul 5 03:19 2012187014902-20120705-27766-CDS_Rel_17.0_1.458-hfcds1-1fac41e2.tar.gz
-rw-r--r-- 1 101 102 892K Jul 5 03:19 2012187015201-20120705-21743-MS_Rel_17.sp2_1.88-ykms3-1fac20ae.tar.gz
-rw-r--r-- 1 101 102 303M Jul 5 03:21 2012187015201-20120705-406-XSP_Rel_17.sp2_1.88-hfxsp2-1fac83e2.tar.gz
-rw-r--r-- 1 101 102 800K Jul 5 03:21 2012187015602-20120705-6037-MS_Rel_17.sp2_1.88-hfms3-1fac20e2.tar.gz
-rw-r--r-- 1 101 102 1.2M Jul 5 03:21 2012187015701-20120705-5567-PS_Rel_17.sp2_1.88-hfps1-1fac19e2.tar.gz

  • 2 Contributors
  • 4 Replies
  • 230 Views
  • 5 Months Discussion Span
  • Latest Post Latest Post by erezz

Dani AI

Generated

Short summary and recommended approach (for and follow-up): the safest way is to parse the saved listing to pull the recorded, human-readable size and the short token (examples like ykms2), then re-check the actual directory to (a) detect which expected short names are missing, and (b) flag files whose real size is smaller than the recorded size. As pointed out, querying the filesystem is more reliable than depending on a human-readable dump — but if you only have the dump, the script below parses it, converts sizes to bytes, and compares against the live files. The script uses the second-to-last dash-separated field from the filename to get the short name (this matches your samples; change that logic if your naming varies).

Usage: run with Python 3:
python3 compare_listing.py listing.txt /path/to/dir out.log

#!/usr/bin/env python3
import os,sys,re

def human_to_bytes(s):
    m = re.match(r'^\s*(\d+(?:\.\d+)?)([KMGT]?)(?:B)?\s*$', s, re.I)
    if not m: return int(float(s))
    n, u = float(m.group(1)), m.group(2).upper()
    mult = {'':1, 'K':1024, 'M':1024**2, 'G':1024**3, 'T':1024**4}
    return int(n * mult.get(u,1))

def parse_listing(path):
    E = {}
    for ln in open(path):
        cols = ln.split()
        if len(cols) < 9: continue
        size_str, fname = cols[4], cols[-1]
        short = fname.split('-')[-2] if '-' in fname else fname
        E.setdefault(short, []).append((fname, human_to_bytes(size_str)))
    return E

def scan_dir(d):
    A = {}
    for f in os.listdir(d):
        p = os.path.join(d,f)
        if not os.path.isfile(p): continue
        short = f.split('-')[-2] if '-' in f else f
        A.setdefault(short, []).append((f, os.path.getsize(p)))
    return A

if __name__ == '__main__':
    if len(sys.argv) < 3:
        print('Usage: compare_listing.py LISTING_FILE DIR [OUTLOG]'); sys.exit(1)
    listing, d = sys.argv[1], sys.argv[2]
    out = sys.argv[3] if len(sys.argv)>3 else 'compare.log'
    E, A = parse_listing(listing), scan_dir(d)

    missing = sorted(k for k in E if k not in A)
    extra   = sorted(k for k in A if k not in E)
    smaller = []
    for k in E:
        if k in A:
            exp = max(x[1] for x in E[k])
            act = max(x[1] for x in A[k])
            if act < exp: smaller.append((k, act, exp))

    with open(out,'w') as h:
        h.write('Missing (%d):\n' % len(missing))
        for m in missing: h.write(m + '\n')
        h.write('\nExtra (%d):\n' % len(extra))
        for e in extra: h.write(e + '\n')
        h.write('\nSmaller than recorded (%d):\n' % len(smaller))
        for s,a,ee in smaller:
            h.write(f'{s} : actual={a} bytes  expected={ee} bytes\n')

Quick notes and troubleshooting:

  • If filenames in your dump contain spaces, use cols[8:] joined back to get the full filename instead of cols[-1].
  • The code assumes K/M/G are 1024-based. If your listing used 1000-based SI units, change the multipliers.
  • If the short token isn’t always the second-to-last dash field, replace that extraction with a regex tuned to your filename pattern (for example, capture the alpha+digit token before the trailing checksum and .tar.gz).
  • Run on Python 3 and inspect the generated out.log for the missing/extra/smaller sections.

Recommended Answers

All 4 Replies

Member Avatar for Watael

hi,

are you sure you need all these informations in the file list?
wouldn't you use stat to keep only useful informations, instead of ls -nh?

Member Avatar for erezz

Hi,
stat will not help because I don't need the info of the files in the directory,
I need to grep the info from a text file contain the print out of ls-ltrh from a directory.

Member Avatar for Watael

again, do you really need all these informations in the file?
if you can't use stat, maybe ls -srth could be enough...

after what you use a while loop to read the listing file by redirecting its content to the loop.

Member Avatar for erezz

Solved,
Thanks.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.