Shell Scripting

Shell Scripting RSS

email log parsing

•

Join Date: Dec 2007

Posts: 3

Reputation: jjamd64 is an unknown quantity at this point

Solved Threads: 0

jjamd64

Offline

Newbie Poster

email log parsing

Dec 20th, 2007

I have an email log that needs some parsing. Can anyone help me to use sed or awk to parse this log and return only the domain names, listing them only once and ignoring case?

Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@cimasys.net
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw
Dec 19 14:59:09 host53a pop3d: LOGIN, user=user@cimasys.net, ip=[::ffff:192.168.1.1]
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@design.com
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw
Dec 19 14:59:09 host53a pop3d: LOGIN, user=user@design.com, ip=[::ffff:192.168.1.1]
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@cimasys.net
Dec 19 14:59:09 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw
Dec 19 14:59:09 host53a pop3d: LOGIN, user=user@cimasys.net, ip=[::ffff:192.168.1.1]
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@altaven.net
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw
Dec 19 14:59:10 host53a pop3d: LOGIN, user=user@altaven.net, ip=[::ffff:192.168.1.1]
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=AUTH
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@seniorben.com
Dec 19 14:59:10 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw
Dec 19 14:59:10 host53a pop3d: LOGIN, user=user@seniorben.com, ip=[::ffff:192.168.1.1]
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=AUTH
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=USER
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], command=PASS
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], username=user@seniorben.com
Dec 19 14:59:11 host53a pop3d: LOGIN: DEBUG: ip=[::ffff:192.168.1.1], password=pw

desired output:
design.com
cimasys.net
altaven.net
seniorben.com

•

Join Date: Oct 2007

Posts: 399

Reputation: eggi will become famous soon enough

Solved Threads: 47

eggi

Offline

Posting Whiz

Re: email log parsing

Dec 20th, 2007

Hey There,

Probably this would do the trick - not elegant but should work:

 Help with Code Tags 
 Shell Scripting Syntax (Toggle Plain Text) 
#!/bin/ksh
 
while read line
do
        awk -F"@" '{if ( $0 ~ /@/ ) print $2}'|awk -F"," '{print $1}' >>file.tmp
done < INPUTFILE
sort -u file.tmp >>output
rm file.tmp
#!/bin/ksh

while read line
do
        awk -F"@" '{if ( $0 ~ /@/ ) print $2}'|awk -F"," '{print $1}' >>file.tmp
done < INPUTFILE
sort -u file.tmp >>output
rm file.tmp

Best wishes

,Mike

Linux and Unix Tips, Tricks and Individual Advice - The Linux and Unix Menagerie!
------------------------------------------------------------------------
The greatest viral marketing idea of all time, get your copy of this Free Report now!

•

Join Date: Feb 2007

Posts: 54

Reputation: radoulov is an unknown quantity at this point

Solved Threads: 5

radoulov

Offline

Junior Poster in Training

Re: email log parsing

Dec 21st, 2007

 Help with Code Tags 
 Shell Scripting Syntax (Toggle Plain Text) 
awk 'NF>1&&!x[$2]++{print $2}' FS="username=[^@]*@" logfile
awk 'NF>1&&!x[$2]++{print $2}' FS="username=[^@]*@" logfile