Query regarding Data mining and plotting graph...

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site
Reply

Join Date: May 2009
Posts: 11
Reputation: you2 is an unknown quantity at this point 
Solved Threads: 0
you2 you2 is offline Offline
Newbie Poster

Query regarding Data mining and plotting graph...

 
0
  #1
Jul 28th, 2009
I have log file which gives CPU utilization and timestamp value at regular interval.

Logs snippet below:
==========LOGS==============

a:CPU [ 85%]: asdf asd 123 xyz
A: Ts 23086, Netvalue 3286, someothervalues 3456
abc abc
xyz xyz
a:CPU [ 75%]: asdf asd 123 xyz
A: Ts 24088, Netvalue 3266, someothervalues 6576
======End of Logs ===========

What I want to do:
I want to make a graph of CPU load vs timestamp(Ts) value
So I thought if I can create a Dictionary like Dict = {"Timestamp":"CPU Load"} (eg: Dict = {24088:85} for above logs)
then plot a graph somehow using entries in dictionary.

My Queries:
1. How to get a dictionary in above format? I'm weak at data mining...
2. For plotting graph - suggest if this can be a correct approach using dictionary? ..or some other approach would work ...(something like exporting values to MS Excel and creating plot cpu/time.)


Any help appreciated....
Reply With Quote Quick reply to this message  
Join Date: May 2009
Posts: 11
Reputation: you2 is an unknown quantity at this point 
Solved Threads: 0
you2 you2 is offline Offline
Newbie Poster

Re: Query regarding Data mining and plotting graph...

 
0
  #2
Jul 30th, 2009
still hoping for some hint on this...
Reply With Quote Quick reply to this message  
Join Date: Jul 2008
Posts: 1,054
Reputation: jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough 
Solved Threads: 265
Sponsor
jlm699's Avatar
jlm699 jlm699 is offline Offline
Knows where his Towel is

Re: Query regarding Data mining and plotting graph...

 
0
  #3
Jul 30th, 2009
Originally Posted by you2 View Post
Logs snippet below:
==========LOGS==============

a:CPU [ 85%]: asdf asd 123 xyz
A: Ts 23086, Netvalue 3286, someothervalues 3456
abc abc
xyz xyz
a:CPU [ 75%]: asdf asd 123 xyz
A: Ts 24088, Netvalue 3266, someothervalues 6576
======End of Logs ===========
Will your logs always have a: A: on the lines that your relevant information is on?

If so you could do something like this:
  1. fh = open('my_log.inf')
  2. flines = fh.readlines()
  3. fh.close()
  4.  
  5. for line in flines:
  6. if line[:2\] == 'a:'
  7. #get CPU
  8. # etc...
Do you understand what I'm saying? But the important question is, is this the consisent pattern in your source data? Once you hammer that out this will be a simple task.
1. Use Code Tags.
2. Homework? Show Effort.
3. Keep discussions on the forum: no PMs
Reply With Quote Quick reply to this message  
Join Date: May 2009
Posts: 11
Reputation: you2 is an unknown quantity at this point 
Solved Threads: 0
you2 you2 is offline Offline
Newbie Poster

Re: Query regarding Data mining and plotting graph...

 
0
  #4
Jul 31st, 2009
Thanks for the input... I'm able to extract keywords ...but Im stuck at next level:
Now I want to check a condition that
if
"CPU" keyword occurs in line 1 (#I'm able to do this)
then
check for next line for keyword "Ts" (How do I do this ???)
and then only
create an entry in dictionary like D{CPU : timestamp} (#I'm able to do this)


Code Snippet below:
===========
a:CPU [ 75%]: asdf asd 123 xyz
A: Ts 24088,
=========
Reply With Quote Quick reply to this message  
Join Date: Jul 2008
Posts: 1,054
Reputation: jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough 
Solved Threads: 265
Sponsor
jlm699's Avatar
jlm699 jlm699 is offline Offline
Knows where his Towel is

Re: Query regarding Data mining and plotting graph...

 
0
  #5
Jul 31st, 2009
If you read your lines into a list via readlines() you can simply use indexing... something like this:
  1. my_index = 0
  2. while my_index < len(my_lines):
  3. current_line = my_lines[index]
  4. if 'CPU' in current_line:
  5. #get CPU data
  6. index += 1
  7. next_line = my_lines[index]
  8. # Get Ts data
  9. # Save data
  10. index += 1
That's an extremely simple solution that you could use. There's many ways to do it, but I hope this at least gets you on a path to answering your query
1. Use Code Tags.
2. Homework? Show Effort.
3. Keep discussions on the forum: no PMs
Reply With Quote Quick reply to this message  
Join Date: May 2009
Posts: 11
Reputation: you2 is an unknown quantity at this point 
Solved Threads: 0
you2 you2 is offline Offline
Newbie Poster

Re: Query regarding Data mining and plotting graph...

 
0
  #6
Aug 4th, 2009
Thanks a lot for useful tip... it worked for me...
Python really rocks...
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC