stdout howto, bottlenecks, overhead?

Please support our C++ advertiser: Intel Parallel Studio Home
Thread Solved

Join Date: Jun 2009
Posts: 33
Reputation: dzhugashvili is an unknown quantity at this point 
Solved Threads: 0
dzhugashvili dzhugashvili is offline Offline
Light Poster

stdout howto, bottlenecks, overhead?

 
0
  #1
Jul 26th, 2009
hey, I am writing a password cracker. But I now have been thinking about separating the word generator from the actual encrypt and check function (for the sake of flexibility and convenience). I noticed John the Ripper does some operations like this. I was thinking something like this:

  1. wordgen --stdout | cracker <cracker options>
or
  1. wordgen --stdout | --stdin cracker <cracker options>
or
  1. wordgen --stdout | wordmangle --stdout | cracker <cracker options>

or however it works. Here are my questions:

how does --stdout work (as far as implementation in C++)? How can I output my words? Something with cout? And --stdin? Is that some command with cin?

then speed: would this method slow my code significantly? Would I have to worry about some sort of processing overhead of using this method, or some type of bottleneck from --stdout and --stdin not being able to transfer data between the programs fast enough?

Right now I am generating about 1 billion words per second. I can encrypt and check them at the rate of ~7.6 million/sec for MD5. Would there be a problem with the word generator generating words faster than they can be tested? Or would it know to wait?

speed is essential. I would like the flexibility, but any performance it is significant.
Reply With Quote Quick reply to this message  
Join Date: Dec 2005
Posts: 5,850
Reputation: Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute 
Solved Threads: 749
Team Colleague
Salem's Avatar
Salem Salem is offline Offline
Void main'ers are DOOMed

Re: stdout howto, bottlenecks, overhead?

 
0
  #2
Jul 26th, 2009
> how does --stdout work (as far as implementation in C++)? How can I output my words? Something with cout?
Exactly.
You parse your command line, and if some argv[i] is "--stdout" then you output to stdout.
Otherwise, one would presume it expects a filename on the command line, and it would output to that file instead.

> then speed: would this method slow my code significantly?
This is entirely down to your OS/Shell.
On real operating systems (those with an IX in the name), pipes are a central feature of the OS and therefore very efficient.

On some dumb systems, pipes get faked by using files - you don't want to go there!
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 33
Reputation: dzhugashvili is an unknown quantity at this point 
Solved Threads: 0
dzhugashvili dzhugashvili is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #3
Jul 27th, 2009
This is entirely down to your OS/Shell.
On real operating systems (those with an IX in the name), pipes are a central feature of the OS and therefore very efficient.
what about Windows XP/ Vista and Command prompt? Efficient?

actually, I was wondering more about the actual code. I read something like this:

output.cpp:

  1. char wordtopass[]="passme";
  2. int main()
  3. {
  4. fputs(wordtopass,stdout)
  5. }

print.cpp:

  1. int main()
  2. {
  3. fgets(wordtopass);
  4. cout << wordtopass << "\n";
  5. }

so I could call it something like this?

output | print

and it would give this result:

C:\test\output | print
passme
C:\test\

correct? If not, what is the proper method?
Reply With Quote Quick reply to this message  
Join Date: Dec 2005
Posts: 5,850
Reputation: Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute Salem has a reputation beyond repute 
Solved Threads: 749
Team Colleague
Salem's Avatar
Salem Salem is offline Offline
Void main'ers are DOOMed

Re: stdout howto, bottlenecks, overhead?

 
0
  #4
Jul 28th, 2009
> what about Windows XP/ Vista and Command prompt? Efficient?
Dunno - maybe?
You would have to try a long test, write a large number of MB and watch what happens with say process explorer to see if any large temp files get created, and only one of the processes running at once.

As for the technique, you got it.
Though you might want to avoid mixing C and C++ I/O
Reply With Quote Quick reply to this message  
Join Date: Aug 2008
Posts: 37
Reputation: thelamb is on a distinguished road 
Solved Threads: 7
thelamb thelamb is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #5
Jul 29th, 2009
When I started out with c++ I did some projects on projecteuler.net. I was always printing my results(in a while loop) untill one day I didn't do this and was blown away by how much faster the calculation was. So if you're going to print a lot you should ask yourself how useful that will be because in my experience it significantly slowed down my algorithm, and I didn't do anything with the output.
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 33
Reputation: dzhugashvili is an unknown quantity at this point 
Solved Threads: 0
dzhugashvili dzhugashvili is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #6
Jul 29th, 2009
well, I wasn't intending to print to the screen, as i know screen refresh would slow down my program way too much. I was just hoping to pass it to another program. Hopefully this won't slow me down, but I still must write the program and benchmark it.
Reply With Quote Quick reply to this message  
Join Date: Aug 2008
Posts: 37
Reputation: thelamb is on a distinguished road 
Solved Threads: 7
thelamb thelamb is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #7
Jul 30th, 2009
Ok sorry I misread your question then, I thought you were printing to the screen.

But indeed, I guess the best way to go is just compare with and without printing.
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 33
Reputation: dzhugashvili is an unknown quantity at this point 
Solved Threads: 0
dzhugashvili dzhugashvili is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #8
Aug 14th, 2009
alright, I got output working fine:

read_out.cpp

  1. #include <iostream>
  2. int main()
  3. {
  4. for(int c=0;c<20;c++)
  5. cout << "6";
  6. return 0;
  7. }

if i just run read_out on cmd line, i get this:

66666666666666666666

or I can pipe the input to a text file:

read_out > readout.txt

and get this:

66666666666666666666

what I want to do is read each string into another program, string by string. I would call it like this:

read_out > read_in

read_in.cpp

  1. #include <iostream>
  2. #include <string>
  3. int main()
  4. {
  5. string word;
  6.  
  7. //while(getline(cin,word))
  8. while(cin>>word)
  9. cout << word << "\n";
  10. }

it doesn't work, but do you see what i'm trying to accomplish? I'm trying to get the read_in program to read in words for as many words are supplied on the command line and process them word by word. What I don't know how to do is the 'while' loop which will retrieve the strings from the command line. I tried getline and cin, but neither worked the way I wanted.

Here is what I want the output to be:

read_out > read_in

read_out reads out 66666666666666666666, on 6 at a time
read_in reads in 66666666666666666666, one 6 at a time for as many 6s as read_out supplies. read_in will then process these words. In this example, it prints the strings to the screen with a newline after each string:

read_out > read_in

6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6

so does anybody know how to do the while loop to retrieve the strings?
Reply With Quote Quick reply to this message  
Join Date: Oct 2008
Posts: 44
Reputation: zalezog is an unknown quantity at this point 
Solved Threads: 11
zalezog zalezog is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #9
Aug 14th, 2009
I doubt whether this will help.. , your read_in file remains intact, your read_out now is:
  1. #include <iostream>
  2. #include <string>
  3. using namespace std;
  4. int main()
  5. {
  6. string word;
  7. // even while((cin >> word)) works
  8. while(getline(cin, word))
  9. for (int i = 0; i < word.length(); i++)
  10. cout << word[i] << "\n";
  11.  
  12.  
  13. }
In the cmd line,
  1. read_out | read_in
gave me the desired output.
After you get each character , you can process it.
Reply With Quote Quick reply to this message  
Join Date: Jun 2009
Posts: 33
Reputation: dzhugashvili is an unknown quantity at this point 
Solved Threads: 0
dzhugashvili dzhugashvili is offline Offline
Light Poster

Re: stdout howto, bottlenecks, overhead?

 
0
  #10
Aug 15th, 2009
Ah, I see. It was just a matter of formatting - I thought that the code for the while loop was wrong. Thanks It works now. Evidently, if you use

while(cin>>word)

it translates some characters as delimiters. Like space, for example. So if you feed out the string "two cats", it will translate it as being fed two strings, "two" and "cats". If you use

while(getline(cin,word))

however, it only translates newlines as delimiters.
Reply With Quote Quick reply to this message  
Reply

This thread has been marked solved.
Perhaps start a new thread instead?
Message:



Similar Threads
Other Threads in the C++ Forum
Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC