stdout howto, bottlenecks, overhead?

[dzhugashvili]

hey, I am writing a password cracker. But I now have been thinking about separating the word generator from the actual encrypt and check function (for the sake of flexibility and convenience). I noticed John the Ripper does some operations like this. I was thinking something like this:

wordgen --stdout | cracker <cracker options>

or

wordgen --stdout | --stdin cracker <cracker options>

or

wordgen --stdout | wordmangle --stdout | cracker <cracker options>

or however it works. Here are my questions:

how does --stdout work (as far as implementation in C++)? How can I output my words? Something with cout? And --stdin? Is that some command with cin?

then speed: would this method slow my code significantly? Would I have to worry about some sort of processing overhead of using this method, or some type of bottleneck from --stdout and --stdin not being able to transfer data between the programs fast enough?

Right now I am generating about 1 billion words per second. I can encrypt and check them at the rate of ~7.6 million/sec for MD5. Would there be a problem with the word generator generating words faster than they can be tested? Or would it know to wait?

speed is essential. I would like the flexibility, but any performance it is significant.

[Salem]

> how does --stdout work (as far as implementation in C++)? How can I output my words? Something with cout?
Exactly.
You parse your command line, and if some argv is "--stdout" then you output to stdout.
Otherwise, one would presume it expects a filename on the command line, and it would output to that file instead.

> then speed: would this method slow my code significantly?
This is entirely down to your OS/Shell.
On real operating systems (those with an IX in the name), pipes are a central feature of the OS and therefore very efficient.

On some dumb systems, pipes get faked by using files - you don't want to go there!

[dzhugashvili]

This is entirely down to your OS/Shell.
On real operating systems (those with an IX in the name), pipes are a central feature of the OS and therefore very efficient.

what about Windows XP/ Vista and Command prompt? Efficient?

actually, I was wondering more about the actual code. I read something like this:

output.cpp:

char wordtopass[]="passme";
int main()
{
     fputs(wordtopass,stdout)
}

print.cpp:

int main()
{
   fgets(wordtopass);
   cout << wordtopass << "\n";
}

so I could call it something like this?

output | print

and it would give this result:

C:\test\output | print
passme
C:\test\

correct? If not, what is the proper method?

[Salem]

> what about Windows XP/ Vista and Command prompt? Efficient?
Dunno - maybe?
You would have to try a long test, write a large number of MB and watch what happens with say process explorer to see if any large temp files get created, and only one of the processes running at once.

As for the technique, you got it.
Though you might want to avoid mixing C and C++ I/O

[thelamb]

When I started out with c++ I did some projects on projecteuler.net. I was always printing my results(in a while loop) untill one day I didn't do this and was blown away by how much faster the calculation was. So if you're going to print a lot you should ask yourself how useful that will be because in my experience it significantly slowed down my algorithm, and I didn't do anything with the output.

[dzhugashvili]

well, I wasn't intending to print to the screen, as i know screen refresh would slow down my program way too much. I was just hoping to pass it to another program. Hopefully this won't slow me down, but I still must write the program and benchmark it.

[thelamb]

Ok sorry I misread your question then, I thought you were printing to the screen.

But indeed, I guess the best way to go is just compare with and without printing.

[dzhugashvili]

alright, I got output working fine:

read_out.cpp

#include <iostream>
int main()
{
     for(int c=0;c<20;c++)
          cout << "6";
return 0;
}

if i just run read_out on cmd line, i get this:

66666666666666666666

or I can pipe the input to a text file:

read_out > readout.txt

and get this:

66666666666666666666

what I want to do is read each string into another program, string by string. I would call it like this:

read_out > read_in

read_in.cpp

#include <iostream>
#include <string>
int main()
{
     string word;

     //while(getline(cin,word))
     while(cin>>word)
          cout << word << "\n";
}

it doesn't work, but do you see what i'm trying to accomplish? I'm trying to get the read_in program to read in words for as many words are supplied on the command line and process them word by word. What I don't know how to do is the 'while' loop which will retrieve the strings from the command line. I tried getline and cin, but neither worked the way I wanted.

Here is what I want the output to be:

read_out > read_in

read_out reads out 66666666666666666666, on 6 at a time
read_in reads in 66666666666666666666, one 6 at a time for as many 6s as read_out supplies. read_in will then process these words. In this example, it prints the strings to the screen with a newline after each string:

read_out > read_in

6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6
6

so does anybody know how to do the while loop to retrieve the strings?

[zalezog]

I doubt whether this will help..:S , your read_in file remains intact, your read_out now is:

#include <iostream>
#include <string>
using namespace std;
int main()
{
     string word;
     // even while((cin >> word)) works
       while(getline(cin, word))
             for (int i = 0; i < word.length(); i++)
                  cout << word[i] << "\n";


}

In the cmd line,

read_out | read_in

gave me the desired output.
After you get each character , you can process it.

[dzhugashvili]

Ah, I see. It was just a matter of formatting - I thought that the code for the while loop was wrong. Thanks :) It works now. Evidently, if you use

while(cin>>word)

it translates some characters as delimiters. Like space, for example. So if you feed out the string "two cats", it will translate it as being fed two strings, "two" and "cats". If you use

while(getline(cin,word))

however, it only translates newlines as delimiters.