Problem with String Builder(I think)

[word.smith]

Hello, I have wrote a program that reads in words from a text file stored on my hard drive and converts each word into a corresponding code depending on the combination of letters of each word. Now I have two text files that I want to code, one is the first 228 words of a dictionary, the second is full dictionary of 79769 words.

Once I have converted each word to its related code I then want to remove any zero's and replace any successive numbers with one digit for example, 06558 would become 658. I have wrote the code below and it works fine for the file that is 228 words long, but when I try and read in the full dictionary and change my array sizes, I run the program and nothing happens, the complier just presents me with a black screen whereas before it printed out each word and its corresponding code.

At first I thought ok maybe it takes a while to process 80000 words but I have left the program running for half an hour with nothing but a flashing cursor.

Is there a limit on the amount of string processing a string builder can do? Does anyone know where i'm going wrong in my code?

string word = string.Empty, strOut = string.Empty;
            StreamReader sr = new StreamReader("c:/dictionary.txt");
            string line;
            string[] dictionary = new string[228];//79769 or 228
            int q = 0;
            while ((line = sr.ReadLine()) != null)
            {

                dictionary[q] = line;
                q++;
            }
            StringBuilder builder = new StringBuilder();
            foreach (string s in dictionary)
            {
                word = s;
                foreach (char c in word)
                {
                    switch (c)
                    {
                        case 'r':
                            strOut = strOut + "6";
                            break;

                        case 'l':
                            strOut = strOut + "4";
                            break;

                        case 'm':
                        case 'n':
                            strOut = strOut + "5";
                            break;


                        case 'd':
                        case 't':
                            strOut = strOut + "3";
                            break;

                        case 'b':
                        case 'f':
                        case 'p':
                        case 'v':
                            strOut = strOut + "1";
                            break;

                        case 'c':
                        case 'g':
                        case 'j':
                        case 'k':
                        case 'q':
                        case 's':
                        case 'x':
                        case 'z':
                            strOut = strOut + "2";
                            break;

                        case 'a':
                        case 'e':
                        case 'i':
                        case 'o':
                        case 'u':
                        case 'h':
                        case 'y':
                        case 'w':
                            strOut = strOut + "0";
                            break;
                    }
                }

                builder.Append(strOut);
                builder.AppendLine();
                builder.Replace("0", "");
                builder.Replace("11", "1");
                builder.Replace("22", "2");
                builder.Replace("33", "3");
                builder.Replace("44", "4");
                builder.Replace("55", "5");
                builder.Replace("66", "6");
                strOut = String.Empty;
            }

[Geekitygeek]

Its quite possible that you are exceeding the physical memory available to your application and/or the stack.

If you want to see if it is still running, try adding a reference to System.Diagnostics and put the following inside your for loop after you increment q:

if (q % 500 == 0)
                Debug.WriteLine("Finished " + q.ToString() + " Entries");

You will be able to see inthe VS output window whether it is still running or not.

[apegram]

I would step through the code and make sure it is doing what you think it should be doing. I can run your code on a smaller file, and a StringBuilder should be able to handle the number of "words" you're trying to throw at it, it can hold over 2 billion characters.

Consider this code:

StringBuilder builder = new StringBuilder();

        for (int i = 0; i < 80000; i++)
        {
            if (i % 1000 == 0)
                Console.WriteLine(i); // to show progress

            builder.Append("reallyreallybigword");
        }

        Console.Write(builder.Length);
        Console.Read();

It executes lightning quick.

[Geekitygeek]

i ran some tests on this last night and it takes ~623Mb to allocate an empty array of 79769 strings...and filling each with 26 characters pushes it up to ~743Mb.
So if your machine has limited available memory its possible you are running out of it. I would defnitely agree with apegram on stepping through your code if you havent already done so...generally that should be your first port of call when something isnt working. Also, check the output window in VS to see if there are any FirstChanceExcpetions being thrown behind the scenes.

[word.smith]

Thanks for the feedback guys.

I have added the code and observed the output. 'q' is incrementing correctly, it appears there is a problem with the builder.replace lines as nothing executes at this point, and when I comment out all of the builder.replace lines it executes fine.
I've also noticed that all of the builder.replace lines have to be excluded for the program to run, it freezes if even one of these lines is included in the code.

Is it a problem alluding to the vast amount of characters it has to deal with?

EDIT: I've also tried reducing the number of words being read in, the limit is about 1500 words. Excess of 1500 causes it to freeze.

[apegram]

Yes, that is entirely likely to be the problem. I glanced right past the fact you were using the Replace function of builder.

The calls to builder.Replace() should not be necessary, at least not with the main instance of StringBuilder. Since you are appending a line between each word, the replace function would only essentially be applicable to the current value of strOut, as everything else would have been fixed beforehand and the new lines would render matches carrying over from prior iterations obsolete. So you're better off fixing the contents of strOut before you append them to the builder rather than appending and then fixing the builder.