Out of memory exception on a list

[ogsirus]

Hi Guys

I ran into a bit of a problem with some code I am writing. I am trying to store all possible combinations of a set of numbers which are current being stored in List. Problem is my code run into an out of memory exception, heres my code:

public void GetAllPossibleCombos(Dictionary<int, decimal> numbers)
        {
            var allNumbers = numbers.Keys.ToArray();

            var possibleLines = new List<Line>();

            Combinations<int> normalCombo = new Combinations<int>(allNumbers, 5);

            foreach(IList<int> n in normalCombo)
            {
                var line = new Line();

                line.Number1 = n[0];
                line.Number2 = n[1];
                line.Number3 = n[2];
                line.Number4 = n[3];
                line.Number5 = n[4];

                possibleLines.Add(line);
                }
            }

        }

The dictionary has 50 numbers, the Combinations method was something I found on CSharpCorner. Dont know if anyone has any ideas. Cheers.

[tinstaafl]

You seem to be copying a lot of your data. Plus you have no code for Combinations and Line, which may do more copying, but without the code it's hard to say for sure. Just from a quick glance allNumbers is only used to pass to Combinations, passing the Keys property itself might work better. Also if you have the code for Combinations, modifying it to build the list instead of the collection it is returning, should work better as well.

[overwraith]

I can't find the code for Combinations, but you appear to have a lot of data coming out of that class. If I was you I would review the code, and see if perhaps I could convert combinations, or something else into an IEnumerable, aka a state machine. Use Yield Return, so you only have one of the lists or something in memory at any given time. I can't see the code though, so I don't know if it is feasible, especially if there are other dependencies. Could you explain what Combinations does, because the name honestly isn't that descriptive. You are providing it a dictionary, but when I think"combinations" intuitivily I would think of something like, hey we got one number, and another number, and I want all combinations between the two, but you are providing it a collection, so I have no idea what it does. Almost literally any place you can turn a collection or file or output into an IEnumerable is a good application of it due to the fact that typically you can keep your memory footprint small. Would a stream help? I don't know. Remember, computers still have limited memory, and even though you have oodles of it to play with these days, Microsoft will eventually smack you down if it thinks your process is too memory hungry.

[ogsirus]

public Combinations(IList<T> values, int lowerIndex) {
            Initialize(values, lowerIndex, GenerateOption.WithoutRepetition);
        }

private void Initialize(IList<T> values, int lowerIndex, GenerateOption type) {
            myMetaCollectionType = type;
            myLowerIndex = lowerIndex;
            myValues = new List<T>();
            myValues.AddRange(values);
            List<bool> myMap = new List<bool>();
            if(type == GenerateOption.WithoutRepetition) {
                for(int i = 0; i < myValues.Count; ++i) {
                    if(i >= myValues.Count - myLowerIndex) {
                        myMap.Add(false);
                    }
                    else {
                        myMap.Add(true);
                    }
                }
            }
            else {
                for(int i = 0; i < values.Count - 1; ++i) {
                    myMap.Add(true);
                }
                for(int i = 0; i < myLowerIndex; ++i) {
                    myMap.Add(false);
                }
            }
            myPermutations = new Permutations<bool>(myMap);
        }

public Permutations(IList<T> values) {
            Initialize(values, GenerateOption.WithoutRepetition, null);
        }

private void Initialize(IList<T> values, GenerateOption type, IComparer<T> comparer) {
            myMetaCollectionType = type;
            myValues = new List<T>(values.Count);
            myValues.AddRange(values);
            myLexicographicOrders = new int[values.Count];
            if(type == GenerateOption.WithRepetition) {
                for(int i = 0; i < myLexicographicOrders.Length; ++i) {
                    myLexicographicOrders[i] = i;
                }
            }
            else {
                if(comparer == null) {
                    comparer = new SelfComparer<T>();
                }
                myValues.Sort(comparer);
                int j = 1;
                if(myLexicographicOrders.Length > 0) {
                    myLexicographicOrders[0] = j;
                }
                for(int i = 1; i < myLexicographicOrders.Length; ++i) {
                    if(comparer.Compare(myValues[i - 1], myValues[i]) != 0) {
                        ++j;
                    }
                    myLexicographicOrders[i] = j;
                }
            }
            myCount = GetCount();
        }

private long GetCount() {
    int runCount = 1;
    List<int> divisors = new List<int>();
    List<int> numerators = new List<int>();
    for(int i = 1; i < myLexicographicOrders.Length; ++i) {
    numerators.AddRange(SmallPrimeUtility.Factor(i + 1));
    if(myLexicographicOrders[i] == myLexicographicOrders[i - 1]) {
    ++runCount;
    }
    else {
    for(int f = 2; f <= runCount; ++f) {
    divisors.AddRange(SmallPrimeUtility.Factor(f));
    }
    runCount = 1;
    }
    }
    for(int f = 2; f <= runCount; ++f) {
    divisors.AddRange(SmallPrimeUtility.Factor(f));
    }
    return SmallPrimeUtility.EvaluatePrimeFactors(SmallPrimeUtility.DividePrimeFactors(numerators, divisors));
}

[ogsirus]

Sorry the code was from codeproject http://www.codeproject.com/Articles/26050/Permutations-Combinations-and-Variations-using-C-G

[tinstaafl]

I've been trying to duplicate your error but even on a vista machine with only 2 GB of memory getting all the 5 number combinations of 50 numbers doesn't eat up all the memory. This leads me to believe that there may be something else in your project that is eating up your memory leaving you too short to run this code.

[ogsirus]

I think your right. My program is doing a shed load of other things. What I have decided to do is store the values in several text files and load them up when I actually need then. This means I wouldn't have to run the combination method again thus saving time. I think I would end up storing the values in a database once I got one set up. Which might have a good impact on performance too. We'll see.

Cheers