944,008 Members | Top Members by Rank

Ad:
  • C++ Discussion Thread
  • Marked Solved
  • Views: 1368
  • C++ RSS
You are currently viewing page 1 of this multi-page discussion thread
Jul 30th, 2008
1

Problem with contest program

Expand Post »
I'm writing a program for a contest (it's a demo, not the actual thing - so I'm not cheating by asking for help - and my question isn't directly related to the algorithm they want anyway). To submit a program, you send them the .cpp file, and they execute it on their server (automatically) for a number of test trials (around 10), and show you the results. This problem has 11 trials. For the first five trials, the program runs fine. However, the sixth run of the program fails, and it throws std::bad_alloc (I'm guessing its caused by vector::push_back - the only other things I'm using from std are file streams and strings). Then, trial 7 works fine. Trials 8 and 9 failed, same reason. Trial 10's runtime was too long (it was stopped at around 1.5 secs, shouldn't go over 1 second), and trial 11 had the bad_alloc again.

I wanted to see exactly where the bad_alloc was occurring, so I copied the test input data for trial 6 into the test input file on my machine, and ran the program. I didn't get bad_alloc, but the program didn't end (for the time I ran it, maybe 10 seconds - I'm going to try leaving it for longer while outputting debug info).

Anyway - this is my assumption of what's going on, tell me if I'm right:
On the virtual machine on the competition's servers, the program is only given the amount of memory that they think it needs to execute. So, my program is memory inefficient, and on their machine, it runs out of memory. However, on my machine, the memory isn't limited, so it keeps using it up, and so instead of getting bad_alloc, the program just runs for a long time. Is this a correct assumption?
Similar Threads
Reputation Points: 77
Solved Threads: 40
Posting Pro in Training
CoolGamer48 is offline Offline
401 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

I would surmise you're using a brute force solution when perhaps a dynamic solution exists.

This is quite common on online contests such as topcoder etc.
Featured Poster
Reputation Points: 1536
Solved Threads: 431
Posting Expert
iamthwee is offline Offline
5,865 posts
since Aug 2005
Jul 30th, 2008
0

Re: Problem with contest program

I'm writing a program for a contest (it's a demo, not the actual thing - so I'm not cheating by asking for help - and my question isn't directly related to the algorithm they want anyway). To submit a program, you send them the .cpp file, and they execute it on their server (automatically) for a number of test trials (around 10), and show you the results. This problem has 11 trials. For the first five trials, the program runs fine. However, the sixth run of the program fails, and it throws std::bad_alloc (I'm guessing its caused by vector::push_back - the only other things I'm using from std are file streams and strings). Then, trial 7 works fine. Trials 8 and 9 failed, same reason. Trial 10's runtime was too long (it was stopped at around 1.5 secs, shouldn't go over 1 second), and trial 11 had the bad_alloc again.

I wanted to see exactly where the bad_alloc was occurring, so I copied the test input data for trial 6 into the test input file on my machine, and ran the program. I didn't get bad_alloc, but the program didn't end (for the time I ran it, maybe 10 seconds - I'm going to try leaving it for longer while outputting debug info).

Anyway - this is my assumption of what's going on, tell me if I'm right:
On the virtual machine on the competition's servers, the program is only given the amount of memory that they think it needs to execute. So, my program is memory inefficient, and on their machine, it runs out of memory. However, on my machine, the memory isn't limited, so it keeps using it up, and so instead of getting bad_alloc, the program just runs for a long time. Is this a correct assumption?

Sounds like a reasonable assumption. You could easily have a program that ran out of memory in one environment with limited memory resources, but not run out in an environment with more resources. push_back is going to result in a bad_alloc error when you run out of memory. I got about 134,000,000 allocations in the program below before hitting the bad_alloc exception, but it could easily vary with different systems.

C++ Syntax (Toggle Plain Text)
  1. #include <vector>
  2. #include <string>
  3. #include <iostream>
  4. using namespace std;
  5.  
  6. int main ()
  7. {
  8. vector <int> v;
  9. cout << "Absolute maximum allocations = " << v.max_size() << endl;
  10.  
  11. int counter = 0;
  12. bool done = false;
  13. while(!done)
  14. {
  15. try
  16. {
  17. v.push_back(counter);
  18. counter++;
  19. }
  20. catch (std::bad_alloc)
  21. {
  22. done = true;
  23. cout << "Number of allocations = " << counter << endl;
  24. cout << "Bad allocation" << endl;
  25. }
  26. }
  27.  
  28. cout << "Done with program" << endl;
  29. return 0;
  30. }
Featured Poster
Reputation Points: 2614
Solved Threads: 687
Posting Expert
VernonDozier is offline Offline
5,375 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

Click to Expand / Collapse  Quote originally posted by iamthwee ...
I would surmise you're using a brute force solution when perhaps a dynamic solution exists.

This is quite common on online contests such as topcoder etc.
My solution may be a brute force - I'm not entirety sure. I would have preferred to think of the algorithm on my own - but this is the basic structure of the problem/program, is it a brute force?

The problem:
You're given x number of lowercase characters and a password length. You need to find all the valid passwords you can construct from the given letters.
valid passwords:
- have at least one vowel
- have at least two consonants
- have characters in alphabetical order, ie: abc is ok, bac, or cab, is not

You need to then alphabetize the results you get and print up to 25000 of them. Here's how I go about it (I'm away from home so I don't have the code with me).

Find all the possible combos given the letters.

Weed out the incorrect ones

Alphabetize what's left.

Output up to 25000

Is this a brute force method? If so, what would be a possible dynamic method (if you can sort of hint at the answer or put me on the right track without giving me the answer, that'd be nice).
Reputation Points: 77
Solved Threads: 40
Posting Pro in Training
CoolGamer48 is offline Offline
401 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

My solution may be a brute force - I'm not entirety sure. I would have preferred to think of the algorithm on my own - but this is the basic structure of the problem/program, is it a brute force?

The problem:
You're given x number of lowercase characters and a password length. You need to find all the valid passwords you can construct from the given letters.
valid passwords:
- have at least one vowel
- have at least two consonants
- have characters in alphabetical order, ie: abc is ok, bac, or cab, is not

You need to then alphabetize the results you get and print up to 25000 of them. Here's how I go about it (I'm away from home so I don't have the code with me).

Find all the possible combos given the letters.

Weed out the incorrect ones

Alphabetize what's left.

Output up to 25000

Is this a brute force method? If so, what would be a possible dynamic method (if you can sort of hint at the answer or put me on the right track without giving me the answer, that'd be nice).
I'd say that was overly brute-force. One, assume that by "combos" you mean "combinations" rather than "permutations". If you are calculating all permutations, that's definitely too brute-force since you know in advance these are going to be alphabetized. So if you have a vector and you are adding both "ab" and "ba" onto that vector, definitely don't. At the very minimum, only add "ab", but you really shouldn't add "ab" at all. Test for legality BEFORE pushing a potential solution onto the vector. There's no need to add something to the vector that you are going to take off later. If you only add actual solutions to your vector, that may solve your bad_alloc right there.

A good algorithm, I think, will actually work in such a way that the solutions are actually added to the vector already in alphabetical order, so there is no need to sort. I would also NOT generate all possible combinations (let alone permutations). Some you know right away are illegal (i.e. all consonants), so don't even generate them. Start with a vowel and two consonants, and THEN generate all combinations of the remaining letters. Those will all be legal combinations (you'll need to sort the letter into alphabetical order before adding it to the vector, but it'll be a legal combination of letters).
Featured Poster
Reputation Points: 2614
Solved Threads: 687
Posting Expert
VernonDozier is offline Offline
5,375 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

I would definitely agree with the above. There are a lot of shortcuts you can apply to drastically cut out the number of calculations you need to do.

Have another think about it.
Featured Poster
Reputation Points: 1536
Solved Threads: 431
Posting Expert
iamthwee is offline Offline
5,865 posts
since Aug 2005
Jul 30th, 2008
0

Re: Problem with contest program

I got home recently, and I'll try to make the program more memory efficient, but I just let the program run for 40 mins with one of the failed sets of test data, and it didn't get past the portion where it finds all the possible combinations you can have, so I'm thinking there may be some other issue, but I'll try to make it less of a brute-force and see what happens.

Here's the code, if anyone cares to read it:
C++ Syntax (Toggle Plain Text)
  1. #include <fstream>
  2. #include <iostream>
  3. #include <string>
  4. #include <vector>
  5. using namespace std;
  6.  
  7. int password_length;
  8. int num_chars;
  9. vector<string> possible_combos;
  10. char* chars;
  11.  
  12. void FindAll(string current,vector<int> char_ids_used)
  13. {
  14. if(current.length() == password_length)
  15. {
  16. possible_combos.push_back(current);
  17. return;
  18. }
  19.  
  20. for(int i = 0; i < num_chars;i++)
  21. {
  22. bool used = false;
  23. for(int j = 0;j < char_ids_used.size();j++)
  24. {
  25. if(i == char_ids_used[j])
  26. used = true;
  27. }
  28. if(!used)
  29. {
  30. vector<int> my_used = char_ids_used;
  31. my_used.push_back(i);
  32. FindAll(current + chars[i],my_used);
  33. }
  34. }
  35. }
  36.  
  37. bool IsVowel(char c)
  38. {
  39. switch(c)
  40. {
  41. case 97:
  42. case 101:
  43. case 105:
  44. case 111:
  45. case 117:
  46. return true;
  47. }
  48.  
  49. return false;
  50. }
  51.  
  52. bool HasVowel(string str)
  53. {
  54. int length = str.length();
  55. for(int i = 0;i < length;i++)
  56. {
  57. if(IsVowel(str[i]))
  58. return true;
  59. }
  60.  
  61. return false;
  62. }
  63.  
  64. bool HasTwoConsonants(string str)
  65. {
  66. int numConsonants = 0;
  67.  
  68. int length = str.length();
  69. for(int i = 0;i < length;i++)
  70. {
  71. if(!IsVowel(str[i]))
  72. numConsonants++;
  73. }
  74.  
  75. if(numConsonants >= 2)
  76. return true;
  77.  
  78. return false;
  79. }
  80.  
  81. bool InAlphaOrder(string str)
  82. {
  83. for(int i = 1;i < str.length();i++)
  84. {
  85. char prev = str[i-1];
  86. char current = str[i];
  87. if(int(prev) > int(current))
  88. return false;
  89. }
  90.  
  91. return true;
  92. }
  93.  
  94. bool InAlphaOrder(string first,string second)
  95. {
  96. int length = first.length();
  97. for(int i = 0;i < length;i++)
  98. {
  99. if(int(first[i]) < int(second[i]))
  100. return true;
  101. else if(int(first[i]) > int(second[i]))
  102. return false;
  103. }
  104.  
  105. return true;//strings are the same
  106. }
  107.  
  108. int main()
  109. {
  110. ofstream debug("passwd.debug");
  111.  
  112. debug << "Program start\n";
  113. cout << "Program start\n";
  114.  
  115. debug << "Reading data from passwd.in\n";
  116. cout << "Reading data from passwd.in\n";
  117. ifstream fin("passwd.in");
  118. fin >> password_length >> num_chars;
  119. chars = new char[num_chars];
  120. for(int i = 0;i < num_chars;i++)
  121. fin >> chars[i];
  122.  
  123. fin.close();
  124. debug << "Done reading\n";
  125. cout << "Done reading\n";
  126.  
  127. debug << "Finding possible combos\n";
  128. cout << "Finding possible combos\n";
  129. for(int i = 0;i < num_chars;i++)
  130. {
  131. vector<int> used_chars;
  132. used_chars.push_back(i);
  133. string current;
  134. current = chars[i];
  135. FindAll(current,used_chars);
  136. }
  137. debug << "Done finding\n";
  138. cout << "Done finding\n";
  139.  
  140. debug << "Validating results\n";
  141. cout << "Validating results\n";
  142. int num_combos = possible_combos.size();
  143. vector<string> valid_combos;
  144. for(int i = 0;i < num_combos;i++)//added i < 25001
  145. {
  146. if(!InAlphaOrder(possible_combos[i]))
  147. continue;
  148. if(!HasVowel(possible_combos[i]))
  149. continue;
  150. if(!HasTwoConsonants(possible_combos[i]))
  151. continue;
  152.  
  153. valid_combos.push_back(possible_combos[i]);
  154. }
  155. debug << "Done validating\n";
  156. cout << "Done validating\n";
  157.  
  158. possible_combos.clear();//all valid results have been copied to the valid_combos vector
  159.  
  160. debug << "Alphabetizing results\n";
  161. cout << "Alphabetizing results\n";
  162. for(int i = 1;i < valid_combos.size() && i <= 25000;i++)
  163. {
  164. if(!InAlphaOrder(valid_combos[i-1],valid_combos[i]))
  165. {
  166. string temp = valid_combos[i];
  167. valid_combos[i] = valid_combos[i-1];
  168. valid_combos[i-1] = temp;
  169.  
  170. for(int j = i-1;j > 0;j--)
  171. {
  172. valid_combos[j-1];
  173. valid_combos[j];
  174. if(InAlphaOrder(valid_combos[j-1],valid_combos[j]))
  175. break;
  176. else
  177. {
  178. temp = valid_combos[j];
  179. valid_combos[j] = valid_combos[j-1];
  180. valid_combos[j-1] = temp;
  181. }
  182. }
  183. }
  184. }
  185. debug << "Done alphabetizing\n";
  186. cout << "Done alphabetizing\n";
  187.  
  188. debug << "Outputting results\n";
  189. cout << "Outputting results\n";
  190. ofstream fout("passwd.out");
  191. int size = valid_combos.size();
  192. if(size > 25000)
  193. size = 25000;
  194. for(int i = 0;i < size;i++)
  195. {
  196. fout << valid_combos[i] << "\n";
  197. }
  198.  
  199. debug << "Done outputting\n";
  200. cout << "Done outputting\n";
  201.  
  202. debug << "Program end\n";
  203. cout << "Program end\n";
  204.  
  205. fout.close();
  206.  
  207. return 0;
  208. }
Reputation Points: 77
Solved Threads: 40
Posting Pro in Training
CoolGamer48 is offline Offline
401 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

Alright, I added the validations before I store the combo in a vector, and that did get rid of the bad_alloc. However, now I'm having issues with runtime. All of the tests that failed before still all fail, it's just that now they fail due their runtime passing one second. I'll try to make the program more efficient as far as runtime, but I still think that there's something up with the brute-force logic I have in place. I can't imagine that one of the tests would take 40+ mins. while others would take fractions of seconds. Though perhaps that 's because I'm not used to these types of problems. I'll look into both improving my logic and seeing if there are issues with it as it is currently implemented.
Reputation Points: 77
Solved Threads: 40
Posting Pro in Training
CoolGamer48 is offline Offline
401 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

Question: Can you have more than one of the same character or must all characters be unique? If you can have multiple characters (i.e. aaert), that's a harder program, I would imagine. Or must all characters be unique (i.e. aert)?
Featured Poster
Reputation Points: 2614
Solved Threads: 687
Posting Expert
VernonDozier is offline Offline
5,375 posts
since Jan 2008
Jul 30th, 2008
0

Re: Problem with contest program

Question: Can you have more than one of the same character or must all characters be unique? If you can have multiple characters (i.e. aaert), that's a harder program, I would imagine. Or must all characters be unique (i.e. aert)?
I thought of that too (while I was out), and I thought of a possible solution. I just checked, and yes, the chars given in the input must be unique. It gave me an idea:

Right after you get the input chars, alphabetize them. Then, go to the first char, and cycle through all the possible combos, but stop if you find that any of the chars you are adding to the string make the string not be in alphabetical order. That will make each output be in alphabetical order, and the outputs themselves will be organized alphabetically.

That logic might be slightly flawed, I just spurred it out. As I'm implementing it I might see any flaws in it, but I think I'm on the right track.

Was this what you were thinking of?
Reputation Points: 77
Solved Threads: 40
Posting Pro in Training
CoolGamer48 is offline Offline
401 posts
since Jan 2008

This thread is solved

Either the thread starter or a moderator has marked this thread as solved. You can most likely trust the responses and answers given. There is most likely no reason for any further responses to be posted here. If you have a related question, please start a new thread in this forum instead.

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in C++ Forum Timeline: Code Speed Question
Next Thread in C++ Forum Timeline: Trying to solve a past thread from another user.





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC