Problem with contest program

Please support our C++ advertiser: Programming Forums - DaniWeb Sister Site
Thread Solved

Join Date: Jan 2008
Posts: 401
Reputation: CoolGamer48 is on a distinguished road 
Solved Threads: 40
CoolGamer48's Avatar
CoolGamer48 CoolGamer48 is offline Offline
Posting Pro in Training

Problem with contest program

 
1
  #1
Jul 30th, 2008
I'm writing a program for a contest (it's a demo, not the actual thing - so I'm not cheating by asking for help - and my question isn't directly related to the algorithm they want anyway). To submit a program, you send them the .cpp file, and they execute it on their server (automatically) for a number of test trials (around 10), and show you the results. This problem has 11 trials. For the first five trials, the program runs fine. However, the sixth run of the program fails, and it throws std::bad_alloc (I'm guessing its caused by vector::push_back - the only other things I'm using from std are file streams and strings). Then, trial 7 works fine. Trials 8 and 9 failed, same reason. Trial 10's runtime was too long (it was stopped at around 1.5 secs, shouldn't go over 1 second), and trial 11 had the bad_alloc again.

I wanted to see exactly where the bad_alloc was occurring, so I copied the test input data for trial 6 into the test input file on my machine, and ran the program. I didn't get bad_alloc, but the program didn't end (for the time I ran it, maybe 10 seconds - I'm going to try leaving it for longer while outputting debug info).

Anyway - this is my assumption of what's going on, tell me if I'm right:
On the virtual machine on the competition's servers, the program is only given the amount of memory that they think it needs to execute. So, my program is memory inefficient, and on their machine, it runs out of memory. However, on my machine, the memory isn't limited, so it keeps using it up, and so instead of getting bad_alloc, the program just runs for a long time. Is this a correct assumption?
I'm a student. If my statements seem too absolute, feel free to coat them with "In my opinion..." or "I believe...".
Reply With Quote Quick reply to this message  
Join Date: Aug 2005
Posts: 5,275
Reputation: iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold 
Solved Threads: 378
Featured Poster
iamthwee's Avatar
iamthwee iamthwee is offline Offline
Posting Expert

Re: Problem with contest program

 
0
  #2
Jul 30th, 2008
I would surmise you're using a brute force solution when perhaps a dynamic solution exists.

This is quite common on online contests such as topcoder etc.
*Voted best profile in the world*
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 3,844
Reputation: VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute 
Solved Threads: 503
Featured Poster
VernonDozier VernonDozier is offline Offline
Senior Poster

Re: Problem with contest program

 
0
  #3
Jul 30th, 2008
Originally Posted by CoolGamer48 View Post
I'm writing a program for a contest (it's a demo, not the actual thing - so I'm not cheating by asking for help - and my question isn't directly related to the algorithm they want anyway). To submit a program, you send them the .cpp file, and they execute it on their server (automatically) for a number of test trials (around 10), and show you the results. This problem has 11 trials. For the first five trials, the program runs fine. However, the sixth run of the program fails, and it throws std::bad_alloc (I'm guessing its caused by vector::push_back - the only other things I'm using from std are file streams and strings). Then, trial 7 works fine. Trials 8 and 9 failed, same reason. Trial 10's runtime was too long (it was stopped at around 1.5 secs, shouldn't go over 1 second), and trial 11 had the bad_alloc again.

I wanted to see exactly where the bad_alloc was occurring, so I copied the test input data for trial 6 into the test input file on my machine, and ran the program. I didn't get bad_alloc, but the program didn't end (for the time I ran it, maybe 10 seconds - I'm going to try leaving it for longer while outputting debug info).

Anyway - this is my assumption of what's going on, tell me if I'm right:
On the virtual machine on the competition's servers, the program is only given the amount of memory that they think it needs to execute. So, my program is memory inefficient, and on their machine, it runs out of memory. However, on my machine, the memory isn't limited, so it keeps using it up, and so instead of getting bad_alloc, the program just runs for a long time. Is this a correct assumption?

Sounds like a reasonable assumption. You could easily have a program that ran out of memory in one environment with limited memory resources, but not run out in an environment with more resources. push_back is going to result in a bad_alloc error when you run out of memory. I got about 134,000,000 allocations in the program below before hitting the bad_alloc exception, but it could easily vary with different systems.

  1. #include <vector>
  2. #include <string>
  3. #include <iostream>
  4. using namespace std;
  5.  
  6. int main ()
  7. {
  8. vector <int> v;
  9. cout << "Absolute maximum allocations = " << v.max_size() << endl;
  10.  
  11. int counter = 0;
  12. bool done = false;
  13. while(!done)
  14. {
  15. try
  16. {
  17. v.push_back(counter);
  18. counter++;
  19. }
  20. catch (std::bad_alloc)
  21. {
  22. done = true;
  23. cout << "Number of allocations = " << counter << endl;
  24. cout << "Bad allocation" << endl;
  25. }
  26. }
  27.  
  28. cout << "Done with program" << endl;
  29. return 0;
  30. }
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 401
Reputation: CoolGamer48 is on a distinguished road 
Solved Threads: 40
CoolGamer48's Avatar
CoolGamer48 CoolGamer48 is offline Offline
Posting Pro in Training

Re: Problem with contest program

 
0
  #4
Jul 30th, 2008
Originally Posted by iamthwee View Post
I would surmise you're using a brute force solution when perhaps a dynamic solution exists.

This is quite common on online contests such as topcoder etc.
My solution may be a brute force - I'm not entirety sure. I would have preferred to think of the algorithm on my own - but this is the basic structure of the problem/program, is it a brute force?

The problem:
You're given x number of lowercase characters and a password length. You need to find all the valid passwords you can construct from the given letters.
valid passwords:
- have at least one vowel
- have at least two consonants
- have characters in alphabetical order, ie: abc is ok, bac, or cab, is not

You need to then alphabetize the results you get and print up to 25000 of them. Here's how I go about it (I'm away from home so I don't have the code with me).

Find all the possible combos given the letters.

Weed out the incorrect ones

Alphabetize what's left.

Output up to 25000

Is this a brute force method? If so, what would be a possible dynamic method (if you can sort of hint at the answer or put me on the right track without giving me the answer, that'd be nice).
I'm a student. If my statements seem too absolute, feel free to coat them with "In my opinion..." or "I believe...".
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 3,844
Reputation: VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute 
Solved Threads: 503
Featured Poster
VernonDozier VernonDozier is offline Offline
Senior Poster

Re: Problem with contest program

 
0
  #5
Jul 30th, 2008
Originally Posted by CoolGamer48 View Post
My solution may be a brute force - I'm not entirety sure. I would have preferred to think of the algorithm on my own - but this is the basic structure of the problem/program, is it a brute force?

The problem:
You're given x number of lowercase characters and a password length. You need to find all the valid passwords you can construct from the given letters.
valid passwords:
- have at least one vowel
- have at least two consonants
- have characters in alphabetical order, ie: abc is ok, bac, or cab, is not

You need to then alphabetize the results you get and print up to 25000 of them. Here's how I go about it (I'm away from home so I don't have the code with me).

Find all the possible combos given the letters.

Weed out the incorrect ones

Alphabetize what's left.

Output up to 25000

Is this a brute force method? If so, what would be a possible dynamic method (if you can sort of hint at the answer or put me on the right track without giving me the answer, that'd be nice).
I'd say that was overly brute-force. One, assume that by "combos" you mean "combinations" rather than "permutations". If you are calculating all permutations, that's definitely too brute-force since you know in advance these are going to be alphabetized. So if you have a vector and you are adding both "ab" and "ba" onto that vector, definitely don't. At the very minimum, only add "ab", but you really shouldn't add "ab" at all. Test for legality BEFORE pushing a potential solution onto the vector. There's no need to add something to the vector that you are going to take off later. If you only add actual solutions to your vector, that may solve your bad_alloc right there.

A good algorithm, I think, will actually work in such a way that the solutions are actually added to the vector already in alphabetical order, so there is no need to sort. I would also NOT generate all possible combinations (let alone permutations). Some you know right away are illegal (i.e. all consonants), so don't even generate them. Start with a vowel and two consonants, and THEN generate all combinations of the remaining letters. Those will all be legal combinations (you'll need to sort the letter into alphabetical order before adding it to the vector, but it'll be a legal combination of letters).
Reply With Quote Quick reply to this message  
Join Date: Aug 2005
Posts: 5,275
Reputation: iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold iamthwee is a splendid one to behold 
Solved Threads: 378
Featured Poster
iamthwee's Avatar
iamthwee iamthwee is offline Offline
Posting Expert

Re: Problem with contest program

 
0
  #6
Jul 30th, 2008
I would definitely agree with the above. There are a lot of shortcuts you can apply to drastically cut out the number of calculations you need to do.

Have another think about it.
*Voted best profile in the world*
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 401
Reputation: CoolGamer48 is on a distinguished road 
Solved Threads: 40
CoolGamer48's Avatar
CoolGamer48 CoolGamer48 is offline Offline
Posting Pro in Training

Re: Problem with contest program

 
0
  #7
Jul 30th, 2008
I got home recently, and I'll try to make the program more memory efficient, but I just let the program run for 40 mins with one of the failed sets of test data, and it didn't get past the portion where it finds all the possible combinations you can have, so I'm thinking there may be some other issue, but I'll try to make it less of a brute-force and see what happens.

Here's the code, if anyone cares to read it:
  1. #include <fstream>
  2. #include <iostream>
  3. #include <string>
  4. #include <vector>
  5. using namespace std;
  6.  
  7. int password_length;
  8. int num_chars;
  9. vector<string> possible_combos;
  10. char* chars;
  11.  
  12. void FindAll(string current,vector<int> char_ids_used)
  13. {
  14. if(current.length() == password_length)
  15. {
  16. possible_combos.push_back(current);
  17. return;
  18. }
  19.  
  20. for(int i = 0; i < num_chars;i++)
  21. {
  22. bool used = false;
  23. for(int j = 0;j < char_ids_used.size();j++)
  24. {
  25. if(i == char_ids_used[j])
  26. used = true;
  27. }
  28. if(!used)
  29. {
  30. vector<int> my_used = char_ids_used;
  31. my_used.push_back(i);
  32. FindAll(current + chars[i],my_used);
  33. }
  34. }
  35. }
  36.  
  37. bool IsVowel(char c)
  38. {
  39. switch(c)
  40. {
  41. case 97:
  42. case 101:
  43. case 105:
  44. case 111:
  45. case 117:
  46. return true;
  47. }
  48.  
  49. return false;
  50. }
  51.  
  52. bool HasVowel(string str)
  53. {
  54. int length = str.length();
  55. for(int i = 0;i < length;i++)
  56. {
  57. if(IsVowel(str[i]))
  58. return true;
  59. }
  60.  
  61. return false;
  62. }
  63.  
  64. bool HasTwoConsonants(string str)
  65. {
  66. int numConsonants = 0;
  67.  
  68. int length = str.length();
  69. for(int i = 0;i < length;i++)
  70. {
  71. if(!IsVowel(str[i]))
  72. numConsonants++;
  73. }
  74.  
  75. if(numConsonants >= 2)
  76. return true;
  77.  
  78. return false;
  79. }
  80.  
  81. bool InAlphaOrder(string str)
  82. {
  83. for(int i = 1;i < str.length();i++)
  84. {
  85. char prev = str[i-1];
  86. char current = str[i];
  87. if(int(prev) > int(current))
  88. return false;
  89. }
  90.  
  91. return true;
  92. }
  93.  
  94. bool InAlphaOrder(string first,string second)
  95. {
  96. int length = first.length();
  97. for(int i = 0;i < length;i++)
  98. {
  99. if(int(first[i]) < int(second[i]))
  100. return true;
  101. else if(int(first[i]) > int(second[i]))
  102. return false;
  103. }
  104.  
  105. return true;//strings are the same
  106. }
  107.  
  108. int main()
  109. {
  110. ofstream debug("passwd.debug");
  111.  
  112. debug << "Program start\n";
  113. cout << "Program start\n";
  114.  
  115. debug << "Reading data from passwd.in\n";
  116. cout << "Reading data from passwd.in\n";
  117. ifstream fin("passwd.in");
  118. fin >> password_length >> num_chars;
  119. chars = new char[num_chars];
  120. for(int i = 0;i < num_chars;i++)
  121. fin >> chars[i];
  122.  
  123. fin.close();
  124. debug << "Done reading\n";
  125. cout << "Done reading\n";
  126.  
  127. debug << "Finding possible combos\n";
  128. cout << "Finding possible combos\n";
  129. for(int i = 0;i < num_chars;i++)
  130. {
  131. vector<int> used_chars;
  132. used_chars.push_back(i);
  133. string current;
  134. current = chars[i];
  135. FindAll(current,used_chars);
  136. }
  137. debug << "Done finding\n";
  138. cout << "Done finding\n";
  139.  
  140. debug << "Validating results\n";
  141. cout << "Validating results\n";
  142. int num_combos = possible_combos.size();
  143. vector<string> valid_combos;
  144. for(int i = 0;i < num_combos;i++)//added i < 25001
  145. {
  146. if(!InAlphaOrder(possible_combos[i]))
  147. continue;
  148. if(!HasVowel(possible_combos[i]))
  149. continue;
  150. if(!HasTwoConsonants(possible_combos[i]))
  151. continue;
  152.  
  153. valid_combos.push_back(possible_combos[i]);
  154. }
  155. debug << "Done validating\n";
  156. cout << "Done validating\n";
  157.  
  158. possible_combos.clear();//all valid results have been copied to the valid_combos vector
  159.  
  160. debug << "Alphabetizing results\n";
  161. cout << "Alphabetizing results\n";
  162. for(int i = 1;i < valid_combos.size() && i <= 25000;i++)
  163. {
  164. if(!InAlphaOrder(valid_combos[i-1],valid_combos[i]))
  165. {
  166. string temp = valid_combos[i];
  167. valid_combos[i] = valid_combos[i-1];
  168. valid_combos[i-1] = temp;
  169.  
  170. for(int j = i-1;j > 0;j--)
  171. {
  172. valid_combos[j-1];
  173. valid_combos[j];
  174. if(InAlphaOrder(valid_combos[j-1],valid_combos[j]))
  175. break;
  176. else
  177. {
  178. temp = valid_combos[j];
  179. valid_combos[j] = valid_combos[j-1];
  180. valid_combos[j-1] = temp;
  181. }
  182. }
  183. }
  184. }
  185. debug << "Done alphabetizing\n";
  186. cout << "Done alphabetizing\n";
  187.  
  188. debug << "Outputting results\n";
  189. cout << "Outputting results\n";
  190. ofstream fout("passwd.out");
  191. int size = valid_combos.size();
  192. if(size > 25000)
  193. size = 25000;
  194. for(int i = 0;i < size;i++)
  195. {
  196. fout << valid_combos[i] << "\n";
  197. }
  198.  
  199. debug << "Done outputting\n";
  200. cout << "Done outputting\n";
  201.  
  202. debug << "Program end\n";
  203. cout << "Program end\n";
  204.  
  205. fout.close();
  206.  
  207. return 0;
  208. }
I'm a student. If my statements seem too absolute, feel free to coat them with "In my opinion..." or "I believe...".
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 401
Reputation: CoolGamer48 is on a distinguished road 
Solved Threads: 40
CoolGamer48's Avatar
CoolGamer48 CoolGamer48 is offline Offline
Posting Pro in Training

Re: Problem with contest program

 
0
  #8
Jul 30th, 2008
Alright, I added the validations before I store the combo in a vector, and that did get rid of the bad_alloc. However, now I'm having issues with runtime. All of the tests that failed before still all fail, it's just that now they fail due their runtime passing one second. I'll try to make the program more efficient as far as runtime, but I still think that there's something up with the brute-force logic I have in place. I can't imagine that one of the tests would take 40+ mins. while others would take fractions of seconds. Though perhaps that 's because I'm not used to these types of problems. I'll look into both improving my logic and seeing if there are issues with it as it is currently implemented.
I'm a student. If my statements seem too absolute, feel free to coat them with "In my opinion..." or "I believe...".
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 3,844
Reputation: VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute VernonDozier has a reputation beyond repute 
Solved Threads: 503
Featured Poster
VernonDozier VernonDozier is offline Offline
Senior Poster

Re: Problem with contest program

 
0
  #9
Jul 30th, 2008
Question: Can you have more than one of the same character or must all characters be unique? If you can have multiple characters (i.e. aaert), that's a harder program, I would imagine. Or must all characters be unique (i.e. aert)?
Reply With Quote Quick reply to this message  
Join Date: Jan 2008
Posts: 401
Reputation: CoolGamer48 is on a distinguished road 
Solved Threads: 40
CoolGamer48's Avatar
CoolGamer48 CoolGamer48 is offline Offline
Posting Pro in Training

Re: Problem with contest program

 
0
  #10
Jul 30th, 2008
Originally Posted by VernonDozier View Post
Question: Can you have more than one of the same character or must all characters be unique? If you can have multiple characters (i.e. aaert), that's a harder program, I would imagine. Or must all characters be unique (i.e. aert)?
I thought of that too (while I was out), and I thought of a possible solution. I just checked, and yes, the chars given in the input must be unique. It gave me an idea:

Right after you get the input chars, alphabetize them. Then, go to the first char, and cycle through all the possible combos, but stop if you find that any of the chars you are adding to the string make the string not be in alphabetical order. That will make each output be in alphabetical order, and the outputs themselves will be organized alphabetically.

That logic might be slightly flawed, I just spurred it out. As I'm implementing it I might see any flaws in it, but I think I'm on the right track.

Was this what you were thinking of?
I'm a student. If my statements seem too absolute, feel free to coat them with "In my opinion..." or "I believe...".
Reply With Quote Quick reply to this message  
Reply

This thread has been marked solved.
Perhaps start a new thread instead?
Message:




Views: 1111 | Replies: 10
Thread Tools Search this Thread



Tag cloud for C++
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC