I'm not sure... I copied the code you posted, and I started enlarging the array until I stopped working... I suppose it is a real array. I don't know how to do a dynamic array
"until I stopped working" = "until it stopped working"
#1 What is the system RAM size? WinXP can use only a bit more than 2Gigs, (it's only 32 bit)but Win7 and Linux (64 bit versions of each), can use a LOT more memory. For speed, that would definitely be the way to go. Ubuntu 64 bit is one I've used. The live CD you d/l makes it relatively painless, sets up a multi-boot loader by default, and has a very active help forum as well.
#2 The alternative is to somehow break up the problem, and work with "chunks" of say, 200,000, each or 4 chunks. (any even number of equal sized chunks is best). Say you want to sort all the records, to quickly remove any duplicates (or do some work).
You can make a multi-tier record handler:
File 15 / \ / \ / \ file 13 file 14 / \ / \ file 9 file 10 file 11 file 12 / \ / \ / \ / \ file 1 file 2 file 3 file 4 file 5 file 6 file 7 file 8
I used logic like this, (but more "tiers"), to handle 50,000 records when I was using a 16 bit compiler. It had only 64KB of memory the compiler could access. This can handle merging (with or w/o sorting), the smaller chunks of memory, using files on modern HD's, (with their big cache's), much faster than I expected.
From experience, using the latest i5 or i7 Intel cpu's (especially overclocked moderately), will far outperform any other cpu we have tested at the Folding@Home distributed project. AMD cpu's especially, can't begin to keep up. GPU's are computational beasts, but require very small "chunks" of data be fed to them, and somewhat specialized programming. The Cell (Sony MP3),cpu from IBM/Toshiba/Sony, is also a computational wonder, but also needs very specialized programming. (which very few programmers are familiar with)
Fun project, glad you brought it up. ;)
Edited by Adak: n/a
This is a static array declaration:
The memory comes from a block the compiler sets aside called the "stack". it's convenient, but 1) lasts only until the function that called it goes out of scope, (for main() that's forever), and 2) the total memory available is about 2 MB, max.
This is an example of a dynamic array:
int *array; array = malloc(50 * sizeof(int));
These arrays can be can be multi-dimensional (same as a static array in that regard), and can take all the rest of the "heap" portion of memory, set aside by the compiler. Typically, that's MUCH larger, depending on the system you're on (the hardware), and the memory model the compiler is using, and the size the compiler is coded for. A 32 bit compiler will not be able to access nearly as much memory as a 64 bit compiler, even on the same system, with lots of memory.
Dynamic data remains until either the program ends, or you explicitly free() it. It never goes out of scope.
You need to include stdlib.h for malloc(). While static arrays are set to 0 or "", by most compilers, malloc()'d arrays are not, generally. Use calloc() if you need to set the memory to zero or NULL.
There are special considerations for dealing with multi-dimensional dynamic arrays, and arrays of structs with dynamic arrays inside them -- like yours. Those inner dynamic arrays (as Narue showed you), must be allocated in a separate line of code.
Edited by Adak: n/a
i am not sure but may be it is a answer. Pls close the file after your output statement.