Memory not bearing a huge ARRAY

[RahulV]

Hi,
I've a really huge recordset having lacks of records. For some reasons I've to take all the records into an array (using loop) to do some processing and then put it into a grid.
Now since the recordset is huge the memory is not able to handle the array and giving out errors.
When I set the upper bound of the array manually to almost 18000 to 19000 It works fine. But when the real upper bound of the array is set, then it gives out error and not all records are shown.

The windows also gives out message that says "Your system is low on virtual memory. Windows is increasing the size of your virtual memory paging file. During this process, memory requests for some applications may be denied. For more information, see Help."

I am using an array of datatype 'Variant'
What is an alternative?

[vb5prgrmr]

First, increase the page file size and second, never use a varient array...

In vb's help type, on index tab, type in "data types" and hit enter. When the dialog comes up, scroll down to Data Types Keyword Summary Visual Basic Reference>Display. Click on Boolean (2nd line first word). Click on See Also>Select Data Type Summary Visual Basic Reference. Now, on that page scroll down to where it shows the overhead for varients...16 bytes for each element in the array that is a number and 22 bytes for each element that is considered a string by the unknown (runtimes) mysterious forces behind vb6.

My Goodness my good man/woman do you realize just how much memory you are eating up by using a varient??? Use a string if you must, or a long... BUT NEVER USE A VARIANT!!!!

Good Luck

[RahulV]

HI Friend,
I knew that already, ... but was just looking out for alternatives.
But now i hope there is no other alternative.
Thank you again.

[QVeen72]

Hi,

You actual problem is not the Memory.. it is the logic as to : WHY DO YOU NEED TO LOAD UP SO MUCH DATA IN AN ARRAY OR GRID.......?

there are alternative solutions without having to load such a huge data in Array or Grid...... You need to find such solutions....
Imagine with 20k records you are getting such problem, and then again with time, your data is going to increase.. Find the solution now itself...
Post your actual problem here..

Regards
Veena

[RahulV]

Hi,

You actual problem is not the Memory.. it is the logic as to : WHY DO YOU NEED TO LOAD UP SO MUCH DATA IN AN ARRAY OR GRID.......?

there are alternative solutions without having to load such a huge data in Array or Grid...... You need to find such solutions....
Imagine with 20k records you are getting such problem, and then again with time, your data is going to increase.. Find the solution now itself...
Post your actual problem here..

Regards
Veena

Dear Veena,
Then what are the alternatives that you suggest.
My scenario go as follows:
1) I have a really huge data, say records are 7 Lakhs in number
2) I have to calculate subtotals for almost 7 out of 10 columns. Apart from that I have to do several calculations on these columns before presenting them on the grid.

Thanks.

[vb5prgrmr]

Well why do you need an array? You can use the recordset as your array. So far I only see a need for seven variables and even these for your subtotals can be replaced with the recordset by letting the db do it for you...

strSQL = "SELECT SUM(field1) AS TOT1, SUM(field2) AS TOT2, ... TOT7

Good Luck

[QVeen72]

Dear Veena,
Then what are the alternatives that you suggest.
My scenario go as follows:
1) I have a really huge data, say records are 7 Lakhs in number
2) I have to calculate subtotals for almost 7 out of 10 columns. Apart from that I have to do several calculations on these columns before presenting them on the grid.

Thanks.

Hi,

Like vb5prgrmr, For sub totals open one more reordset..
And as far as displaying 7lakh records is considered, then, as it is, it is not possible with Grids, most of the grids have 64k cells capacity (note: not rows, its the count of cells.. This also true for Excel Per sheet capacity)..
You may have to use few grids to display all..
AND If at all calculating is beign done more often.. Why not store Calculated Value in extra columns in table itself....? Why ReCalculate every time..?

But still not sure, why anybody wants to scan thru 7 lakh records..
Say, if your grid has 25 rows visible, so user need to "PageDown" 28000 times just to see every thing populated (not reading)..
dosent seem very practical..

One more solution would be.. Just save all the calculated columns required in the table itself and with Crystal Reports just show the report.... (I guess this option will save lots of time unnecessary loop thru)

Regards
Veena