DaniWeb Forum Index > Software Development > C++

C++ RSS

Problem with large array

Please support our C++ advertiser: Intel Parallel Studio Home

Thread Solved

•

Join Date: Aug 2007

Posts: 6

Reputation: Trevora is an unknown quantity at this point

Solved Threads: 0

Trevora

Offline

Newbie Poster

Problem with large array

Aug 14th, 2007

Hello
I have a need to output a large array for a CNC machine I have built. The attached code includes comments which explains my problem.

Thank you for your consideration.

Attached Files

P2.CPP (1.4 KB, 11 views)

•

Join Date: Dec 2006

Posts: 1,089

Reputation: vijayan121 is a name known to all

Solved Threads: 164

vijayan121 vijayan121 is offline

Offline

Veteran Poster

Re: Problem with large array

Aug 15th, 2007

using far pointers you are still limited by 64K memory regions; its just that, that region can be outside your current CS or DS, and you can have many such regions. pointer arithmetic on far pointers do not modify the segment portion of the pointer, only its offset (modulo 64K arithmetic).

huge pointers are normalized every time they are modified; pointer arithmetic is *very* slow, but arrays of size >64k can be safely accessed. changing line 18 to

Help with Code Tags

C++ Syntax (Toggle Plain Text)

 char huge* I ;

should work.

Last edited by vijayan121; Aug 15th, 2007 at 1:30 am.

•

Join Date: Dec 2005

Posts: 5,850

Reputation: Salem has a reputation beyond repute

Solved Threads: 749

Salem

Offline

Void main'ers are DOOMed

Re: Problem with large array

Aug 15th, 2007

Also, delete I; must be delete [ ] I;

If you're processing the file in a linear fashion, is there any real need to store the whole file in memory at the same time?

•

Join Date: Aug 2007

Posts: 6

Reputation: Trevora is an unknown quantity at this point

Solved Threads: 0

Trevora

Offline

Newbie Poster

Re: Problem with large array

Aug 15th, 2007

Thank you Vajayan for your time. I tried your suggestion of declaring the pointer as huge but this did not overcome the problem. I would be interested if you have successfully run the program with the pointer declared as huge, as this may indicate that I have a problem with running my compiler.

•

Join Date: Aug 2007

Posts: 6

Reputation: Trevora is an unknown quantity at this point

Solved Threads: 0

Trevora

Offline

Newbie Poster

Re: Problem with large array

Aug 15th, 2007

Hi.
I changed the delete I to delete[] (and acknowledge that is good practice to always incorporate brackets) but this did not overcome the problem. I need to output in real time an array larger than 64k as 64k is just 304 mm of movement of the machine whereas I need at least 2000 mm.

•

Join Date: Dec 2006

Posts: 1,089

Reputation: vijayan121 is a name known to all

Solved Threads: 164

vijayan121 vijayan121 is offline

Offline

Veteran Poster

Re: Problem with large array

Aug 15th, 2007

•

Originally Posted by Trevora

...I would be interested if you have successfully run the program with the pointer declared as huge...

no, i have not run the program. what i posted was based on (somewhat foggy) memory of the days when intel tortured programmers with segented addressing and memory models. i do not have the compiler; so i cannot test it.

the following suggestions are also based on guesswork:
a. verify that you are using a library which returns *huge* pointers for things like operator new / malloc. perhaps, the compiler ships with different libraries; one for each memory model. one way to check this is verify that for the huge pointer you try to use (of the form segment:offset) segment is non-zero and not equal to youir DS
b. try an malloc/free instead of a new/delete.
c. try using an array (declared as huge) with a static storage duration.

do you really have to use this compiler? (the real question is: do you really have to execute your code under the segmented addressing mode of intel?)

•

Join Date: Dec 2005

Posts: 5,850

Reputation: Salem has a reputation beyond repute

Solved Threads: 749

Salem

Offline

Void main'ers are DOOMed

Re: Problem with large array

Aug 16th, 2007

So how is your CNC machine going to know whether you stored the whole thing in one massive array, or output them individually?

> 64k is just 304 mm of movement of the machine whereas I need at least 2000 mm.
Like who cares that it travels 300mm, appears to pause for a moment, then moves another 300mm. Your PC is so fast that you might not even notice.

Does your CNC machine store the whole program first, then only does things when you send a 'go' command? If so, there is absolutely no reason I can see to complicate the sending of the data by trying to create large arrays on your restricted OS.

> I have run the program on Pentium1/dos5, Pentium1/dos6.2
> and P4/XP machines.
If XP is a real choice, then get a better compiler. Any modern 32-bit compiler suitable for XP would have no trouble at all dealing with a couple of MB of array size.

Since you're using C++, maybe create a small class to do some work for you, by providing some kind of simulation of a large array?

 Help with Code Tags 
 C++ Syntax (Toggle Plain Text) 
#include <iostream>
using namespace std;
 
const int BLOCK_SIZE = 50000;
 
class bigA {
  private:
    char  **m_array;
    size_t  m_size;
    size_t  m_blocks;
  public:
    bigA( size_t size );
    ~bigA( void );
    char & operator [] ( unsigned int i );
};
 
bigA::bigA ( size_t size ) {
    m_size = size;
    m_blocks = size / BLOCK_SIZE;
    if ( size % BLOCK_SIZE != 0 ) m_blocks++;
    m_array = new char*[m_blocks];
    for ( size_t i = 0 ; i < m_blocks ; i++ ) {
        m_array[i] = new char[BLOCK_SIZE];
    }
}
 
bigA::~bigA ( void ) {
    for ( size_t i = 0 ; i < m_blocks ; i++ ) {
        delete [] m_array[i];
    }
    delete [] m_array;
}
 
char & bigA::operator [] ( unsigned int i ) {
    if ( i < m_size ) {
        size_t  row = i / BLOCK_SIZE;
        size_t  col = i % BLOCK_SIZE;
        return m_array[row][col];
    }
    // panic now, out of bounds
}
 
 
int main ( ) {
    bigA foo(123456);
    foo[1234] = 'a';
    cout << foo[1234] << endl;
    return 0;
}
#include <iostream>
using namespace std;

const int BLOCK_SIZE = 50000;

class bigA {
  private:
    char  **m_array;
    size_t  m_size;
    size_t  m_blocks;
  public:
    bigA( size_t size );
    ~bigA( void );
    char & operator [] ( unsigned int i );
};

bigA::bigA ( size_t size ) {
    m_size = size;
    m_blocks = size / BLOCK_SIZE;
    if ( size % BLOCK_SIZE != 0 ) m_blocks++;
    m_array = new char*[m_blocks];
    for ( size_t i = 0 ; i < m_blocks ; i++ ) {
        m_array[i] = new char[BLOCK_SIZE];
    }
}

bigA::~bigA ( void ) {
    for ( size_t i = 0 ; i < m_blocks ; i++ ) {
        delete [] m_array[i];
    }
    delete [] m_array;
}

char & bigA::operator [] ( unsigned int i ) {
    if ( i < m_size ) {
        size_t  row = i / BLOCK_SIZE;
        size_t  col = i % BLOCK_SIZE;
        return m_array[row][col];
    }
    // panic now, out of bounds
}


int main ( ) {
    bigA foo(123456);
    foo[1234] = 'a';
    cout << foo[1234] << endl;
    return 0;
}

•

Join Date: Aug 2007

Posts: 6

Reputation: Trevora is an unknown quantity at this point

Solved Threads: 0

Trevora

Offline

Newbie Poster

Re: Problem with large array

Aug 16th, 2007

I will follow up on your suggestions and come back again later Vajayan121.

In the meantime I would be grateful if there is a kind person out there able to compile and run the program on a more current compiler.

Also I would value advice on which replacement compiler I should acquire. I would want an IDE and the machine computer interface board and real time limits me to using dos.

•

Join Date: Dec 2006

Posts: 1,089

Reputation: vijayan121 is a name known to all

Solved Threads: 164

vijayan121 vijayan121 is offline

Offline

Veteran Poster

Re: Problem with large array

Aug 16th, 2007

•

Originally Posted by Trevora

...compile and run the program on a more current compiler

runs without any problems on microsoft vc++ 8.0 and gcc 3.4.x (need to change headers iostream.h => iostream etc.)

•

Also I would value advice on which replacement compiler I should acquire. I would want an IDE and the machine computer interface board and real time limits me to using dos.

if you have to run under dos, this is reported to be a workable solution: http://www.delorie.com/djgpp/doc/ug/...-is-djgpp.html & http://www.delorie.com/djgpp/v2faq/ it supports 32 bit addressing; so array sizes are not an issue if you have sufficient ram. see: http://www.delorie.com/djgpp/doc/ug/basics/32bit.html there is an ide for gjgpp: ttp://www.rhide.com/ djgpp is particularly popular among game programmers.

note: Salem's idea (sending a large array in smaller chunks of <64k) would be a simple and elegant solution; you should not rule it out without investigating the possibility.

Last edited by vijayan121; Aug 16th, 2007 at 7:08 am.

•

Join Date: Aug 2007

Posts: 6

Reputation: Trevora is an unknown quantity at this point

Solved Threads: 0

Trevora

Offline

Newbie Poster

Re: Problem with large array

#10

Aug 18th, 2007

I will answer the respondents and pose a question at the end.

Thank you for your contribution Salem of breaking up the large array into smaller chunks. The machine starts on receiving char I[0]. It demands I[1]whilst moving according to I[0] and so on to the end of the array. This arrangement plus dos puts the computer under the control of the machine and satisfies the real time and safety considerations of the machine.

Pausing every 300mm of movement would slow/stall the machine and cause bit to overheat, unless mitigated for. However, the time duration between each output of I[x] is 100microseconds when movement is at maximum speed so there is time to progressively manipulate arrays between movement steps.

Thank you Vijayan121 for confirming that the program can run under another (more modern!) compiler and your suggestion of an alternative to the compiler I have.

Should you Salem or Vijayan121 decide to visit Wellington, New Zealand in the future please email me.

What was puzzling me when I originally posted my problem was that I interpreted the Turbo C++ V3.0 documentation to mean that I could do as follows:

Invoke the Huge Memory Model and declare the array to be local for it to be compiled into dynamic memory. I understood the size of the array would, under these circumstances, be limited to the amount of available heap, not just one segment.

If there is some venerable sage who can confirm this is the case and/or point out the error of my ways I would be indeed be grateful.