2Gb output limit

Please support our C++ advertiser: Intel Parallel Studio Home
Reply

Join Date: Mar 2009
Posts: 8
Reputation: pagu is an unknown quantity at this point 
Solved Threads: 0
pagu pagu is offline Offline
Newbie Poster

2Gb output limit

 
0
  #1
Mar 17th, 2009
Dear all
I made a code in C++, for data processing.
The code read, processing and write the output several times.
The process abort when the output file get 2Gb fo size.
I know that it is not a disk limit.
Is that a flag mistake on the makefile?
thanks
Reply With Quote Quick reply to this message  
Join Date: Aug 2005
Posts: 15,348
Reputation: Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute 
Solved Threads: 1462
Team Colleague
Featured Poster
Ancient Dragon's Avatar
Ancient Dragon Ancient Dragon is offline Offline
Still Learning

Re: 2Gb output limit

 
0
  #2
Mar 17th, 2009
what compiler and operating system. It could be a limit of your os and/or compiler. Try your program on a 64-bit os, such as 64-bit XP or Vista and see if that fixes it.
Don't PM me with questions -- you might get a nasty PM in response. If you have a question then post it in one of the forums.
Reply With Quote Quick reply to this message  
Join Date: Mar 2009
Posts: 8
Reputation: pagu is an unknown quantity at this point 
Solved Threads: 0
pagu pagu is offline Offline
Newbie Poster

Re: 2Gb output limit

 
0
  #3
Mar 17th, 2009
I am using centus 4 and gnu compiler

Originally Posted by Ancient Dragon View Post
what compiler and operating system. It could be a limit of your os and/or compiler. Try your program on a 64-bit os, such as 64-bit XP or Vista and see if that fixes it.
Reply With Quote Quick reply to this message  
Join Date: Aug 2005
Posts: 15,348
Reputation: Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute 
Solved Threads: 1462
Team Colleague
Featured Poster
Ancient Dragon's Avatar
Ancient Dragon Ancient Dragon is offline Offline
Still Learning

Re: 2Gb output limit

 
0
  #4
Mar 17th, 2009
I don't use those, but I think I read someone say g++ and gcc have an option to produce 64-bit programs.
Don't PM me with questions -- you might get a nasty PM in response. If you have a question then post it in one of the forums.
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 1
Reputation: mad_ivan is an unknown quantity at this point 
Solved Threads: 0
mad_ivan mad_ivan is offline Offline
Newbie Poster

Re: 2Gb output limit

 
0
  #5
Mar 17th, 2009
Originally Posted by pagu View Post
...
I know that it is not a disk limit.
...
Um - it may not be a "disk limit" as in a full disk, but your file system type (ext3 at a guess) probably has a maximum file size.

Hope this helps.
Reply With Quote Quick reply to this message  
Join Date: Mar 2009
Posts: 8
Reputation: pagu is an unknown quantity at this point 
Solved Threads: 0
pagu pagu is offline Offline
Newbie Poster

Re: 2Gb output limit

 
0
  #6
Mar 17th, 2009
I had checked and that is not the case.


Originally Posted by mad_ivan View Post
Um - it may not be a "disk limit" as in a full disk, but your file system type (ext3 at a guess) probably has a maximum file size.

Hope this helps.
Reply With Quote Quick reply to this message  
Join Date: Aug 2007
Posts: 1,674
Reputation: vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold 
Solved Threads: 193
vmanes's Avatar
vmanes vmanes is offline Offline
Posting Virtuoso

Re: 2Gb output limit

 
0
  #7
Mar 17th, 2009
You say you process it several times. Do you attempt to read in the full file, process it, add to it, then write a new, larger file? Perhaps showing us pertinent portions of the code would help.
"We Americans got so tired of being thought of as dumb by the rest of the world that we went to the polls last November and removed all doubt."
~~~~~~~~~~~~~~~~~~
Looking for an exciting graduate degree? Robotics and Intelligent Autonomous Systems (RIAS) at SDSM&T See the program brochure here.
Reply With Quote Quick reply to this message  
Join Date: Mar 2009
Posts: 8
Reputation: pagu is an unknown quantity at this point 
Solved Threads: 0
pagu pagu is offline Offline
Newbie Poster

Re: 2Gb output limit

 
0
  #8
Mar 17th, 2009
No.
The data is a large 3d matrix.
Read one line, process this line and write the output line.
each line is like 1megabite of file.

Originally Posted by vmanes View Post
You say you process it several times. Do you attempt to read in the full file, process it, add to it, then write a new, larger file? Perhaps showing us pertinent portions of the code would help.
Reply With Quote Quick reply to this message  
Join Date: Aug 2007
Posts: 1,674
Reputation: vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold vmanes is a splendid one to behold 
Solved Threads: 193
vmanes's Avatar
vmanes vmanes is offline Offline
Posting Virtuoso

Re: 2Gb output limit

 
0
  #9
Mar 17th, 2009
Is it aborting because the OS can't handle the file, or is there a file size counter you're maintaining in the program?

Remember that a signed int has an upper limit of 2GB.
"We Americans got so tired of being thought of as dumb by the rest of the world that we went to the polls last November and removed all doubt."
~~~~~~~~~~~~~~~~~~
Looking for an exciting graduate degree? Robotics and Intelligent Autonomous Systems (RIAS) at SDSM&T See the program brochure here.
Reply With Quote Quick reply to this message  
Join Date: Mar 2009
Posts: 8
Reputation: pagu is an unknown quantity at this point 
Solved Threads: 0
pagu pagu is offline Offline
Newbie Poster

Re: 2Gb output limit

 
0
  #10
Mar 17th, 2009
there is not a counter inside the code.
The morery is clened avery type the programa finish to process one line and write its output.


Originally Posted by vmanes View Post
Is it aborting because the OS can't handle the file, or is there a file size counter you're maintaining in the program?

Remember that a signed int has an upper limit of 2GB.
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Other Threads in the C++ Forum
Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC