Pipe broken at 2GB when tarring to a samba mount

Reply

Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Pipe broken at 2GB when tarring to a samba mount

 
0
  #1
Nov 14th, 2007
I am running some makeshift tar backups, because our tape drive is broken.

The target location of the tar file is a mounted samba share on another server on the network.

Once the tar file reaches 2 GB, I get the following error.

backupjob.bash: line 79: 25891 Broken pipe

This only happens if I run the tar command on the server it is being backed up FROM. If I run the tar command, on the server it is being backed up TO (which is grabbing the files to tar off of a mounted samba share on the main server) then it does not have this error. I dont want to have to do that though - because it has to transfer everything to be backed up, uncompressed.


What seems kind of odd about this to me -- is that my cp/mv/rm commands are all versions that can't handle large files (I think 2GB is the limit)(my tar command does not have the 2GB limit). So whenever I have had to delete a really large file, I always do it on my windows PC through samba -- which is able to delete and move large files.

So it still seems odd to me (considering samba is how I delete 2GB+ files) that I am getting this error, seemingly just because it's trying to access a 2GB+ file over samba.

Does anyone have any idea why this is happening?
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #2
Nov 14th, 2007
I have done some tests. And if I even try to copy a file bigger than 2GB to the other server through the mounted samba share -- when it gets to 2GB - i get the following error

Filesize limit exceeded (core dumped)

However, I drives mapped to both servers on my windows PC -- and I can drag and copy a 5GB file from one server to the other -- with no problems at all.

So the problem does not seem to be the samba share itself -- but rather somehow, how it is mounted.

Any Idea's?
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #3
Nov 14th, 2007
Think I found the problem.

I had mounted the mounts in question using webmin, and it apparently used "mount.smbfs" instead of "smbmount".

I am about to remount with smbmount, and hopefully it works.
Last edited by bdb4269; Nov 14th, 2007 at 6:52 pm.
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #4
Nov 14th, 2007
nope -- that didn't solve the problem -- seems my smbmount command is limited to 2GB -- I have kernel 2.4.21-27.0.2 -- am I able to get a smbmount that supports +2GB files without updating kernel?
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 1
Reputation: mrhua is an unknown quantity at this point 
Solved Threads: 0
mrhua mrhua is offline Offline
Newbie Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #5
May 7th, 2008
try the lfs option when you mount the smbfs ; it worked for me

mount -t smbfs //10.1.1.5/SHARE /mnt/backup0 -o username=your_user,password=your_pw,lfs
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #6
May 13th, 2008
Originally Posted by mrhua View Post
try the lfs option when you mount the smbfs ; it worked for me

mount -t smbfs //10.1.1.5/SHARE /mnt/backup0 -o username=your_user,password=your_pw,lfs
Thanks for the idea. This didn't work though. -- I think it might have something to do with my mv and cp commands, being old versions that dont support large files.
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 46
Reputation: Major Major is an unknown quantity at this point 
Solved Threads: 2
Major Major Major Major is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #7
May 23rd, 2008
Your problem might have something to do with samba itself. try mounting the samba share with ,lfs at the end of your -o string

that being said it sounds like both your servers are *NIX based, why use samba at all?

read up on rsync.
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #8
May 23rd, 2008
Thanks for the suggestions.

I did actually just try the ,lfs option, and it still did not work.

I am familiar with rsync. We use it sync our main server(server 1), with our "backup" server(server 2), for use in case our main server fails.

What I need in this case though, is to be able to mount a directory from another server (server 3) on the main server (server 1). So that we can easily copy files between them from the command line, and in shell scripts. I suppose I could use NFS if needed, but I already have samba set up, because we need to be able to access the files on any of the server from our windows workstations. So I figured it would just be easiest to mount the samba shares, that already existed.

The strange thing is, I have mapped drives to shares on both servers, and I can "drag n drop" 2GB+ files from server to server -- so it's not the samba share it self, it's seems to be how its mounted.

It's not a huge deal really. We never need to copy 2GB+ files in a script, and rarely need to at all, and the few times we do, I can just drag n drop from windows.

I was just hoping I could find out what is wrong with the mount (i.e. do I need a new version of smbmount; am I mounting it wrong; etc), mainly just so I could understand mounting samba shares on linux a little better.
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 46
Reputation: Major Major is an unknown quantity at this point 
Solved Threads: 2
Major Major Major Major is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #9
May 23rd, 2008
Problem is if you're using two linux or *NIX boxes Samba is nasty because it's basically translating from *nix fs -> samba's kludge of MS SMB protocol ->*nix fs. a lot can get lost in that translation.

It's a moving target trying to hit another moving target (microsoft's SMB protocol) which changes with every release of MS operating systems and the service packs. "DOS ain't done 'til Lotus won't run." I don't know if you've come across that quote but it applies to Samba and SMB and Microsoft's stance on the two. If you don't need to use Samba to interface with Windows computers, don't use it, is probably the best advice I can give.

SMB is a really chatty protocol too. the latest version requires something on the order of 1500 (I think the number that tridge gave was 1506) packets to delete a file!


getting back to the subject at hand try this

tar -M -L 2000000 -cv -f archive1 -f archive2[...] -farchiveN /dir/to/backup

this will make tar swicth archives after 2000000*1024 bytes, just make sure you have enough
'-f archiveN' defined to cover the size of the backup or it will prompt you in a very cryptic way to mount a new tape (and will overwrite the one it just finished if it doesn't like your answer )

a drawback is that you can't use compression when spanning archives.

if you're good at writing shell or have time on your hands you can use the -F directive to point to a script that will automatically 'change the tape' for you.
Reply With Quote Quick reply to this message  
Join Date: Sep 2007
Posts: 27
Reputation: bdb4269 is an unknown quantity at this point 
Solved Threads: 0
bdb4269 bdb4269 is offline Offline
Light Poster

Re: Pipe broken at 2GB when tarring to a samba mount

 
0
  #10
May 23rd, 2008
Thanks for the info!
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Similar Threads
Other Threads in the *nix Software Forum
Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC