| | |
Pipe broken at 2GB when tarring to a samba mount
![]() |
•
•
Join Date: Sep 2007
Posts: 27
Reputation:
Solved Threads: 0
I am running some makeshift tar backups, because our tape drive is broken.
The target location of the tar file is a mounted samba share on another server on the network.
Once the tar file reaches 2 GB, I get the following error.
This only happens if I run the tar command on the server it is being backed up FROM. If I run the tar command, on the server it is being backed up TO (which is grabbing the files to tar off of a mounted samba share on the main server) then it does not have this error. I dont want to have to do that though - because it has to transfer everything to be backed up, uncompressed.
What seems kind of odd about this to me -- is that my cp/mv/rm commands are all versions that can't handle large files (I think 2GB is the limit)(my tar command does not have the 2GB limit). So whenever I have had to delete a really large file, I always do it on my windows PC through samba -- which is able to delete and move large files.
So it still seems odd to me (considering samba is how I delete 2GB+ files) that I am getting this error, seemingly just because it's trying to access a 2GB+ file over samba.
Does anyone have any idea why this is happening?
The target location of the tar file is a mounted samba share on another server on the network.
Once the tar file reaches 2 GB, I get the following error.
backupjob.bash: line 79: 25891 Broken pipe
This only happens if I run the tar command on the server it is being backed up FROM. If I run the tar command, on the server it is being backed up TO (which is grabbing the files to tar off of a mounted samba share on the main server) then it does not have this error. I dont want to have to do that though - because it has to transfer everything to be backed up, uncompressed.
What seems kind of odd about this to me -- is that my cp/mv/rm commands are all versions that can't handle large files (I think 2GB is the limit)(my tar command does not have the 2GB limit). So whenever I have had to delete a really large file, I always do it on my windows PC through samba -- which is able to delete and move large files.
So it still seems odd to me (considering samba is how I delete 2GB+ files) that I am getting this error, seemingly just because it's trying to access a 2GB+ file over samba.
Does anyone have any idea why this is happening?
•
•
Join Date: Sep 2007
Posts: 27
Reputation:
Solved Threads: 0
I have done some tests. And if I even try to copy a file bigger than 2GB to the other server through the mounted samba share -- when it gets to 2GB - i get the following error
However, I drives mapped to both servers on my windows PC -- and I can drag and copy a 5GB file from one server to the other -- with no problems at all.
So the problem does not seem to be the samba share itself -- but rather somehow, how it is mounted.
Any Idea's?
Filesize limit exceeded (core dumped)
However, I drives mapped to both servers on my windows PC -- and I can drag and copy a 5GB file from one server to the other -- with no problems at all.
So the problem does not seem to be the samba share itself -- but rather somehow, how it is mounted.
Any Idea's?
•
•
Join Date: Sep 2007
Posts: 27
Reputation:
Solved Threads: 0
•
•
•
•
try the lfs option when you mount the smbfs ; it worked for me
mount -t smbfs //10.1.1.5/SHARE /mnt/backup0 -o username=your_user,password=your_pw,lfs
-- I think it might have something to do with my mv and cp commands, being old versions that dont support large files. •
•
Join Date: Sep 2007
Posts: 27
Reputation:
Solved Threads: 0
Thanks for the suggestions.
I did actually just try the ,lfs option, and it still did not work.
I am familiar with rsync. We use it sync our main server(server 1), with our "backup" server(server 2), for use in case our main server fails.
What I need in this case though, is to be able to mount a directory from another server (server 3) on the main server (server 1). So that we can easily copy files between them from the command line, and in shell scripts. I suppose I could use NFS if needed, but I already have samba set up, because we need to be able to access the files on any of the server from our windows workstations. So I figured it would just be easiest to mount the samba shares, that already existed.
The strange thing is, I have mapped drives to shares on both servers, and I can "drag n drop" 2GB+ files from server to server -- so it's not the samba share it self, it's seems to be how its mounted.
It's not a huge deal really. We never need to copy 2GB+ files in a script, and rarely need to at all, and the few times we do, I can just drag n drop from windows.
I was just hoping I could find out what is wrong with the mount (i.e. do I need a new version of smbmount; am I mounting it wrong; etc), mainly just so I could understand mounting samba shares on linux a little better.
I did actually just try the ,lfs option, and it still did not work.
I am familiar with rsync. We use it sync our main server(server 1), with our "backup" server(server 2), for use in case our main server fails.
What I need in this case though, is to be able to mount a directory from another server (server 3) on the main server (server 1). So that we can easily copy files between them from the command line, and in shell scripts. I suppose I could use NFS if needed, but I already have samba set up, because we need to be able to access the files on any of the server from our windows workstations. So I figured it would just be easiest to mount the samba shares, that already existed.
The strange thing is, I have mapped drives to shares on both servers, and I can "drag n drop" 2GB+ files from server to server -- so it's not the samba share it self, it's seems to be how its mounted.
It's not a huge deal really. We never need to copy 2GB+ files in a script, and rarely need to at all, and the few times we do, I can just drag n drop from windows.
I was just hoping I could find out what is wrong with the mount (i.e. do I need a new version of smbmount; am I mounting it wrong; etc), mainly just so I could understand mounting samba shares on linux a little better.
•
•
Join Date: May 2008
Posts: 46
Reputation:
Solved Threads: 2
Problem is if you're using two linux or *NIX boxes Samba is nasty because it's basically translating from *nix fs -> samba's kludge of MS SMB protocol ->*nix fs. a lot can get lost in that translation.
It's a moving target trying to hit another moving target (microsoft's SMB protocol) which changes with every release of MS operating systems and the service packs. "DOS ain't done 'til Lotus won't run." I don't know if you've come across that quote but it applies to Samba and SMB and Microsoft's stance on the two. If you don't need to use Samba to interface with Windows computers, don't use it, is probably the best advice I can give.
SMB is a really chatty protocol too. the latest version requires something on the order of 1500 (I think the number that tridge gave was 1506) packets to delete a file!
getting back to the subject at hand try this
tar -M -L 2000000 -cv -f archive1 -f archive2[...] -farchiveN /dir/to/backup
this will make tar swicth archives after 2000000*1024 bytes, just make sure you have enough
'-f archiveN' defined to cover the size of the backup or it will prompt you in a very cryptic way to mount a new tape (and will overwrite the one it just finished if it doesn't like your answer
)
a drawback is that you can't use compression when spanning archives.
if you're good at writing shell or have time on your hands you can use the -F directive to point to a script that will automatically 'change the tape' for you.
It's a moving target trying to hit another moving target (microsoft's SMB protocol) which changes with every release of MS operating systems and the service packs. "DOS ain't done 'til Lotus won't run." I don't know if you've come across that quote but it applies to Samba and SMB and Microsoft's stance on the two. If you don't need to use Samba to interface with Windows computers, don't use it, is probably the best advice I can give.
SMB is a really chatty protocol too. the latest version requires something on the order of 1500 (I think the number that tridge gave was 1506) packets to delete a file!
getting back to the subject at hand try this
tar -M -L 2000000 -cv -f archive1 -f archive2[...] -farchiveN /dir/to/backup
this will make tar swicth archives after 2000000*1024 bytes, just make sure you have enough
'-f archiveN' defined to cover the size of the backup or it will prompt you in a very cryptic way to mount a new tape (and will overwrite the one it just finished if it doesn't like your answer
) a drawback is that you can't use compression when spanning archives.
if you're good at writing shell or have time on your hands you can use the -F directive to point to a script that will automatically 'change the tape' for you.
![]() |
Similar Threads
- Linux newbee- help with Samba in Win2K domain (*nix Software)
- Cost to have muffler repaired? (Geeks' Lounge)
- samba server error (Shell Scripting)
- Weird problems with Samba in RHEL4 (*nix Software)
- samba 3.0.22 on solaris 9 (Getting Started and Choosing a Distro)
- Can't get samba to work! (*nix Software)
- A broken down Intel D845Wn mother board (Motherboards, CPUs and RAM)
- Samba writing stall error; linux to linux transfer (*nix Software)
- configuring samba networking on windows (Kernels and Modules)
Other Threads in the *nix Software Forum
- Previous Thread: superblock last mounted in the future
- Next Thread: why dead letter?
| Thread Tools | Search this Thread |





