 |  |  |  |  |  |  |  |
A shell script that makes backup copies of changed files (also involving CRON)
 |
•
•
Join Date: May 2008
Posts: 82
Reputation: 
Solved Threads: 0
Junior Poster in Training
Hey
Ive been asked (and I may add this to my system as well) to make a shell script that backups to all files in x location to another location in z. The thing is that the backup must only be made if one file has been changed. Example:
/x contains:
a (1 byte)
b (1 byte)
c (1 byte)
/y contains:
nothing
Ill put this script into CRON's file and everytime (say 3) it will execute. 3 arrives and there is nothing in /y and it will copy all files in /x to /y. To compensate space I think a good idea (this I will do to my script for my system) is gzip it all up. Anyways the current state right now is.
/x contains:
a (1 byte)
b (1 byte)
c (1 byte)
/y contains:
a (1 byte)
b (1 byte)
c (1 byte)
Again 3 arrives and /y is the same as /x so nothing changes and nothing happens. But now, I open c in /x and type something inside now it is:
/x contains:
a (1 byte)
b (1 byte)
c (2 bytes)
This changes it so now when the script excutes at 3 it has to delete everything in /y and rewrite it all over again (regardless that the other files havent changed).
How do I do this? I imagine something like doing a ls -a on /x and using cut to cut certain columns and seeing if it equals /y's files/file sizes but im not sure how to make a filename and file size relation and a bit humiliating but I dont know how to use cut either (I know something but still)
If someone could help thank you very much.
Ive been asked (and I may add this to my system as well) to make a shell script that backups to all files in x location to another location in z. The thing is that the backup must only be made if one file has been changed. Example:
/x contains:
a (1 byte)
b (1 byte)
c (1 byte)
/y contains:
nothing
Ill put this script into CRON's file and everytime (say 3) it will execute. 3 arrives and there is nothing in /y and it will copy all files in /x to /y. To compensate space I think a good idea (this I will do to my script for my system) is gzip it all up. Anyways the current state right now is.
/x contains:
a (1 byte)
b (1 byte)
c (1 byte)
/y contains:
a (1 byte)
b (1 byte)
c (1 byte)
Again 3 arrives and /y is the same as /x so nothing changes and nothing happens. But now, I open c in /x and type something inside now it is:
/x contains:
a (1 byte)
b (1 byte)
c (2 bytes)
This changes it so now when the script excutes at 3 it has to delete everything in /y and rewrite it all over again (regardless that the other files havent changed).
How do I do this? I imagine something like doing a ls -a on /x and using cut to cut certain columns and seeing if it equals /y's files/file sizes but im not sure how to make a filename and file size relation and a bit humiliating but I dont know how to use cut either (I know something but still)
If someone could help thank you very much.
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#2 Aug 26th, 2009
Why don't you just use rsync? This is a significant undertaking when the wheel has already been invented.
•
•
Join Date: May 2008
Posts: 82
Reputation:
Solved Threads: 0
Junior Poster in Training
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#3 Aug 27th, 2009
•
•
•
•
Originally Posted by sknake 
Why don't you just use rsync? This is a significant undertaking when the wheel has already been invented.
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#4 Aug 27th, 2009
Yeah .. look at the rsync source for ideas 
•
•
Join Date: May 2008
Posts: 82
Reputation:
Solved Threads: 0
Junior Poster in Training
Re: A shell script that makes backup copies of changed files (also involving CRON)
1
#5 Aug 28th, 2009
•
•
•
•
Originally Posted by sknake
Yeah .. look at the rsync source for ideas
Id just like to make this using a shell script. Please no more comments suggesting rsync. Thank you
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#6 Aug 28th, 2009
Ok well build a file index of your source directories and destination directories. Then build a list of source/dest files that do not exist in either folders and delete/add them as necessary. For files that do exist in both directories you should use an md5sum and filesize check depending on how careful you want to be. Collision rates are very low (unless malicious and intentional) for md5sums so you could probably get away with just do that.
You could use ls and parse the columns, or use
Just because the size is the same doesn't mean that it has not changed so also compare the md5 stamp:
Now just automate all of that and you're good to go!
You could use ls and parse the columns, or use
du or stat : bash Syntax (Toggle Plain Text)
Just because the size is the same doesn't mean that it has not changed so also compare the md5 stamp:
bash Syntax (Toggle Plain Text)
Now just automate all of that and you're good to go!
•
•
Join Date: May 2008
Posts: 82
Reputation:
Solved Threads: 0
Junior Poster in Training
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#7 Aug 29th, 2009
•
•
•
•
Originally Posted by sknake
Ok well build a file index of your source directories and destination directories. Then build a list of source/dest files that do not exist in either folders and delete/add them as necessary. For files that do exist in both directories you should use an md5sum and filesize check depending on how careful you want to be. Collision rates are very low (unless malicious and intentional) for md5sums so you could probably get away with just do that.
You could use ls and parse the columns, or useduorstat:
bash Syntax (Toggle Plain Text)
Just because the size is the same doesn't mean that it has not changed so also compare the md5 stamp:
bash Syntax (Toggle Plain Text)
Now just automate all of that and you're good to go!
Thank you for helping
Lets analize everything.
du lists with details everything in the current folder, -b showing it in bytes and sed cuts off all the chars listed. That I understand.
md5sum would actually show me the md5 of backup.sh right? Not the actual files in the backed up file/folder/etc. And why would I cut off those chars if md5 includes letters and numbers?
Also reexplain please the part about "build a file index of your source directories and destination directories. Then build a list of source/dest files that do not exist in either folders and delete/add them as necessary."
would I have to do something like:
Shell Scripting Syntax (Toggle Plain Text)
Is that what you ment?
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#8 Aug 29th, 2009
>>md5sum would actually show me the md5 of backup.sh right? Not the actual files in the backed up file/folder/etc. And why would I cut off those chars if md5 includes letters and numbers?
The expression I gave for the md5 didn't cut off anything, it took the entire hash. Just like
As far as your concept for builing the directories -- yes, that is how you would go about it. However ls prints out a lot of crap you don't need for this so you would be better off using
The expression I gave for the md5 didn't cut off anything, it took the entire hash. Just like
du you will need to run md5sum on every file to get the file size & check sum.As far as your concept for builing the directories -- yes, that is how you would go about it. However ls prints out a lot of crap you don't need for this so you would be better off using
find /src/dir -type d to make that directories with mkdir -p and then use find /src/dir -type f to get the files. •
•
Join Date: May 2008
Posts: 82
Reputation:
Solved Threads: 0
Junior Poster in Training
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#9 Aug 30th, 2009
•
•
•
•
Originally Posted by sknake
As far as your concept for builing the directories -- yes, that is how you would go about it. However ls prints out a lot of crap you don't need for this so you would be better off usingfind /src/dir -type dto make that directories withmkdir -pand then usefind /src/dir -type fto get the files.
Re: A shell script that makes backup copies of changed files (also involving CRON)
0
#10 Aug 30th, 2009
.... you have a long road ahead of you
Take your pick
Take your pick
bash Syntax (Toggle Plain Text)
|
Similar Threads
- load-intensive shell script (Shell Scripting)
- Script to delete or backup logs at a specific time and under specific user (Shell Scripting)
- Shell Script to Zip / FTP / Delete transactional files. (Shell Scripting)
- For Pay Shell Script - Zip / FTP / Delete transactional files (Shell Scripting)
- shell script fo backup archiving (Shell Scripting)
- Korn Shell Script for deleting files older than 2 months (Shell Scripting)
- c++ or shell script to delete some files (C++)
- How to delete files in UNIX using shell script (Shell Scripting)
- Shell Script Help (Shell Scripting)
Other Threads in the Shell Scripting Forum
- Previous Thread: Change Layout
- Next Thread: How can i display time 5mins ago on Solaris 10
| Thread Tools | Search this Thread |
You need to join our community in order to build your list of favorite forums. You can visit the forum index for a listing of all forums.
Related Forum Features
Latest Shell Scripting Posts
- Today's Posts
- Unanswered Threads
