Usage:
Changed Functionality:
Current cron job:
Every 4 hours, at 10 minutes past the hour, backup - add
10 7-23/4 * * * /root/backup.sh >> /backup/logs/backup_log.txt 2>>/backup/logs/backup_log.txt
Proposed cron job:
New logfile for every backup.
$date = `date +%Y-%m-%d-%T`
# example date is 2008-01-18-13:00:15
# ordering by descending unit of time (year month day) allows listing logs by time order
10 7-23/4 * * * /root/backup.sh > /backup/logs/backup_log_$date.txt 2>/backup/logs/backup_log_$date.txt
# we use > to create a new file (not >> to append)
# Additionally, backup.sh should add the -i parameter to rsync to capture the benefits of detailed changelogs
rsync has a built-in change log feature, enabled with -i.
It will log the exact changes and the filename.
A sample run looks like this:
sent 52467 bytes received 136 bytes 105206.00 bytes/sec
total size is 52051 speedup is 0.99
building file list ... done
04-docs/
04-docs/00-system_information/Client address list-REV.xls
04-docs/00-system_information/_FileZilla/
04-docs/00-system_information/_FileZilla/filezilla.xml
04-docs/00-system_information/_FileZilla/layout.xml
04-docs/00-system_information/_FileZilla/queue.xml
04-docs/00-system_information/_FileZilla/sitemanager.xml
04-docs/disaster_recovery/
sent 227185 bytes received 148 bytes 454666.00 bytes/sec
total size is 16705805 speedup is 73.49
This is nice for auditing purposes; we can grep through the log and categorize/review changes en masse.
YXcstpogz is a 9 character field before every entry in the itemized change log.
If a period replaces any of these characters, for example:
.d..t...... ./
this means there was no change.
Y Update types:
< = sent to remote host
> = received from remote host
c = local change/creation
h = hard link
. = no data update, but file attributes might be changed
X File types:
f = file
d = directory
L = symlink
D = device
S = named sockets/fifos
c checksum was different
s size was different
t,T modification time was different, T = symlink AND not using --times
+ Newly created item
. Identical item
? Unknown attribute (mismatched rsync version)
--delete
This tells rsync to delete extraneous files from the receiving
side (ones that arenât on the sending side), but only for the
directories that are being synchronized. You must have asked
rsync to send the whole directory (e.g. "dir" or "dir/") without
using a wildcard for the directoryâs contents (e.g. "dir/*")
since the wildcard is expanded by the shell and rsync thus gets
a request to transfer individual files, not the filesâ parent
directory. Files that are excluded from transfer are also
excluded from being deleted unless you use the --delete-excluded
option or mark the rules as only matching on the sending side
(see the include/exclude modifiers in the FILTER RULES section).
Prior to rsync 2.6.7, this option would have no effect unless
--recursive was in effect. Beginning with 2.6.7, deletions will
also occur when --dirs (-d) is in effect, but only for directo-
ries whose contents are being copied.
This option can be dangerous if used incorrectly! It is a very
good idea to run first using the --dry-run option (-n) to see
what files would be deleted to make sure important files arenât
listed.
If the sending side detects any I/O errors, then the deletion of
any files at the destination will be automatically disabled.
This is to prevent temporary filesystem failures (such as NFS
errors) on the sending side causing a massive deletion of files
on the destination. You can override this with the --ignore-
errors option.
The --delete option may be combined with one of the --delete-
WHEN options without conflict, as well as --delete-excluded.
However, if none of the --delete-WHEN options are specified,
rsync will currently choose the --delete-before algorithm. A
future version may change this to choose the --delete-during
algorithm. See also --delete-after.
--delete-before
Request that the file-deletions on the receiving side be done
before the transfer starts. This is the default if --delete or
--delete-excluded is specified without one of the --delete-WHEN
options. See --delete (which is implied) for more details on
file-deletion.
Deleting before the transfer is helpful if the filesystem is
tight for space and removing extraneous files would help to make
the transfer possible. However, it does introduce a delay
before the start of the transfer, and this delay might cause the
transfer to timeout (if --timeout was specified).
-l, --links
When symlinks are encountered, recreate the symlink on the des-
tination.
-H, --hard-links
This tells rsync to look for hard-linked files in the transfer
and link together the corresponding files on the receiving side.
Without this option, hard-linked files in the transfer are
treated as though they were separate files.
Note that rsync can only detect hard links if both parts of the
link are in the list of files being sent.
-p, --perms
This option causes the receiving rsync to set the destination
permissions to be the same as the source permissions. (See also
the --chmod option for a way to modify what rsync considers to
be the source permissions.)
When this option is off, permissions are set as follows:
o Existing files (including updated files) retain their
existing permissions, though the --executability option
might change just the execute permission for the file.
o New files get their "normal" permission bits set to the
source fileâs permissions masked with the receiving
directoryâs default permissions (either the receiving
processâs umask, or the permissions specified via the
destination directoryâs default ACL), and their special
permission bits disabled except in the case where a new
directory inherits a setgid bit from its parent direc-
tory.
Thus, when --perms and --executability are both disabled,
rsyncâs behavior is the same as that of other file-copy utili-
ties, such as cp(1) and tar(1).
In summary: to give destination files (both old and new) the
source permissions, use --perms. To give new files the destina-
tion-default permissions (while leaving existing files
unchanged), make sure that the --perms option is off and use
--chmod=ugo=rwX (which ensures that all non-masked bits get
enabled). If youâd care to make this latter behavior easier to
type, you could define a popt alias for it, such as putting this
line in the file ~/.popt (this defines the -s option, and
includes --no-g to use the default group of the destination
dir):
rsync alias -s --no-p --no-g --chmod=ugo=rwX
You could then use this new option in a command such as this
one:
rsync -asv src/ dest/
(Caveat: make sure that -a does not follow -s, or it will re-
enable the "--no-*" options.)
The preservation of the destinationâs setgid bit on newly-cre-
ated directories when --perms is off was added in rsync 2.6.7.
Older rsync versions erroneously preserved the three special
permission bits for newly-created files when --perms was off,
while overriding the destinationâs setgid bit setting on a
newly-created directory. Default ACL observance was added to
the ACL patch for rsync 2.6.7, so older (or non-ACL-enabled)
rsyncs use the umask even if default ACLs are present. (Keep in
mind that it is the version of the receiving rsync that affects
these behaviors.)
-E, --executability
This option causes rsync to preserve the executability (or non-
executability) of regular files when --perms is not enabled. A
regular file is considered to be executable if at least one âxâ
is turned on in its permissions. When an existing destination
fileâs executability differs from that of the corresponding
source file, rsync modifies the destination fileâs permissions
as follows:
o To make a file non-executable, rsync turns off all its
âxâ permissions.
o To make a file executable, rsync turns on each âxâ per-
mission that has a corresponding ârâ permission enabled.
If --perms is enabled, this option is ignored.
-t, --times
This tells rsync to transfer modification times along with the
files and update them on the remote system. Note that if this
option is not used, the optimization that excludes files that
have not been modified cannot be effective; in other words, a
missing -t or -a will cause the next transfer to behave as if it
used -I, causing all files to be updated (though the rsync algo-
rithm will make the update fairly efficient if the files havenât
actually changed, youâre much better off using -t).
-z, --compress
With this option, rsync compresses the file data as it is sent
to the destination machine, which reduces the amount of data
being transmitted -- something that is useful over a slow con-
nection.
Note that this option typically achieves better compression
ratios than can be achieved by using a compressing remote shell
or a compressing transport because it takes advantage of the
implicit information in the matching data blocks that are not
explicitly sent over the connection.