[Novalug] Fwd: [rdiff-backup-users] why does exclude move files to increments?

Brandon Saxe brandon20va@yahoo.com
Fri Aug 31 10:39:46 EDT 2007


I was wondering if anybody on this list could shed
some light on this as well.

TIA!

--- Brandon Saxe <brandon20va@yahoo.com> wrote:

> Date: Fri, 31 Aug 2007 07:30:31 -0700 (PDT)
> From: Brandon Saxe <brandon20va@yahoo.com>
> To: rdiff-backup-users@nongnu.org
> Subject: [rdiff-backup-users] why does exclude move
> files to increments?
> 
> I want to use rdiff-backup to do incrementals, but I
> also only want to process files that have changed in
> the last 24 hours. How come when I use exclude
> directives that the directories excluded in
> subsequent
> backups are moved to incrementals? This is causing
> huge amounts of unnecessary I/O on my disk. I want
> to
> create a baseline rdiff-backup and then only push
> updates daily by using mtime. I have 300GB worth of
> data and it takes an insane 16 hours to run. That's
> fine for a baseline,but not an incremental. Does
> rdiff-backup to a file compare on every single run?
> I
> just want to compare files that changed in last day.
> Here is an example of how just trying to process
> changed files turns out:
> 
> I have a test directory tree:
> /test
> |-- dir1
> |   |-- file1
> |   |-- file2
> |   `-- file3
> |-- dir2
> |   |-- file1
> |   `-- file2
> `-- file1
> 
> I create my baseline rdiff-backup with the
> following:
> rdiff-backup --print-statistics
> --exclude-other-filesystems --exclude-sockets
> --exclude-device-files --exclude-fifos --include
> /test
> --exclude '**' / /backup/testing
> 
> This results in the following tree:
> /backup/testing/
> |-- rdiff-backup-data
> |   |-- backup.log
> |   |-- chars_to_quote
> |   |--
> current_mirror.2007-08-31T10:25:11-04:00.data
> |   |-- error_log.2007-08-31T10:25:11-04:00.data.gz
> |   |--
>
extended_attributes.2007-08-31T10:25:11-04:00.snapshot.gz
> |   |--
> file_statistics.2007-08-31T10:25:11-04:00.data.gz
> |   |-- increments
> |   |--
>
mirror_metadata.2007-08-31T10:25:11-04:00.snapshot.gz
> |   `--
> session_statistics.2007-08-31T10:25:11-04:00.data
> `-- test
>     |-- dir1
>     |   |-- file1
>     |   |-- file2
>     |   `-- file3
>     |-- dir2
>     |   |-- file1
>     |   `-- file2
>     `-- file1
> 
> Now when I run rdiff-backup on just those files that
> have changed, it moves the non-changed files to
> increments! Why? I am simulating the non-changed
> files
> by excluding dir2 from my processing. I only want to
> update the rdiff-backup tree for changed files based
> on mtime to avoid the processing and I/O caused by
> comparing all files' signatures. Here is my command
> for only processing changed files:
> find /test -mtime 0 | rdiff-backup
> --print-statistics
> --exclude-other-filesystems --exclude-sockets
> --exclude-device-files --exclude-fifos --exclude
> /test/dir2 --include-filelist-stdin --exclude '**' /
> /backup/testing
> 
> And here is the rdiff-backup resulting backup tree:
> /backup/testing
> |-- rdiff-backup-data
> |   |-- backup.log
> |   |-- chars_to_quote
> |   |--
> current_mirror.2007-08-31T10:28:04-04:00.data
> |   |-- error_log.2007-08-31T10:25:11-04:00.data.gz
> |   |-- error_log.2007-08-31T10:28:04-04:00.data.gz
> |   |--
>
extended_attributes.2007-08-31T10:25:11-04:00.snapshot.gz
> |   |--
>
extended_attributes.2007-08-31T10:28:04-04:00.snapshot.gz
> |   |--
> file_statistics.2007-08-31T10:25:11-04:00.data.gz
> |   |--
> file_statistics.2007-08-31T10:28:04-04:00.data.gz
> |   |-- increments
> |   |   |-- test
> |   |   |   |-- dir2
> |   |   |   |   |--
> file1.2007-08-31T10:25:11-04:00.snapshot.gz
> |   |   |   |   `--
> file2.2007-08-31T10:25:11-04:00.snapshot.gz
> |   |   |   `-- dir2.2007-08-31T10:25:11-04:00.dir
> |   |   `-- test.2007-08-31T10:25:11-04:00.dir
> |   |-- increments.2007-08-31T10:25:11-04:00.dir
> |   |--
>
mirror_metadata.2007-08-31T10:25:11-04:00.snapshot.gz
> |   |--
>
mirror_metadata.2007-08-31T10:28:04-04:00.snapshot.gz
> |   |--
> session_statistics.2007-08-31T10:25:11-04:00.data
> |   `--
> session_statistics.2007-08-31T10:28:04-04:00.data
> `-- test
>     |-- dir1
>     |   |-- file1
>     |   |-- file2
>     |   `-- file3
>     `-- file1
> 
> Why in the world did it do *anything* to the files I
> excluded? Is there any way to do this without
> operating on the entire backup set *every* time?
> 
> Thanks.
> --Brandon
> 
> 
> _______________________________________________
> rdiff-backup-users mailing list at
> rdiff-backup-users@nongnu.org
>
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL:
>
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
> 




More information about the Novalug mailing list