Backup Kernel Patch

Seems I am the devil because I didn’t put some code up that I said I would.  So here it is.  This is a patch to the 2.6.27 kernel that outputs all the files that have been modified, chmod, created, etc.  Basically, all the files that would need to be backed up WITHOUT having to do a full system scan of a block device(Hard drive, etc).  You can then use that list to know what files to back up.  The backing up of the files is not part of this patch.  That portion is the easy part.  Exporting the files from the kernel was the harder part, and that is what this patch does.
There is also no hashing, sorting, or filtering of any kind so duplicate files could be reported – meaning a file could be modified and then before you backed it up become modified again.  In that case it would output the file twice.  Simply, remove the double entries to solve this issue.

If you have issues with this patch please let me know, but please don’t ask for feature enhancements or code changes unless it is critical to the use of the patch.  We use a modified and updated version of this patch in house with a little different functionality and don’t have a lot of resources to keep updating a patch that we no longer use.  Also, please don’t take that to mean this patch doesn’t work well, because it is fantastic for backup purposes.   I only mention that we use a different version because of other functionality that we needed for a different purpose that we have combined with this patch.

Here is the link to this patch –

http://www.mattheaton.com/backup-patch.rtf

There are now 2 on/off switches:
/proc/sys/fs/bh_logging_user and
/proc/sys/fs/bh_logging_root

They are off by default so there should be no affect on the systems until they are turned on.

To turn them on simply echo 1 to the proper file as shown below –

echo 1 > /proc/sys/fs/bh_logging_user
echo 1 > /proc/sys/fs/bh_logging_root

It also requires a change to /etc/syslog.conf to direct the messages somewhere.

Here’s the addition I made on bhtest:
kern.debug    /var/log/modified

Thanks,
Matt Heaton / President Bluehost.com

12 Responses to “Backup Kernel Patch”

  1. Byron Clark says:

    Looks like an interesting patch, but it would be far more useful just to post the patch instead of the patch wrapped in an rtf doc.

  2. […] Source posted under Web Host […]

  3. Marian Marinov says:

    Hello Matt,
    Can I ask why you did all this in the kernel when you could easily do it in the user space with a simple inotify daemon?
    As far as I know syslog will miss some lines during high system loads while a process with high priority and no direct writing to disk will not.

  4. matt says:

    Inotify is SLOW SLOW SLOW. You have to assign it to a directory at startup and wait for it to process all those files before it is ready to go. On our servers with tens of millions of files on them it takes Inotify almost 4 hours just to be ready to start. That is not acceptable to us. This patch we have is immediate, and requires no startup penalty. We can’t see any problem with syslog at all, although our current version in house no longer uses syslog for other reasons.

    Thanks,
    Matt

  5. Paul says:

    This patch is racy:
    1. syslog may not get all the names, since kernel log buffer may overflow.
    2. you may do the backup at the moment of file rename. In this case, you may simply miss the file – backup system may think it was modified under some name, while it was recently renamed while you were starting the backup and you will no longer find the file under the old name.

  6. […] everything up at once Backup Kernel Patch

  7. Matt,
    Why you don’t put a web page for all this sources you post?
    And more importantly why are you not saying what is the exact license of these sources? Are they under GPL or for that matter any other Open Source Licence ?

  8. Shad Vick says:

    Matt – thank you. It’s rare to find a leader in the industry willing to help out the rest. I like your model of sharing with the competition. This is only good for the industry and will help evolve where shared hosting continues to go. Now we’re going to buy you out….hehe j/k. Later.

  9. nrg says:

    Beautiful patch, thx Matt!

Leave a Reply