abarbarian Posted April 28, 2013 Posted April 28, 2013 (edited) https://wiki.archlin...ckup_with_rsync Well after me near disaster I thought it best to do a backup. So I followed the ARCH WIKI made a script called " phew.sh " placed it in a newly made folder called " ~/Scripts " (/home/me/Scripts) and followed the instructions, While the system is running, open up a terminal and run (as root): # ~/Scripts/phew.sh /some/destination So I was quite surprised that me terminal tells me there is no " ~/Scripts/phew.sh ". However " # /home/me/Scripts/phew.sh /some/destination " does exist and the backup is now hopefully running. So am I doing something wrong with " # ~/Scripts/phew.sh /some/destination " or is the Arch Wiki wrong ? Edited April 28, 2013 by abarbarian Quote
amenditman Posted April 28, 2013 Posted April 28, 2013 (edited) So I was quite surprised that me terminal tells me there is no " ~/Scripts/phew.sh ". However " # /home/me/Scripts/phew.sh /some/destination " does exist and the backup is now hopefully running. So am I doing something wrong with " # ~/Scripts/phew.sh /some/destination " or is the Arch Wiki wrong ? The ~/ is the same as your user /home directory. The only thing you did differently is run the command as root. Did you change the permissions on the script to be able to run it as user. Usually, <chmod a+x phew.sh> - executable by any. If you don't want just anybody to be able to run the script, but you want to run it as user, make sure who is the owner. root or your user. If root is the owner (you created the script as root) then as root, <chown user phew.sh> and put your username as user. Then <chmod 750 /home/me/Scripts/phew.sh> will make it so that the owner (you) can read, write, execute; the group can read, execute; and everyone else can go slice bread. Edited April 28, 2013 by amenditman Quote
securitybreach Posted April 28, 2013 Posted April 28, 2013 Well for one you made the directory in ~/Scripts as root so therefore the folder is in /root/Scripts not /home/user/Script. So yes, the wiki entry is mistaken as the command should of been: # /home/user/Scripts/phew.sh /some/destination instead of: # ~/Scripts/phew.sh /some/destination since you made the directory in /home/user not /root/ Quote
securitybreach Posted April 28, 2013 Posted April 28, 2013 Also, if you are just backing up certain folders; I would use this instead: https://wiki.archlin...utomated_backup I have been using that for years and it works beautifully: comhack@Cerberus ~ % su Password: root@Cerberus /home/comhack # crontab -l 00 15 * * * rsync -ar --delete /home/comhack/Videos /home/comhack/Music /home/comhack/Android /home/comhack/Documents /home/comhack/Downloads /home/comhack/Pictures /home/comhack/Shared /MEDIA &> /dev/null Quote
securitybreach Posted April 28, 2013 Posted April 28, 2013 I just edited the page to reflect the correction. Quote
abarbarian Posted April 28, 2013 Author Posted April 28, 2013 (edited) Thanks Josh I looked at your script before in other threads.I'm after a full backup though at the moment. Guys I followed the wiki I pretty sure I did. Anyway I had another run through and did, $ cd ~/Scripts $ nano phew.sh An copy/pasted the script given with a few alterations, All of the above is as USER right. Then I followed the wiki, $ chmod +x phew.sh which gives me $ chmod +x phew.sh chmod: changing permissions of 'phew.sh': Operation not permitted Which means that the wiki is wrong and this " $ chmod +x phew.sh " will not work. Am I right on this or am I going crazy. Edited April 28, 2013 by abarbarian Quote
securitybreach Posted April 28, 2013 Posted April 28, 2013 The command chmod +x filename, basically just makes it executable and it will work if you have the correct permissions to do it. To check permissions, run: ls -al ~/Scripts/phew.sh 1 Quote
amenditman Posted April 28, 2013 Posted April 28, 2013 You have to be root to run chmod, even on a script that is yours. Should be: #chmod +x phew.sh Quote
securitybreach Posted April 28, 2013 Posted April 28, 2013 You have to be root to run chmod, even on a script that is yours. Should be: #chmod +x phew.sh You do not have to be root to use chmod. I just tested it with one of my scripts. That said, it would bypass your issue if you did it as root. Quote
abarbarian Posted April 29, 2013 Author Posted April 29, 2013 Yup I understand about permissions somewhat. I have had the script running but it refuses to exclude any folders and files. $ cd ~/Scripts $ nano backup.sh $ chmod +x backup.sh I tried the three commands after deleting the script and starting again and it gives me ," -rwxr-xr-x 1 bloodaxe user " So owner group and others can all execute the script. So it look like I need to apologise to the Arch Wiki as it looks like I made a boo boo. However the script is not excluding folders and files. I'll have to have a look at it tomorrow as all this stress has worn me out. Ta fer yer patience guys. 1 Quote
V.T. Eric Layton Posted April 29, 2013 Posted April 29, 2013 Why make things so difficult? Shut your computer off. Reboot with your Porteus flash drive stuck in that hole in the front. Once Porteus has booted, open a terminal, su to root, type "grsync" w/o the "". When that fabulous little graphic front end for rsync opens, set the source and the destination properly . For example: source -- /dev/sda1/ destination -- /dev/sdb1 *Note: if you want a true mirrored backup, you must have that all-important following / you see above in red. If you forget it, you'll get a mirror within a folder called /dev/sda1 on the sdb1 partition. That won't be bootable. However, it would still be a valid mirror backup that you could sync back to the original partition. I prefer 100% bootable root partition mirrors myself. Anywho, you'll need to make sure you have some of the options in grsync set correctly, too. You'll want to copy all hard links and sym-links as-is. There are some other critical options you'll need, but I'd have to boot into Porteus to check for you. Let me know if you want me to do that. The first rsync will take a while, depending on your system's speed. The latter rsyncs will go much faster because of rsync's characteristic ability to copy over only changed files, rather than everything again. Rsync is a very cool little tool. I rsync my Slackware /(root) and /home every Sunday... manually, not with cron. 2 Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 Why make things so difficult. Shut your computer off. Reboot with your Porteus flash drive stuck in that hole in the front. Once Porteus has booted, open a terminal, su to root, type "grsync" w/o the "". When that fabulous little graphic front end for rsync opens, set the source and the destination properly (your destination partition). For example: source -- /dev/sda1/ | destination -- /dev/sdb1 *Note: if you want a true mirrored backup, you must have that all-important following / you see above in red. If you forget it, you'll get a mirror within a folder called /dev/sda1 on the sdb1 partition. That won't be bootable. However, it would still be a valid mirror backup that you could sync back to the original partition. I prefer 100% bootable root partition mirrors myself. Anywho, you'll need to make sure you have some of the options in grsync set correctly, too. You'll want to copy all hard links and sym-links as-is. There are some other critical options you'll need, but I'd have to boot into Porteus to check for you. Let me know if you want me to do that. The first rsync will take a while, depending on your system's speed. The latter rsyncs will go much faster because of rsync's characteristic ability to copy over only changed files, rather than everything again. Rsync is a very cool little tool. I rsync my Slackware /(root) and /home every Sunday... manually, not with cron. Hmm, that sounds interesting.... So what requirements are needed to do that? A partition the size of your / directory? And does it just copy the / partition or will it try to copy the mounts as well: /home /MEDIA /backup etc.? Quote
V.T. Eric Layton Posted April 29, 2013 Posted April 29, 2013 I don't remember all the settings. I'd have to boot into Porteus and start up grsync to let you know. I'll do it tomorrow. I'm outta' here shortly. Anyway, it creates a 100% perfect mirror of my root and home partitions. It is 100% bootable image of my primary Slackware installation. The partitions that I'm rsync'ing to are the exact same size as the originals. They don't have to be, though. They just have to be big enough. For my /(root): source: /dev/sda1 destination: /dev/sda5 For my /home: source: /dev/sda2/ destination: /dev/sda6 In my case, I'm sync-ing to the same drive. For a true backup, you'd want to sync to a separate drive, of course. The only thing I have to do to make the mirror bootable is adjust the /(root) and /home entries in the fstab. That's it. Easy-peasy! Be patient. I'll post the complete grsync settings tomorrow sometime. G'night. AH! I just remembered. I already posted how I did this here --> http://forums.scotsnewsletter.com/index.php?showtopic=60642&hl=grsync Have fun! 1 Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 Well the reason I ask is because I would only want to clone my / partition as the other partitions mounted under / are ~4tb in size: Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 Never mind, I get it... it only clones the partition you selected so it wouldn't clone the other mount points? Right? Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 Ok it worked beautifully but grsync is not installed by default and you must install it on Porteus. It took me about 15 minutes to copy 16gb, not bad /dev/sda2 46G 16G 29G 35% / /dev/sdb3 46G 16G 29G 35% /mnt/usb I am not clear about your last part though: - After mirroring completed, to make the mirror bootable as the original, fstab must be edited to correct for source/destination partition number discrepancies Quote
V.T. Eric Layton Posted April 29, 2013 Posted April 29, 2013 My primary Slackware fstab looks like this: /dev/sda7 swap swap defaults 0 0 /dev/sda1 / ext3 defaults 1 1 /dev/sda2 /home ext3 defaults 1 2 /dev/sr0 /mnt/dvd_sr0 auto noauto,user,ro 0 0 /dev/sr1 /mnt/dvd_sr1 auto noauto,user,ro 0 0 /dev/fd0 /mnt/floppy auto rw,noauto,user,sync 0 0 /dev/sdd4 /mnt/zip vfat rw,noauto,user,sync 0 0 /dev/sde1 /mnt/usb auto rw,noauto,user,sync 0 0 /dev/sda8 /home/vtel57/vtel57_archives ext2 defaults 0 2 /dev/sdc1 /home/vtel57/vtel57_backups ext2 defaults 0 2 /dev/sdc7 /home/vtel57/vtel57_common vfat rw,gid=users,uid=vtel57 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 proc /proc proc defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 On the newly mirrored partitions, / and /home are not sda1 and sda2. They are sda5 and sda6, so I have to make that change in the fstab in order to make the mirrored Slackware bootable. And that's correct about the mounts. Rsync doesn't care about the active mounts. It only mirrors the source partition itself. You had to install grsync in Porteus? Hmm... I don't remember doing that. How old is your Porteus version? Oh, and when you mirror the next time, it'll be a lot quicker because rsync will only copy the changed items. Quote
sunrat Posted April 29, 2013 Posted April 29, 2013 The easiest way to exclude files from rsync is to list them in an exclude file and specify it in the command eg - my music backup:- rsync -av --delete --exclude-from=/mnt/music/rsync-exclude /mnt/music/ /mnt/backup1/Music-sdc And the rsync-exclude file:- lost+found .Trash-1000 2 Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 You had to install grsync in Porteus? Hmm... I don't remember doing that. How old is your Porteus version? I used Porteus-v2.0-x86_64.iso. Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 The easiest way to exclude files from rsync is to list them in an exclude file and specify it in the command eg. rsync -avz --exclude-from 'exclude-list.txt' source/ destination/ I am pretty familiar with rsync, I had just never used Grsync with Porteus Quote
V.T. Eric Layton Posted April 29, 2013 Posted April 29, 2013 That may be the difference. I'm using Porteus-XFCE-v1.2-i486. Grsync is a Gnome app, so it comes with the Xfce version of Porteus, I guess. Quote
securitybreach Posted April 29, 2013 Posted April 29, 2013 That may be the difference. I'm using Porteus-XFCE-v1.2-i486. Grsync is a Gnome app, so it comes with the Xfce version of Porteus, I guess. Ah, yeah that explains it. I used the Standard Edition(KDE). Quote
abarbarian Posted April 29, 2013 Author Posted April 29, 2013 (edited) The easiest way to exclude files from rsync is to list them in an exclude file and specify it in the command eg - my music backup:- rsync -av --delete --exclude-from=/mnt/music/rsync-exclude /mnt/music/ /mnt/backup1/Music-sdc And the rsync-exclude file:- lost+found .Trash-1000 Good tip sunrat. I was going to try that out at some stage as I found similar examples with other's here, http://www.thegeekst...es-and-folders/ An good stuff Eric, will give that a try sometime. Meanwhile back on topic. As I said I tried to follow the Wiki. So I copied and pasted the script and made some adjustments. Like swapping the " $1 " for " /my/destination/ " and adding a few folders to the " --exclude " list, original, rsync -aAXv /* $1 --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/home/*/.gvfs,/var/lib/pacman/sync/*} FINISH=$(date +%s) echo "total time: $(( ($FINISH-$START) / 60 )) minutes, $(( ($FINISH-$START) % 60 )) seconds" touch $1/"Backup from $(date '+%A, %d %B %Y, %T')" altered, rsync -aAXv /* /media/sdf2-ata-SAMSUNG_HD103SJ_/Archphew/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/home/*/.gvfs,/var/lib/pacman/sync/*,/home/*/.thumbnails/*,/home/*/.cache/chromium/*,/home/*/.mozilla/firefox/*.default/Cache/*,/home/*/dwhelper/*,/home/*/Downloads/*} FINISH=$(date +%s) echo "total time: $(( ($FINISH-$START) / 60 )) minutes, $(( ($FINISH-$START) % 60 )) seconds" touch /media/sdf2-ata-SAMSUNG_HD103SJ_/Archphew/"Backup from $(date '+%A, %d %B %Y, %T')" I also made an alias in /etc/bash.bashrc. So I run the script and the exclusions do not work. The ecluded folders still get copied. So is the script wrong ? Or am I missing something ? I tried sunrats suggestion of a exclude file, and so far that looks to be working just fine. The reason I am banging on about this is there is a guide in the Arch Wiki and it does seem to have some holes in it. If that is the case then I would like to be able to fix the guide so that some other poor muppet like me can follow the guide and get a result. Thoughts welcome. Edited April 29, 2013 by abarbarian Quote
abarbarian Posted April 29, 2013 Author Posted April 29, 2013 Well it looks like I have a backup and using sunrats suggestion it seems to be almost perfect. Apart from this, rsync: set_acl: sys_acl_set_file(var/log/journal/b5b4a6c0277be371da753ef9000005ae/.user-1000.journal.kpbTKh, ACL_TYPE_ACCESS): Operation not supported (95) sent 24089809025 bytes received 3252385 bytes 9215169.79 bytes/sec total size is 24077561215 speedup is 1.00 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9] total time: 43 minutes, 35 seconds /usr/local/sbin/phew.sh: line 22: $'+%A, %d %B \n%Y, %T': command not found from the guide, /home/*/.gvfs should be added too, so that it won't complain at the end that "some files/attrs were not transferred". Trouble is I do not have a ".gvfs" so why not ? How do I get one ? Will me backup still boot without one ? The error in green I think I can solve meself. A pox on all net guides ! Everyone should run exactly the same hardware as me then I would not have any problems. Just wait till I rule the world. I'll soon have things tiketty boo. Still I am making progress. I have a backup of sorts done from the cli not with some gui Now where is me Porteus usb stick Quote
sunrat Posted April 29, 2013 Posted April 29, 2013 I hate rsync questions. They always get me started on trying to learn more about it, but as the man is over 3200 lines it often becomes a marathon. For your acl error, look into the behaviour of the -A option and possibly the --fake-super option. Just a guess as I haven't used these. I think the syntax may be wrong for exclude in your script. It looks much easier to get an exclude file working. From man rsync: Note also that the --filter, --include, and --exclude options take one rule/pattern each. To add multiple ones, you can repeat the options on the command-line, use the merge-file syntax of the --filter option, or the --include-from/--exclude-from options. Quote
abarbarian Posted April 29, 2013 Author Posted April 29, 2013 I hate rsync questions. They always get me started on trying to learn more about it, but as the man is over 3200 lines it often becomes a marathon. For your acl error, look into the behaviour of the -A option and possibly the --fake-super option. Just a guess as I haven't used these. I think the syntax may be wrong for exclude in your script. It looks much easier to get an exclude file working. From man rsync: Well your idea to make a exclusion file worked just fine. So there could be something wrong with the script. Maybe I'll see if I can contact the person who posted it. Quote
abarbarian Posted February 1, 2014 Author Posted February 1, 2014 Well I took the plunge. Did the Full Backup with rsync,as per the wiki, used an exclusion text,an alias, to a external hdd. Everything seemed to go ok. Had some error messages they seemed to relate to systemd logs I think. Then I did a Full Backup to a new ssd I came across cheap. I bought the ssd to play with bcache but a wedding album has halted that dream for the moment. As I used the command line for this I used the drives uid for identification. Pressed enter and of it shot faster than the graphics in The Matrix. Super I thought, until the flying text stopped at some ssh file, sat there for ages. I panicked and opened up spacefm to see what was going on, could see no action and some odd files on the new drive,lost and lonely them files looked.So I killed em to put em out of their misery.Noticed that the ssd was showing as /media/sdg1-ata-KINGSTON_SV300S37A60G_50026B723A0BC474. This looked similar to what I have in my rsync script so I entered that as an identifier. An of it shot again,until the flying text stopped at some ssh file, sat there for ages. This time I did not panic, instead I looked at my cpu monitor which showed activity.So I let it run. When rsync had completed I followed the wiki for editing fstab,I use uid's as I find it easier to keep track of different drives.Did the grub updating as per the wiki. Crossed me fingers and shutdown. Rebooted and instead of a nice large text grub I found a tiny text grub squashed in one corner of the screen. I only have a five second slot to choose alternate boot options and by the time I had registered that there seemed to be twice as many boot options as normal I booted into me old Arch. Rebooted and this time saw the new boot option, scrolled down and hit enter, keeping me fingers crossed. Before I could blink I was at a login prompt. Yippeee !!! So I am posting this from me old tech slow ssd and am dead impressed. Now you may wonder why I abandoned the bcache dream. I thought that as I had a load of pictures to fiddle about with in Gimp the speed bump would be good. Also I thought rsync'ing would be a breeze and only take a couple of hours. Wrong ! wrong wrong. It has taken me all evening as I am a slow reader. I'd forgotten that I had forgotten so much about a subject I know so little about. Still the quick test I ran with Gimp shows some considerable time saving. 1 Quote
V.T. Eric Layton Posted February 1, 2014 Posted February 1, 2014 Good for you. Welcome to the rsync club. I do this every Sunday with my Slackware primary installation on my main system. I mirror the entire /(root) and /home partitions onto a secondary internal hdd. I cheat a little, though, as I use grsync, a graphic front-end for rsync. 1 Quote
abarbarian Posted February 1, 2014 Author Posted February 1, 2014 Good for you. Welcome to the rsync club. Is the beer any good at the club ? Every Sunday hmmm. Who a good geek then ! There is me evangelising and laughing at folks not doing backups, smugly superior an all that. When I looked at the date on my one and only backup it shows it is over six months old Not much use apart from being a museum exhibit Ha. I dreamt about os-prober, well more like had a nightmare about it. So I installed it ON THE SSD and ran it( ran it from the original Arch yesterday), which I am assuming is what the wiki means, though it is a tad unclear to a inexperienced chap. Os-prober did not seem to complete properly in the terminal.However a new "grub.conf.new" has been generated and this time it has the right uuid for the ssd. Will that have fixed me funny grub screen ? Time will tell Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.