* set ssh port of remote server
rsync -ahvz --progress --rsh='ssh -p 2222' user@192.168.1.1:/var/log/data ./
Rsync 10가지 사용 예제들
![[-]](http://www.joinc.co.kr/modules/moniwiki/imgs/plugin/arrup.png "[-]")
1 Rsync
2 Rsync의 기능상 장점들
- 원격 시스템으로 부터 파일을 효율적으로 복사하거나 동기화 할 수 있다.
- Link, device, 파일의 소유자와 그룹 권한(permissions)등 파일의 부가정보도 복사할 수 있다.
- scp보다 빠르다. rsync는 remote-update 프로토콜을 이용해서 차이가 있는 파일만 복사한다. 처음에는 모든 파일과 디렉토리를 복사하겠지만, 다음부터는 차이가 있는 파일만 복사하기 때문에 더 빠르고 효율적으로 작동한다.
- 데이터를 압축해서 송/수신하기 때문에 더 적은 대역폭을 사용한다.
3 Rsync의 동기화 알고리즘
3.1 파일 전송 결정
3.2 전송할 파일 부분의 결정
5 주요 옵션들
- -v : verbose
- -r : Recursively 데이터 복사
- -a : archive mode. 이 모드를 이용하면 심볼릭 링크, 파일 유저/그룹 권한, timestamp도 복사한다.
- -z : 파일 데이터 압축. 대역폭을 아낄 수 있다. 물론 시간도
6 rsync 설치하기
# yum install rsync # apt-get install rsync
7 테스트 환경
- 로컬 시스템 : 우분투 리눅스 데스크탑 13.04
- 원격 시스템 : 우분투 리눅스 서버 13.04
- Virtualbox : Virtualbox로 우분투 리눅스 서버를 띄운다음 테스트를 진행했다.
8 로컬 컴퓨터에서 파일과 디렉토리 복사 및 동기화
8.1 로컬 컴퓨터에서 파일 복사 및 동기화
# rsync -zvh backup.tar /tmp/backups/ backup.tar sent 6.56K bytes received 31 bytes 13.18K bytes/sec total size is 6.48K speedup is 0.98만약 목적지 디렉토리가 없다면, 자동으로 디렉토리를 만든다.
8.2 로컴 컴퓨터에서 디렉토리 복사 및 동기화
# rsync -avzh moniwiki/ /tmp/backups/ .... .... wikiseed/WikiSlide wikiseed/WikiWikiWeb wikiseed/WordIndex sent 2.19M bytes received 23.72K bytes 885.45K bytes/sec total size is 5.70M speedup is 2.57moniwiki 디렉토리를 다시 한번 복사해봤다. rsync에 대한 설명이 맞다면, 복사하는 파일이 없어야 할 거다.
# rsync -avzh moniwiki/ /tmp/backups/ sending incremental file list sent 27.16K bytes received 119 bytes 54.56K bytes/sec total size is 5.70M speedup is 208.89
# touch moniwiki/README.txt # rsync -avzh moniwiki/ /tmp/backups/ sending incremental file list ./ README.txt sent 27.22K bytes received 141 bytes 54.73K bytes/sec total size is 5.70M speedup is 208.23예상대로 잘 작동한다.
# echo "hello world" >> moniwiki/README.txt rsync -avzh moniwiki/ /tmp/backups/ sending incremental file list ./ README.txt sent 27.24K bytes received 141 bytes 54.76K bytes/sec total size is 5.70M speedup is 208.11
9 서버로부터 파일과 디렉토리를 복사 혹은 동기화 하기
9.1 로컬 서버에서 원격 서버로 디렉토리 복사하기
# rsync -avz moniwiki/ yundream@192.168.56.101:/home/yundream/backups ...... ...... wikiseed/WikiSlide wikiseed/WikiWikiWeb wikiseed/WordIndex sent 2189995 bytes received 23737 bytes 491940.44 bytes/sec total size is 5698359 speedup is 2.57
9.2 원격 서버의 디렉토리를 로컬 서버로 복사하기
# rsync -avzh yundream@192.168.56.101:/home/yundream/backups ./ backups/wikiseed/WikiSlide backups/wikiseed/WikiWikiWeb backups/wikiseed/WordIndex sent 23.74K bytes received 2.19M bytes 885.51K bytes/sec total size is 5.70M speedup is 2.57
10 SSH 위에서 rsync 사용
11 Rsync server 모드로 사용
11.1 Rsyn 데몬 프로그램 설정
# apt-get install xinetd
# cat /etc/xinetd.d/rsync
service rsync
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/bin/rsync
server_args = --daemon
log_on_failure += USERID
flags = IPv6
}
max connections = 2 log file = /var/log/rsync.log timeout = 300 [share] comment = Public Share path = /home/share read only = no list = yes uid = nobody gid = nogroup auth users = yundram secrets file = /etc/rsyncd.secrets
- 복사(동기화) 디렉토리로 /home/share를 선택했다.
- 읽기/쓰기 모두 가능하게 했다.
- 디렉토리에 대한 소유자는 nobody.nogroup로 했다. /home/share 소유자를 nobody.nogroup로 변경한다.
- 접속 허용 유저로 yundream을 추가했다.
- auth users에 대한 설정파일은 /etc/rsyncd.secrets에 저장한다.
# cat /etc/rsyncd.secrets yundream:mypassword
$ rsync yundream@192.168.56.101::share
# rsync -avz HelloWorld yundream@192.168.56.101::share Password: sending incremental file list HelloWorld/ HelloWorld/HelloWorld.desktop HelloWorld/HelloWorld.pro HelloWorld/HelloWorld.pro.user HelloWorld/HelloWorld.svg ... sent 17798 bytes received 252 bytes 7220.00 bytes/sec total size is 46543 speedup is 2.58
12 데이터 복사 진행 상황 확인하기
# rsync -avzh --progress yundream@192.168.56.101:/home/yundream/backups ./
......
......
backups/wikiseed/WikiWikiWeb
944 100% 1.64kB/s 0:00:00 (xfer#1225, to-check=1/1334)
backups/wikiseed/WordIndex
34 100% 0.06kB/s 0:00:00 (xfer#1226, to-check=0/1334)
sent 23.74K bytes received 2.19M bytes 885.51K bytes/sec
total size is 5.70M speedup is 2.57
13 include, exclude 옵션
# rsync -avz --exclude 'data' yundream@192.168.56.101:/home/yundream/backups ./
# rsync -avz --exclude '*.bak' yundream@192.168.56.101:/home/yundream/backups ./
14 delete 옵션
# touch test.txt # rsync -avz --delete yundream@192.168.56.101:/home/yundream/backups ./ receiving file list ... done deleting test.txt .... ....
15 전송할 파일의 최대 크기 정하기
# rsync -avz --max-size='10k' yundream@192.168.56.101:/home/yundream/backups ./
16 성공적으로 전송한 후 원본파일 지우기
# rsync --remove-source-files -zvh backup.tar /tmp/backups/ backup.tar sent 14.71M bytes received 31 bytes 4.20M bytes/sec total size is 16.18M speedup is 1.10 # ll backup.tar ls: backup.tar: No such file or directory
17 Bandwidth 제한 설정
# rsync --bwlimit=100 -avzhe ssh /var/lib/rpm/ root@192.168.56.101:/root/tmprpm/ root@192.168.0.100's password: sending incremental file list sent 324 bytes received 12 bytes 61.09 bytes/sec total size is 38.08M speedup is 113347.05
Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems. With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.
This article explains 10 basic and advanced usage of the rsync command to transfer your files remotely and locally in Linux based machines. You don’t need to be root user to run rsynccommand.
Some advantages and features of Rsync command
- It efficiently copies and sync files to or from a remote system.
- Supports copying links, devices, owners, groups and permissions.
- It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
- Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.
Basic syntax of rsync command
# rsync options source destination
Some common options used with rsync commands
- -v : verbose
- -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
- -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
- -z : compress file data
- -h : human-readable, output numbers in a human-readable format
Install rsync in your Linux machine
We can install rsync package with the help of following command.
# yum install rsync (On Red Hat based systems) # apt-get install rsync (On Debian based systems)
1. Copy/Sync Files and Directory Locally
Copy/Sync a File on a Local Computer
This following command will sync a single file on a local machine from one location to another location. Here in this example, a file name backup.tar needs to be copied or synced to/tmp/backups/ folder.
[root@tecmint]# rsync -zvh backup.tar /tmp/backups/ created directory /tmp/backups backup.tar sent 14.71M bytes received 31 bytes 3.27M bytes/sec total size is 16.18M speedup is 1.10
In above example, you can see that if the destination is not already exists rsync will create a directory automatically for destination.
Copy/Sync a Directory on Local Computer
The following command will transfer or sync all the files of from one directory to a different directory in the same machine. Here in this example, /root/rpmpkgs contains some rpm package files and you want that directory to be copied inside /tmp/backups/ folder.
[root@tecmint]# rsync -avzh /root/rpmpkgs /tmp/backups/ sending incremental file list rpmpkgs/ rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm rpmpkgs/nagios-3.5.0.tar.gz rpmpkgs/nagios-plugins-1.4.16.tar.gz sent 4.99M bytes received 92 bytes 3.33M bytes/sec total size is 4.99M speedup is 1.00
2. Copy/Sync Files and Directory to or From a Server
Copy a Directory from Local Server to a Remote Server
This command will sync a directory from a local machine to a remote machine. For example: There is a folder in your local computer “rpmpkgs” which contains some RPM packages and you want that local directory’s content send to a remote server, you can use following command.
[root@tecmint]$ rsync -avz rpmpkgs/ root@192.168.0.101:/home/ root@192.168.0.101's password: sending incremental file list ./ httpd-2.2.3-82.el5.centos.i386.rpm mod_ssl-2.2.3-82.el5.centos.i386.rpm nagios-3.5.0.tar.gz nagios-plugins-1.4.16.tar.gz sent 4993369 bytes received 91 bytes 399476.80 bytes/sec total size is 4991313 speedup is 1.00
Copy/Sync a Remote Directory to a Local Machine
This command will help you sync a remote directory to a local directory. Here in this example, a directory /home/tarunika/rpmpkgs which is on a remote server is being copied in your local computer in /tmp/myrpms.
[root@tecmint]# rsync -avzh root@192.168.0.100:/home/tarunika/rpmpkgs /tmp/myrpms root@192.168.0.100's password: receiving incremental file list created directory /tmp/myrpms rpmpkgs/ rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm rpmpkgs/nagios-3.5.0.tar.gz rpmpkgs/nagios-plugins-1.4.16.tar.gz sent 91 bytes received 4.99M bytes 322.16K bytes/sec total size is 4.99M speedup is 1.00
3. Rsync Over SSH
With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.
Also when we use rsync we need to provide the user/root password to accomplish that particular task, so using SSH option will send your logins in an encrypted manner so that your password will be safe.
Copy a File from a Remote Server to a Local Server with SSH
To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using “ssh” with “-e” option and perform data transfer.
[root@tecmint]# rsync -avzhe ssh root@192.168.0.100:/root/install.log /tmp/ root@192.168.0.100's password: receiving incremental file list install.log sent 30 bytes received 8.12K bytes 1.48K bytes/sec total size is 30.74K speedup is 3.77
Copy a File from a Local Server to a Remote Server with SSH
[root@tecmint]# rsync -avzhe ssh backup.tar root@192.168.0.100:/backups/ root@192.168.0.100's password: sending incremental file list backup.tar sent 14.71M bytes received 31 bytes 1.28M bytes/sec total size is 16.18M speedup is 1.10
4. Show Progress While Transferring Data with rsync
To show the progress while transferring the data from one machine to a different machine, we can use ‘–progress’ option for it. It displays the files and the time remaining to complete the transfer.
[root@tecmint]# rsync -avzhe ssh --progress /home/rpmpkgs root@192.168.0.100:/root/rpmpkgs
root@192.168.0.100's password:
sending incremental file list
created directory /root/rpmpkgs
rpmpkgs/
rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm
1.02M 100% 2.72MB/s 0:00:00 (xfer#1, to-check=3/5)
rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm
99.04K 100% 241.19kB/s 0:00:00 (xfer#2, to-check=2/5)
rpmpkgs/nagios-3.5.0.tar.gz
1.79M 100% 1.56MB/s 0:00:01 (xfer#3, to-check=1/5)
rpmpkgs/nagios-plugins-1.4.16.tar.gz
2.09M 100% 1.47MB/s 0:00:01 (xfer#4, to-check=0/5)
sent 4.99M bytes received 92 bytes 475.56K bytes/sec
total size is 4.99M speedup is 1.005. Use of –include and –exclude Options
These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.
Here in this example, rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory.
[root@tecmint]# rsync -avze ssh --include 'R*' --exclude '*' root@192.168.0.101:/var/lib/rpm/ /root/rpm root@192.168.0.101's password: receiving incremental file list created directory /root/rpm ./ Requirename Requireversion sent 67 bytes received 167289 bytes 7438.04 bytes/sec total size is 434176 speedup is 2.59
6. Use of –delete Option
If a file or directory not exist at the source, but already exists at the destination, you might want to delete that existing file/directory at the target while syncing .
We can use ‘–delete‘ option to delete files that are not there in source directory.
Source and target are in sync. Now creating new file test.txt at the target.
[root@tecmint]# touch test.txt [root@tecmint]# rsync -avz --delete root@192.168.0.100:/var/lib/rpm/ . Password: receiving file list ... done deleting test.txt ./ sent 26 bytes received 390 bytes 48.94 bytes/sec total size is 45305958 speedup is 108908.55
Target has the new file called test.txt, when synchronize with the source with ‘–delete‘ option, it removed the file test.txt.
7. Set the Max Size of Files to be Transferred
You can specify the Max file size to be transferred or sync. You can do it with “–max-size” option. Here in this example, Max file size is 200k, so this command will transfer only those files which are equal or smaller than 200k.
[root@tecmint]# rsync -avzhe ssh --max-size='200k' /var/lib/rpm/ root@192.168.0.100:/root/tmprpm root@192.168.0.100's password: sending incremental file list created directory /root/tmprpm ./ Conflictname Group Installtid Name Provideversion Pubkeys Requireversion Sha1header Sigmd5 Triggername __db.001 sent 189.79K bytes received 224 bytes 13.10K bytes/sec total size is 38.08M speedup is 200.43
8. Automatically Delete source Files after successful Transfer
Now, suppose you have a main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server.
So, will you wait for transfer to complete and then delete those local backup file manually? Of Course NO. This automatic deletion can be done using ‘–remove-source-files‘ option.
[root@tecmint]# rsync --remove-source-files -zvh backup.tar /tmp/backups/ backup.tar sent 14.71M bytes received 31 bytes 4.20M bytes/sec total size is 16.18M speedup is 1.10 [root@tecmint]# ll backup.tar ls: backup.tar: No such file or directory
9. Do a Dry Run with rsync
If you are a newbie and using rsync and don’t know what exactly your command going do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.
Use of this option will not make any changes only do a dry run of the command and shows the output of the command, if the output shows exactly same you want to do then you can remove ‘–dry-run‘ option from your command and run on the terminal.
root@tecmint]# rsync --dry-run --remove-source-files -zvh backup.tar /tmp/backups/ backup.tar sent 35 bytes received 15 bytes 100.00 bytes/sec total size is 16.18M speedup is 323584.00 (DRY RUN)
10. Set Bandwidth Limit and Transfer File
You can set the bandwidth limit while transferring data from one machine to another machine with the the help of ‘–bwlimit‘ option. This options helps us to limit I/O bandwidth.
[root@tecmint]# rsync --bwlimit=100 -avzhe ssh /var/lib/rpm/ root@192.168.0.100:/root/tmprpm/ root@192.168.0.100's password: sending incremental file list sent 324 bytes received 12 bytes 61.09 bytes/sec total size is 38.08M speedup is 113347.05
Also, by default rsync syncs changed blocks and bytes only, if you want explicitly want to sync whole file then you use ‘-W‘ option with it.
[root@tecmint]# rsync -zvhW backup.tar /tmp/backups/backup.tar backup.tar sent 14.71M bytes received 31 bytes 3.27M bytes/sec total size is 16.18M speedup is 1.10
That’s all with rsync now, you can see man pages for more options. Stay connected with Tecmintfor more exciting and interesting tutorials in future. Do leave your comments and suggestions.
'System > Linux' 카테고리의 다른 글
| linux - grub2 config (0) | 2014.02.15 |
|---|---|
| linux - rsh set up in fedora (0) | 2013.12.08 |
| linux - 특정 디렉토리 및 파일만 chmod 실행 (0) | 2013.11.08 |
| debian - dpkg (0) | 2013.11.07 |
| linux - chmod 700 변경하기 (0) | 2013.10.31 |
