giovedì 30 giugno 2016

Split and transfer big file size from/to datacenters

Disassemble (on source node):
split -d -b 10G filename

Transfer with N thread (from source node):
nohup scp -i <path_to_keys>/mykey.rsa x00 root@ip_target_node:/dest &
nohup scp -i <path_to_keys>/mykey.rsa x01 root@ip_target_node:/dest &
nohup scp -i <path_to_keys>/mykey.rsa x02 root@ip_target_node:/dest &
nohup scp -i <path_to_keys>/mykey.rsa x03 root@ip_target_node:/dest &
nohup scp -i <path_to_keys>/mykey.rsa x04 root@ip_target_node:/dest &
nohup scp -i <path_to_keys>/mykey.rsa x05 root@ip_target_node:/dest &
(may require more threads if there are more xnn files)

Reassemble (on target node):
nohup cat x* > filename

1 commento: