使用 rclone 将 Google Drive 文件同步至 OneDrive

4 Comments

前几天上了博友「灵尘居」的 Office 365 车,1T OneDrive 到手,手机照片一直喂 Google AI,挺方便的,以前还会同步一份到家里的垃圾西数 NAS,不过并不太自动,后来就懒得弄了。

Google 虽然是大厂,但是数据只有一份,还是怕怕,多一个备份不是坏事。苹果 iCloud 不错,但是贼贵,况且我只有一个进过水的 SE……

那就 OneDrive 吧,Office 365 就有 1T,前阵子在老司机群看到博友合租 Office 365 家庭套餐,还挺划算的,于是就上车了。

原本打算以前的照片就不管了,新的照片直接手机上传 Google Photos 和 OneDrive。试了两天,OneDrive 在后台不能工作,并且开后台上传要允许始终使用定位。试着给了权限,OneDrive 还是不干活,锁屏一会就死掉。

Google 也没找到现成的方法,找到一个第三方服务「MultCloud」,看着可行,免费版每月 50G 流量,群里博友也推荐,不过怕开始流量不够,没有尝试。

继续找其他方法:ifttt,这个只能同步新增;买个大盘 Windows VPS,Google Drive 同步下来再上传到 OneDrive,这个太累,不便宜;Google Drive 同步下来工作电脑再上传到 OneDrive,这个更累,还要爱国流量……

想起在 RSS 订阅的博客看到过将 OneDrive 挂载到 VPS 当本地硬盘用,还可以自动上传文件到 OneDrive。翻了一下,这货是 rclone。

官网看了一下:rclone - rsync for cloud storage

Rclone is a command line program to sync files and directories to and from:Amazon Drive、Amazon S3、Backblaze B2、Box、Ceph、DigitalOcean Spaces、Dreamhost、Dropbox、FTP、Google Cloud Storage、Google Drive、HTTP、Hubic、Jottacloud、IBM COS S3、Memset Memstore、Mega、Microsoft Azure Blob Storage、Microsoft OneDrive、Minio、Nextcloud、OVH、OpenDrive、Openstack Swift、Oracle Cloud Storage、ownCloud、pCloud、put.io、QingStor、Rackspace Cloud Files、SFTP、Wasabi、WebDAV、Yandex Disk、The local filesystem

rclone 看样子可行,VPS 安装 rclone,连接 Google Drive 和 OneDrive,通过 VPS 将 Google Drive 中的相片同步到 OneDrive。

rclone 安装、连接网盘

rclone 安装

执行命令:curl https://rclone.org/install.sh | sudo bash

连接网盘

连接 Google Drive,执行命令进入配置菜单:rclone config

初次进入会有如下菜单提示:

#~:rclone config
No remotes found - make a new one
n) New remote -> 连接新远程网盘
s) Set configuration password -> 设置密码保护配置信息
q) Quit config -> 退出
n/s/q>

需要加密码保护配置信息就按 s 进入设置密码,若不需要,按 n 进入连接网盘。

n/s/q> n
name> gdrive -> 网盘名字,写一个方便自己识别的名字就好
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Alias for a existing remote
   \ "alias"
 2 / Amazon Drive
   \ "amazon cloud drive"
 3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
   \ "s3"
 4 / Backblaze B2
   \ "b2"
 5 / Box
   \ "box"
 6 / Cache a remote
   \ "cache"
 7 / Dropbox
   \ "dropbox"
 8 / Encrypt/Decrypt a remote
   \ "crypt"
 9 / FTP Connection
   \ "ftp"
10 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
11 / Google Drive
   \ "drive"
12 / Hubic
   \ "hubic"
13 / JottaCloud
   \ "jottacloud"
14 / Local Disk
   \ "local"
15 / Mega
   \ "mega"
16 / Microsoft Azure Blob Storage
   \ "azureblob"
17 / Microsoft OneDrive
   \ "onedrive"
18 / OpenDrive
   \ "opendrive"
19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
20 / Pcloud
   \ "pcloud"
21 / QingCloud Object Storage
   \ "qingstor"
22 / SSH/SFTP Connection
   \ "sftp"
23 / Webdav
   \ "webdav"
24 / Yandex Disk
   \ "yandex"
25 / http Connection
   \ "http"
Storage> 11 -> 11 是 Google Drive
Google Application Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id> -> 留空,按回车下一步就好
Google Application Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> -> 留空,按回车下一步就好
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Full access all files, excluding Application Data Folder.
   \ "drive"
 2 / Read-only access to file metadata and file contents.
   \ "drive.readonly"
   / Access to files created by rclone only.
 3 | These are visible in the drive website.
   | File authorization is revoked when the user deauthorizes the app.
   \ "drive.file"
   / Allows read and write access to the Application Data folder.
 4 | This is not visible in the drive website.
   \ "drive.appfolder"
   / Allows read-only access to file metadata but
 5 | does not allow any access to read or download file content.
   \ "drive.metadata.readonly"
scope> 2 -> 1 是拥有所有权限,包括删除;2 是只能访问所有文件,不能删除,按需要选择吧
ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders. (see docs).
Enter a string value. Press Enter for the default ("").
root_folder_id> -> 留空,按回车下一步就好
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press Enter for the default ("").
service_account_file> -> 留空,按回车下一步就好
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n -> 一般不需要高级设置,选择 n 就好
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n> n -> 如果是在 VPS 上连接,选 n 吧
If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=**************************************
Log in and authorize rclone for access
Enter verification code> **************zN5****vd5*******************bwZethESs
Configure this as a team drive?
y) Yes
n) No
y/n> n
--------------------
[remote]
client_id = 
client_secret = 
scope = drive
root_folder_id = 
service_account_file =
token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2018-09-16T13:57:58.955387075Z"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

至此 Google Drive 连接完成,可以执行命令检查一下是否真的能连接上:

#列出顶级文件夹,(gdrive 是前面第一步设置的名字,如果配置设置了密码会提示输入密码)
rclone lsd gdrive:

#列出所有文件
rclone ls gdrive:

连接 OneDrive,方法类似,命令进入配置:rclone config

#~:rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> onedrive
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Alias for a existing remote
   \ "alias"
 2 / Amazon Drive
   \ "amazon cloud drive"
 3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
   \ "s3"
 4 / Backblaze B2
   \ "b2"
 5 / Box
   \ "box"
 6 / Cache a remote
   \ "cache"
 7 / Dropbox
   \ "dropbox"
 8 / Encrypt/Decrypt a remote
   \ "crypt"
 9 / FTP Connection
   \ "ftp"
10 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
11 / Google Drive
   \ "drive"
12 / Hubic
   \ "hubic"
13 / JottaCloud
   \ "jottacloud"
14 / Local Disk
   \ "local"
15 / Mega
   \ "mega"
16 / Microsoft Azure Blob Storage
   \ "azureblob"
17 / Microsoft OneDrive
   \ "onedrive"
18 / OpenDrive
   \ "opendrive"
19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
20 / Pcloud
   \ "pcloud"
21 / QingCloud Object Storage
   \ "qingstor"
22 / SSH/SFTP Connection
   \ "sftp"
23 / Webdav
   \ "webdav"
24 / Yandex Disk
   \ "yandex"
25 / http Connection
   \ "http"
Storage> 17
Microsoft App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>
Microsoft App Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
Choose OneDrive account type?
 * Say b for a OneDrive business account -> b 商业版
 * Say p for a personal OneDrive account -> p 个人版
b) Business
p) Personal
b/p> p
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n
For this to work, you will need rclone available on a machine that has a web browser available.
Execute the following on your machine:
        rclone authorize "onedrive"
Then paste the result below:(需要在能打开浏览器的电脑上获取,方法见下面:OneDrive 授权码获取方法)
result>{"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-03T08:43:55.5290909+08:00"}
2018/09/03 07:47:11 ERROR : Failed to save new token in config file: section 'onedrive' not found
--------------------
[onedrive]
type = onedrive
token = {"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-01T08:43:55.5290909+08:00"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name                 Type
====                 ====
onedrive             onedrive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

OneDrive 授权码获取方法:

官网下载 rclone Windows 版,地址:https://rclone.org/downloads/

如 Rclone Download v1.43 Windows 64 位:https://downloads.rclone.org/v1.43/rclone-v1.43-windows-amd64.zip,解压到 C 盘根目录,文件夹重命名为 rc,然后参考下面方法即可获取授权。

Microsoft Windows [版本 10.0.14393]
(c) 2016 Microsoft Corporation。保留所有权利。
C:\Users\Administrator>cd c:\rc
c:\rc>rclone authorize "onedrive"
Choose OneDrive account type?
 * Say b for a OneDrive business account
 * Say p for a personal OneDrive account
b) Business
p) Personal
b/p> p
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
Paste the following into your remote machine --->
{"access_token":"***************########################################################################################################################**************************************************************************************************************************************************************************************************************$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$","expiry":"2018-09-01T01:43:55.5290909+08:00"}
<---End paste

然后将 { } 内的内容(包括{})复制到前面 result> 后面,回车即可。

同样,执行 ls 或者 lsd 命令检查是否连接成功:

#列出顶级文件夹,(gdrive 是前面第一步设置的名字,如果配置设置了密码会提示输入密码)
rclone lsd onedrive:

#列出所有文件
rclone ls onedrive:

定时执行脚本

rclone 命令大全

如果用来备份或者同步文件,用到的命令可能就两个:rclone_copy 和 rclone_sync。

rclone_copy 是复制,配合附加参数 --ignore-existing 可以假装增量备份;

rclone_sync 是源目录同步到目标目录,使两者一致,会删除目标目录多余内容,测试时可以加上 --dry-run,检查会复制和删除那些文件。

rclone 目前可用命令如下(有需要点击展开查看):▼显示

搬瓦工 VPS(KVM,加拿大机房,1核 CPU,512M 内存,11GB SSD,550G 流量)实测,2018-9-2 15:00 左右,开始执行 rclone copy 命令将 Google Drive 上的 Google Photos 复制到 OneDrive 上的 GooglePhotos 目录,2018-9-4 05:30 左右复制完毕,数据约 166G。

yum install screen
screen -S rclone #第一次同步大量数据使用 screen 会话尽可能保护 rclone 执行不被中断
rclone copy gdrive:GooglePhotos onedrive:GooglePhotos

人工粗略检查了一下,看样子是同步成功了。第一次同步因为数据量可能比较大,需要时间比较长,VPS 流量也会耗掉不少。我实测同步 166G,VPS 耗掉的流量大概是这个两倍。数据量大的话最好在流量充裕的 VPS 上进行。

第一次同步完成,就可以 VPS 上(每天/周/月,按需选择)定时执行脚本自动同步。可以选择使用 rclone_copy --ignore-existing 进行增量备份(没有实测,按官方说明是可以实现),或者使用 rclone sync 同步。

我希望 OneDrive 上的备份与 Google Drive 一致,所以每天定时执行 rclone sync gdrive:GooglePhotos onedrive:GooglePhotos

/usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1

配合 crontab 定时执行,比如每天 4:44 执行一次,日志写到 /root/rclone.log。执行 crontab -e,i,添加下面代码进去,按一下 Esc,然后输入 :wq 回车保存退出。

44 4 * * * /usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1

注:不需要日志的话把 >> /root/rclone.log 2>&1 去掉。

然后重启一下 crond 服务即可。(service crond restart)

注意:最好执行 service crond status 检查 crond 服务是否启动,service crond start 可手动启动,或者设置开机启动:chkconfig --level 35 crond on。Centos 7 x86_64 实测通过。

参考资料

矢澤にこ - 使用 rclone 将文件同步至 Google Dirve

看天博客 - rclone 定时自动备份 VPS 服务器上网站数据到网盘

Leonn - Rclone: 超好用的网盘 / VPS 数据同步、备份工具,支持 Google Drive

彩蛋:Vultr 充$5送$25,详情 | 搬瓦工年付$19补货,优惠码BWH26FXH3HIQ『点击直达』| 爱国:木弟子信息专页VPS信息
声明:除非注明,常阳时光文章均为原创
本文地址:https://cyhour.com/825/
转载时必须以链接形式注明原始出处。
Views: 315 Tags:  ,  ,  ,  , 

Comments:4

  1. 咱还是以本地存储为主,网络存储只是个辅助。你这也费劲的放国外也不嫌麻烦?

    2018.09.04 16:34 # 回复
    1楼

发表留言

彩蛋:Vultr 充$5送$25,详情 | 搬瓦工年付$19补货,优惠码BWH26FXH3HIQ『点击直达』| 爱国:木弟子信息专页VPS信息