前几天上了博友「灵尘居」的 Office 365 车,1T OneDrive 到手,手机照片一直喂 Google AI,挺方便的,以前还会同步一份到家里的垃圾西数 NAS,不过并不太自动,后来就懒得弄了。
Google 虽然是大厂,但是数据只有一份,还是怕怕,多一个备份不是坏事。苹果 iCloud 不错,但是贼贵,况且我只有一个进过水的 SE……
那就 OneDrive 吧,Office 365 就有 1T,前阵子在老司机群看到博友合租 Office 365 家庭套餐,还挺划算的,于是就上车了。
原本打算以前的照片就不管了,新的照片直接手机上传 Google Photos 和 OneDrive。试了两天,OneDrive 在后台不能工作,并且开后台上传要允许始终使用定位。试着给了权限,OneDrive 还是不干活,锁屏一会就死掉。
Google 也没找到现成的方法,找到一个第三方服务「MultCloud」,看着可行,免费版每月 50G 流量,群里博友也推荐,不过怕开始流量不够,没有尝试。
继续找其他方法:ifttt,这个只能同步新增;买个大盘 Windows VPS,Google Drive 同步下来再上传到 OneDrive,这个太累,不便宜;Google Drive 同步下来工作电脑再上传到 OneDrive,这个更累,还要爱国流量……
想起在 RSS 订阅的博客看到过将 OneDrive 挂载到 VPS 当本地硬盘用,还可以自动上传文件到 OneDrive。翻了一下,这货是 rclone。
官网看了一下:rclone - rsync for cloud storage
Rclone is a command line program to sync files and directories to and from:Amazon Drive、Amazon S3、Backblaze B2、Box、Ceph、DigitalOcean Spaces、Dreamhost、Dropbox、FTP、Google Cloud Storage、Google Drive、HTTP、Hubic、Jottacloud、IBM COS S3、Memset Memstore、Mega、Microsoft Azure Blob Storage、Microsoft OneDrive、Minio、Nextcloud、OVH、OpenDrive、Openstack Swift、Oracle Cloud Storage、ownCloud、pCloud、put.io、QingStor、Rackspace Cloud Files、SFTP、Wasabi、WebDAV、Yandex Disk、The local filesystem
rclone 看样子可行,VPS 安装 rclone,连接 Google Drive 和 OneDrive,通过 VPS 将 Google Drive 中的相片同步到 OneDrive。
rclone 安装、连接网盘
rclone 安装
执行命令:curl https://rclone.org/install.sh | sudo bash
(貌似 rclone 升级也可以使用这个安装命令)
rclone 升级
rclone 1.55.0 版本开始可以使用 rclone selfupdate 升级。(旧版本 rclone 执行命令会提示 Fatal error: unknown command "selfupdate" for "rclone")
rclone selfupdate --check ## 只检查最新版本
rclone selfupdate --stable ## 更新到最新稳定版本
rclone version ## 检查当前 rclone 版本
连接网盘
连接 Google Drive,执行命令进入配置菜单:rclone config
初次进入会有如下菜单提示:
#~:rclone config No remotes found - make a new one n) New remote -> 连接新远程网盘 s) Set configuration password -> 设置密码保护配置信息 q) Quit config -> 退出 n/s/q>
需要加密码保护配置信息就按 s 进入设置密码,若不需要,按 n 进入连接网盘。
n/s/q> n
name> gdrive -> 网盘名字,写一个方便自己识别的名字就好
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Alias for a existing remote
\ "alias"
2 / Amazon Drive
\ "amazon cloud drive"
3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
\ "s3"
4 / Backblaze B2
\ "b2"
5 / Box
\ "box"
6 / Cache a remote
\ "cache"
7 / Dropbox
\ "dropbox"
8 / Encrypt/Decrypt a remote
\ "crypt"
9 / FTP Connection
\ "ftp"
10 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
11 / Google Drive
\ "drive"
12 / Hubic
\ "hubic"
13 / JottaCloud
\ "jottacloud"
14 / Local Disk
\ "local"
15 / Mega
\ "mega"
16 / Microsoft Azure Blob Storage
\ "azureblob"
17 / Microsoft OneDrive
\ "onedrive"
18 / OpenDrive
\ "opendrive"
19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
20 / Pcloud
\ "pcloud"
21 / QingCloud Object Storage
\ "qingstor"
22 / SSH/SFTP Connection
\ "sftp"
23 / Webdav
\ "webdav"
24 / Yandex Disk
\ "yandex"
25 / http Connection
\ "http"
Storage> 11 -> 11 是 Google Drive
Google Application Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id> -> 留空,按回车下一步就好
Google Application Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> -> 留空,按回车下一步就好
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Full access all files, excluding Application Data Folder.
\ "drive"
2 / Read-only access to file metadata and file contents.
\ "drive.readonly"
/ Access to files created by rclone only.
3 | These are visible in the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ "drive.file"
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ "drive.appfolder"
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ "drive.metadata.readonly"
scope> 2 -> 1 是拥有所有权限,包括删除;2 是只能访问所有文件,不能删除,按需要选择吧
ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders. (see docs).
Enter a string value. Press Enter for the default ("").
root_folder_id> -> 留空,按回车下一步就好
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Enter a string value. Press Enter for the default ("").
service_account_file> -> 留空,按回车下一步就好
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n -> 一般不需要高级设置,选择 n 就好
Remote config
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n> n -> 如果是在 VPS 上连接,选 n 吧
If your browser doesn't open automatically go to the following link: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=**************************************
Log in and authorize rclone for access
Enter verification code> **************zN5****vd5*******************bwZethESs
Configure this as a team drive?
y) Yes
n) No
y/n> n
--------------------
[remote]
client_id =
client_secret =
scope = drive
root_folder_id =
service_account_file =
token = {"access_token":"XXX","token_type":"Bearer","refresh_token":"XXX","expiry":"2018-09-16T13:57:58.955387075Z"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
至此 Google Drive 连接完成,可以执行命令检查一下是否真的能连接上:
#列出顶级文件夹,(gdrive 是前面第一步设置的名字,如果配置设置了密码会提示输入密码) rclone lsd gdrive:
#列出所有文件 rclone ls gdrive:
连接 OneDrive,方法类似,命令进入配置:rclone config
#~:rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> onedrive
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Alias for a existing remote
\ "alias"
2 / Amazon Drive
\ "amazon cloud drive"
3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
\ "s3"
4 / Backblaze B2
\ "b2"
5 / Box
\ "box"
6 / Cache a remote
\ "cache"
7 / Dropbox
\ "dropbox"
8 / Encrypt/Decrypt a remote
\ "crypt"
9 / FTP Connection
\ "ftp"
10 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
11 / Google Drive
\ "drive"
12 / Hubic
\ "hubic"
13 / JottaCloud
\ "jottacloud"
14 / Local Disk
\ "local"
15 / Mega
\ "mega"
16 / Microsoft Azure Blob Storage
\ "azureblob"
17 / Microsoft OneDrive
\ "onedrive"
18 / OpenDrive
\ "opendrive"
19 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
20 / Pcloud
\ "pcloud"
21 / QingCloud Object Storage
\ "qingstor"
22 / SSH/SFTP Connection
\ "sftp"
23 / Webdav
\ "webdav"
24 / Yandex Disk
\ "yandex"
25 / http Connection
\ "http"
Storage> 17
Microsoft App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>
Microsoft App Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
Choose OneDrive account type?
* Say b for a OneDrive business account -> b 商业版
* Say p for a personal OneDrive account -> p 个人版
b) Business
p) Personal
b/p> p
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n
For this to work, you will need rclone available on a machine that has a web browser available.
Execute the following on your machine:
rclone authorize "onedrive"
Then paste the result below:(需要在能打开浏览器的电脑上获取,方法见下面:OneDrive 授权码获取方法)
result>{"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-03T08:43:55.5290909+08:00"}
2018/09/03 07:47:11 ERROR : Failed to save new token in config file: section 'onedrive' not found
--------------------
[onedrive]
type = onedrive
token = {"access_token":"$$$$$$$$$$$$$$","expiry":"2018-09-01T08:43:55.5290909+08:00"}
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
onedrive onedrive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
OneDrive 授权码获取方法:
官网下载 rclone Windows 版,地址:https://rclone.org/downloads/
如 Rclone Download v1.43 Windows 64 位:https://downloads.rclone.org/v1.43/rclone-v1.43-windows-amd64.zip,解压到 C 盘根目录,文件夹重命名为 rc,然后参考下面方法即可获取授权。
Microsoft Windows [版本 10.0.14393]
(c) 2016 Microsoft Corporation。保留所有权利。
C:\Users\Administrator>cd c:\rc
c:\rc>rclone authorize "onedrive" -·-·- (PowerShell 可能需要使用此命令:.\rclone authorize "onedrive")
Choose OneDrive account type?
* Say b for a OneDrive business account
* Say p for a personal OneDrive account
b) Business
p) Personal
b/p> p
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
Paste the following into your remote machine --->
{"access_token":"***************########################################################################################################################**************************************************************************************************************************************************************************************************************$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$","expiry":"2018-09-01T01:43:55.5290909+08:00"}
<---End paste
然后将 { } 内的内容(包括{})复制到前面 result> 后面,回车即可。
同样,执行 ls 或者 lsd 命令检查是否连接成功:
#列出顶级文件夹,(gdrive 是前面第一步设置的名字,如果配置设置了密码会提示输入密码) rclone lsd onedrive:
#列出所有文件 rclone ls onedrive:
定时执行脚本
rclone 命令大全
如果用来备份或者同步文件,用到的命令可能就两个:rclone_copy 和 rclone_sync。
rclone_copy 是复制,配合附加参数 --ignore-existing 可以假装增量备份;
rclone_sync 是源目录同步到目标目录,使两者一致,会删除目标目录多余内容,测试时可以加上 --dry-run,检查会复制和删除那些文件。
rclone 目前可用命令如下(有需要点击展开查看):▼展开
- rclone about - Get quota information from the remote.
- rclone authorize - Remote authorization.
- rclone cachestats - Print cache stats for a remote
- rclone cat - Concatenates any files and sends them to stdout.
- rclone check - Checks the files in the source and destination match.
- rclone cleanup - Clean up the remote if possible
- rclone config - Enter an interactive configuration session.
- rclone copy - Copy files from source to dest, skipping already copied
- rclone copyto - Copy files from source to dest, skipping already copied
- rclone copyurl - Copy url content to dest.
- rclone cryptcheck - Cryptcheck checks the integrity of a crypted remote.
- rclone cryptdecode - Cryptdecode returns unencrypted file names.
- rclone dbhashsum - Produces a Dropbox hash file for all the objects in the path.
- rclone dedupe - Interactively find duplicate files and delete/rename them.
- rclone delete - Remove the contents of path.
- rclone deletefile - Remove a single file from remote.
- rclone genautocomplete - Output completion script for a given shell.
- rclone gendocs - Output markdown docs for rclone to the directory supplied.
- rclone hashsum - Produces an hashsum file for all the objects in the path.
- rclone link - Generate public link to file/folder.
- rclone listremotes - List all the remotes in the config file.
- rclone ls - List the objects in the path with size and path.
- rclone lsd - List all directories/containers/buckets in the path.
- rclone lsf - List directories and objects in remote:path formatted for parsing
- rclone lsjson - List directories and objects in the path in JSON format.
- rclone lsl - List the objects in path with modification time, size and path.
- rclone md5sum - Produces an md5sum file for all the objects in the path.
- rclone mkdir - Make the path if it doesn’t already exist.
- rclone mount - Mount the remote as a mountpoint. EXPERIMENTAL
- rclone move - Move files from source to dest.
- rclone moveto - Move file or directory from source to dest.
- rclone ncdu - Explore a remote with a text based user interface.
- rclone obscure - Obscure password for use in the rclone.conf
- rclone purge - Remove the path and all of its contents.
- rclone rc - Run a command against a running rclone.
- rclone rcat - Copies standard input to file on remote.
- rclone rmdir - Remove the path if empty.
- rclone rmdirs - Remove empty directories under the path.
- rclone serve - Serve a remote over a protocol.
- rclone sha1sum - Produces an sha1sum file for all the objects in the path.
- rclone size - Prints the total size and number of objects in remote:path.
- rclone sync - Make source and dest identical, modifying destination only.
- rclone touch - Create new file or change file modification time.
- rclone tree - List the contents of the remote in a tree like fashion.
- rclone version - Show the version number.
搬瓦工 VPS(KVM,加拿大机房,1核 CPU,512M 内存,11GB SSD,550G 流量)实测,2018-9-2 15:00 左右,开始执行 rclone copy 命令将 Google Drive 上的 Google Photos 复制到 OneDrive 上的 GooglePhotos 目录,2018-9-4 05:30 左右复制完毕,数据约 166G。
yum install screen
screen -S rclone #第一次同步大量数据使用 screen 会话尽可能保护 rclone 执行不被中断
rclone copy gdrive:GooglePhotos onedrive:GooglePhotos
人工粗略检查了一下,看样子是同步成功了。第一次同步因为数据量可能比较大,需要时间比较长,VPS 流量也会耗掉不少。我实测同步 166G,VPS 耗掉的流量大概是这个两倍。数据量大的话最好在流量充裕的 VPS 上进行。
第一次同步完成,就可以 VPS 上(每天/周/月,按需选择)定时执行脚本自动同步。可以选择使用 rclone_copy --ignore-existing 进行增量备份(没有实测,按官方说明是可以实现),或者使用 rclone sync 同步。
我希望 OneDrive 上的备份与 Google Drive 一致,所以每天定时执行 rclone sync gdrive:GooglePhotos onedrive:GooglePhotos
/usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1
配合 crontab 定时执行,比如每天 4:44 执行一次,日志写到 /root/rclone.log。执行 crontab -e
,i,添加下面代码进去,按一下 Esc,然后输入 :wq 回车保存退出。
44 4 * * * /usr/bin/rclone sync gdrive:GooglePhotos onedrive:GooglePhotos >> /root/rclone.log 2>&1
注:不需要日志的话把 >> /root/rclone.log 2>&1 去掉。
然后重启一下 crond 服务即可。(service crond restart)
注意:最好执行 service crond status 检查 crond 服务是否启动,service crond start 可手动启动,或者设置开机启动:chkconfig --level 35 crond on。Centos 7 x86_64 实测通过。
新方法
参考资料
矢澤にこ - 使用 rclone 将文件同步至 Google Dirve
Comments:13
咱还是以本地存储为主,网络存储只是个辅助。你这也费劲的放国外也不嫌麻烦?
@方室 放本地也麻烦啊……
@老头 自动上传NAS……
@方室 我那个渣渣的西数 NAS 只能半自动,有时候还会传重复……
@方室 别说咱;
我们都是全部丢网盘,秒开电影在线看
@lisa 我说呢,这话题怎么有点陌生,这是在挖坟呢!
网络不稳定的话,秒开就有困难了,清晰度也可能打折扣。
收藏备用,那天有365的车就可以试试了。
@空空裤兜 有机会的。
有流量限制额
@木木 限多少?自用足够吧?
谷歌api失效了
Google的网盘和相册不是在下载个人数据的界面直接有选项支持导出到OneDrive吗,整rclone还是太麻烦了
@nesia takeout.google.com?只适合整体迁移吧?能增量备份?就算可以也得手动吧?
rclone 可以增量备份,还可以保留照片原始信息,配置好就自动化执行。