Understanding rclone-secret
The Rclone Secret provides the configuration details to locate and access the intermediary storage system. It is mounted as a secret on the Rclone data mover pod and provided to the Rclone executable.
The secret should contain the key rclone.conf
that contains the contents of your rclone.conf file. Here is an
example rclone.conf:
[aws-s3-bucket]
type = s3
provider = AWS
env_auth = false
access_key_id = *******
secret_access_key = ******
region = <region>
location_constraint = <region>
acl = private
In the above example AWS S3 is used as the backend for the intermediary storage system.
[aws-s3-bucket]
: Name of the remote
type
: Type of storage
provider
: Backend provider
access_key_id
: AWS credentials
secret_access_key
: AWS credentials
region
: Region to connect to
location_constraint
: Must be set to match theregion
For detailed instructions follow the Rclone documentation on how to create an rclone.conf
configuration file.
Deploy rclone-secret
Assuming the above example is placed in a local file, rclone.conf
, the
Secret can be created via:
# Create the secret (remember to pass the correct namespace name)
$ kubectl create -n source secret generic rclone-secret --from-file=rclone.conf=rclone.conf
$ kubectl get -n source secrets
NAME TYPE DATA AGE
default-token-g9vdx kubernetes.io/service-account-token 3 20s
rclone-secret Opaque 1 17s
This Secret should be created on both the source and the destination locations.
Using RCLONE_
environment variables in rclone-secret
Rclone has the ability to set environment variables for configuration. Environment variables that
start with RCLONE_
can be set as key/value pairs in the rclone-secret
and they will be passed
to the rclone mover job.
Here is an example rclone-secret
that sets RCLONE_BWLIMIT
to 5M:
apiVersion: v1
kind: Secret
metadata:
name: rclone-secret
type: Opaque
stringData:
# equivalent to the --bwlimit command line flag
RCLONE_BWLIMIT: 5M
# rclone.conf
rclone.conf: |
[s3-bucket]
type = s3
provider = Minio
env_auth = false
access_key_id = user1
secret_access_key = abc123
region = us-east-1
endpoint = http://minio.minio.svc.cluster.local:9000
For detailed information on Rclone environment variables see the Rclone environment variable documentation.