Flynn: Custom S3 Url for blobstore backend

Created on 9 Sep 2016  ·  5Comments  ·  Source: flynn/flynn

Is it possible to config the Url to a S3 (compatible) object storage server API which is used as blobstore by flynn?

The documentation is not specific about this:

flynn -a blobstore env set BACKEND_S3MAIN="backend=s3 region=us-east-1 \
bucket=flynnblobstore access_key_id=$AWS_ACCESS_KEY_ID \
secret_access_key=$AWS_SECRET_ACCESS_KEY"

flynn -a blobstore env set DEFAULT_BACKEND=s3main

Background is that the S3 API has become more or less the standard REST API for object storage and is supported by a large majority of object storage solutions. So this would allow to use S3 alternatives which support the S3 API.

For example, Minio in its local filesystem mode can be used as an adapter between a NAS and Object Storage API, which is very usefull for flynn deployments on-prem.

componenblobstore kinenhancement

Most helpful comment

Minio is compatible with aws, if there is an option to change the endpoint then there is no need to implement any other new client. somewhere in this func https://github.com/flynn/flynn/blob/master/blobstore/backend/s3.go#L24

All 5 comments

Not currently, though it could be added. I think the client package https://github.com/minio/minio-go is a good fit for a generic S3-compatible backend.

I just used the minio client (thought the Java one) in a project of mine for the same reason: its very light weight and serves as client for all kinds of S3 compatible object stores.

That being said, the minio server also works with the standard aws s3 sdk / client, so this is not required for this feature.

Is there any work done on this? Or can you give some hints on where to start digging to open a PR that e.g. allows setting a custom S3 URL through ENV variables? I think really that all it takes to make this work on a private minio server, as the server is fully S3 compatible.

@walski An additional "minio" or "s3_compat" backend should be added to the blobstore using the minio client package linked above. I think we should keep using the official S3 client for talking to AWS S3. The layout of the blobstore code should be reasonably straightforward, let me know if you have any questions.

Minio is compatible with aws, if there is an option to change the endpoint then there is no need to implement any other new client. somewhere in this func https://github.com/flynn/flynn/blob/master/blobstore/backend/s3.go#L24

Was this page helpful?
0 / 5 - 0 ratings

Related issues

philiplb picture philiplb  ·  4Comments

qwyang picture qwyang  ·  3Comments

onnimonni picture onnimonni  ·  6Comments

heldopslippers picture heldopslippers  ·  4Comments

amingilani picture amingilani  ·  4Comments