S3 Gateway

An application that mimics the AWS S3 API. Migrate your data from AWS to Tardigrade with minimal effort.

A download can become a chargeable event for 2 times the actual file size if the gateway is running on another cloud provider. We recommend interfacing with the network directly through the Uplink Library.

Dependencies

Tardigrade Account

Tardigrade Project

Tardigrade API Key

Set Up

1. Download, unzip, and install the binary for your OS:

macOS
Windows
Linux

Curl Download

curl -L https://github.com/storj/storj/releases/latest/download/gateway_darwin_amd64.zip -O && unzip gateway_darwin_amd64.zip

Direct Download

macOS Gateway Binary

Curl Download (Powershell)

curl https://github.com/storj/storj/releases/latest/download/gateway_windows_amd64.exe.zip -o gateway_windows_amd64.exe.zip; Expand-Archive gateway_windows_amd64.exe.zip -Destination .

Direct Download

Windows Gateway Binary

AMD64

Curl Download

curl -L https://github.com/storj/storj/releases/latest/download/gateway_linux_amd64.zip -O && unzip gateway_linux_amd64.zip

Direct Download

Linux AMD64 Gateway Binary

ARM

Curl Download

curl -L https://github.com/storj/storj/releases/latest/download/gateway_linux_arm.zip -O && unzip gateway_linux_arm.zip

Direct Download

Linux ARM Gateway Binary

2. Setup your S3 gateway by running the following command and following the instructions provided by the wizard:

macOS
Windows
Linux
./gateway_darwin_amd64 setup
./gateway_windows_amd64.exe setup

AMD64

./gateway_linux_amd64 setup

ARM

./gateway_linux_arm setup

3. Enter the numeric choice corresponding to the satellite you've created your account on:

Do not include brackets in your input

4. Enter the API key you generated:

Didn't save your API key? Simply create another one in the satellite web interface.

Please note that Storj Labs does not know or store your encryption passphrase, so if you lose it, you will not be able to recover your files.

6. Your S3 Gateway is configured and ready to use!

Run the Gateway

The gateway functions as a daemon. Start it and leave it running.

macOS
Windows
Linux
./gateway_darwin_amd64 run
./gateway_windows_amd64.exe run

AMD64

./gateway_linux_amd64 run

ARM

./gateway_linux_arm run

The gateway should output your S3-compatible endpoint, access key, and secret key.

Example terminal output

Reconfiguring the AWS CLI to interface with the Tardigrade Network

To show that this gateway is working, first make sure you have the AWS S3 CLI installed.

Once you do, in a new terminal session, configure it with your Gateway's credentials:

$ aws configure
---
AWS Access Key ID: [Enter your Gateway's Access Key]
AWS Secret Access Key: [Enter your Gateway's Secret Key]
Default region name: [null]
Default output format: [null]

Then, test out some AWS S3 CLI commands!

Create a bucket

AWS CLI
aws s3 --endpoint=http://localhost:7777/ mb s3://bucket-name

Upload an object

AWS CLI
aws s3 --endpoint=http://localhost:7777/ cp ~/Desktop/your-large-file.mp4 s3://bucket-name

List objects in a bucket

AWS CLI
aws s3 --endpoint=http://localhost:7777/ ls s3://bucket-name/

Download an object

AWS CLI
aws s3 --endpoint=http://localhost:7777/ cp s3://bucket-name/your-large-file.mp4 ~/Desktop/your-large-file.mp4

Generate a URL for an object

AWS CLI
aws s3 --endpoint=http://localhost:7777/ presign s3://bucket-name/your-large-file.mp4

(This URL will allow live video streaming through your browser or VLC)

Delete an object

AWS CLI
aws s3 --endpoint=http://localhost:7777/ rm s3://bucket-name/your-large-file.mp4

All Commands

cp - Copies a local file or S3 object to another location locally or in S3

ls - List S3 objects and common prefixes under a prefix or all S3 buckets

mb - Creates an S3 bucket

mv - Moves a local file or S3 object to another location locally or in S3.

presign - Generate a pre-signed URL for an S3 object. This allows anyone who receives the pre-signed URL to retrieve the S3 object with an HTTP GET request.

rb - Deletes an empty S3 bucket

rm - Deletes an S3 object

sync - Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination. Only creates folders in the destination if they contain one or more files

website - Set the website configuration for a bucket

And that's it! You've learned how to use our S3-compatible Gateway. Ideally, you'll see how easy it is to swap out AWS for the Uplink, going forward.