S3 compatibility

Use your existing S3 tools, SDKs, and clients with Graphon: configure credentials, then upload, download, and list objects.

Keep your S3 tools. Graphon speaks the S3 protocol with SigV4, so your S3 clients work as they are. Your API key is also your S3 secret access key. The CLI turns it into the settings that AWS tools read. us-east-1 is only the S3 signing region. Your data lives in the provider region that you chose for the bucket. S3 uses your workspace hostname, while GRAPHON_STORAGE_ORIGIN stays the bare JSON origin.

Configure your S3 tools

Shell

# Sets AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION,
# and your workspace endpoint in AWS_ENDPOINT_URL_S3.
eval "$(graphon s3-credentials --workspace "$GRAPHON_WORKSPACE_ID")"
aws configure set default.s3.addressing_style path

To build an S3 client in code, or to use a credential_process profile, see S3 credentials.

Upload, download, and list

Use the AWS CLI. These examples use a bucket named reports; the gotcha below shows how to create it over S3.

aws CLI

aws --endpoint-url "$AWS_ENDPOINT_URL_S3" \
  s3 cp ./q3-report.pdf s3://reports/finance/q3-report.pdf

aws CLI

aws --endpoint-url "$AWS_ENDPOINT_URL_S3" \
  s3 cp s3://reports/finance/q3-report.pdf ./roundtrip.pdf

aws CLI

aws --endpoint-url "$AWS_ENDPOINT_URL_S3" s3 ls s3://reports/finance/

One gotcha: creating a bucket over S3

S3 CreateBucket needs a {provider}:{location} location constraint, so aws s3 mb won't work. Reach for aws s3api create-bucket instead:

aws CLI

aws --endpoint-url "$AWS_ENDPOINT_URL_S3" s3api create-bucket \
  --bucket reports \
  --create-bucket-configuration LocationConstraint=gcp:us-central1

Use an S3 SDK

Here is the same round trip in boto3:

boto3

# Reads the AWS_* variables that graphon s3-credentials sets.
import os
import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url=os.environ["AWS_ENDPOINT_URL_S3"],
    config=Config(
        signature_version="s3v4",
        s3={"addressing_style": "path"},
    ),
)

s3.upload_file("q3-report.pdf", "reports", "finance/q3-report.pdf")
s3.download_file("reports", "finance/q3-report.pdf", "roundtrip.pdf")

Next steps

See every supported operation in the S3 API reference, or copy data in with Populate your bucket.

Was this page helpful?

View as Markdown