Mastodon`s S3

Hey today we have a short story about Mastodon instance changing its S3 instance between providers.

I have moved from Tebi.io S3 Free Tier (up to 25Gb while writing it) into Oracle Free Tier VPS with 200GB storage with minio S3 app.

First of all I need to tell you that I WAS HAPPY for Tebi. It worked. It have nice bandwith, It worked perfect for that kind of small Mastodon one person instance (more people generates more data so 25Gb would be a very low option).

Now. I have ordered a Free Tier Oracle VPS with AMD Epic (1 core, 1GB). It is important to check if our boot drive is “maxxed” to 200GB so we will have more storage.

I AM NOT AN EXPERT! So i would suggest you to read some people that know what are they doing in this matter. But I have some hints that are obvious for experts and may be hard for noobs.

My configuration:

Minio is hosted on https://storage.domain.com
Bucket is named: “storage” (also)
Mastodon is hosted on https://domain.com

#.env.production mastodon file
LOCAL_DOMAIN=domain.com
#...
S3_ENABLED=true
S3_BUCKET=storage
AWS_ACCESS_KEY_ID=shortAccessKeyWithoutColons
AWS_SECRET_ACCESS_KEY=longerSecurityKeyWithoutColons
S3_ALIAS_HOST=storage.domain.com/storage
S3_HOSTNAME=storage.domain.com/storage
S3_PROTOCOL=https
S3_REGION=de
S3_ENDPOINT=https://storage.domain.com

Minio website -> Bucket -> Policy “Custom” =>

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "*"
                ]
            },
            "Action": [
                "s3:GetBucketLocation",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETNAME"
            ]
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "*"
                ]
            },
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETNAME/*"
            ]
        }
    ]
}

And for bucket access in part “ANONYMOUS” i set “READ ONLY” -> so everybody can read, supposed users can write.

User -> policy => readwrite

readwrite policy in raw is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BucketAccessForUser",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:*",
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETNAME",
                "arn:aws:s3:::BUCKETNAME/*"
            ]
        }
    ]
}

And at last: Administrator / Settings / API

CORS: “http://domain.com” and “https://domain.com”

For me its all…
but no 😉

MCCli -> clone

mc mirror --watch fromHost/bucket ToHost/newBucket

and its simple… but [for my tebi.io and my minio as target] it wasnt as easy 😉

mc alias set --api "s3v4"  fromHost https://s3.tebi.io APIShortKey ApiLongSecretKey

mc alias set ToHost https://storage.domain.com APIShortKey ApiLongSecretKey

yup… and i do not know why BUT this API s3v4 part is crucial for TEBI host. Also without that you will not get a response from s3.tebi.io AND without it you can have a response from yourBucket.s3.tebi.io BUT it will NOT work.

How I did it

1. install minio and make it working (hehe)
2. clone data from tebi to minio (takes loooong time)
3. change .env.production in mastodon and restart mastodon
4. do thing which i didnt: check if it works:
   • Do you see YOUR profile photo on NEW address
   • Do you see others profile photos on NEW address (its different part of bucket)
   • Do you see photos in posts created before migration from NEW address
   • Can you upload new photo to post

All tests are important. I have moved all existing data to new place and was sure that “its all working”… but next day i discovered typo in .env – i forgot “:” sign in “https://storage.domain.com” and … i was able to see images from migration from new server. BUT mastodon cannot upload new images (3rd and 4th dot was not working). Also sidekiq showed… 15? 16? thousands of errors… because none images was downloaded and pushed to s3 properly… yup

Oh and logs says “downloading photo from [[other instance]] incorrect” and not “error on uploading to your s3”.