Metadata
Learn how to setup IonFS CLI metadata repositories
S3
IonFS can use Amazon S3, and other compatible S3 storage services as a metadata repository. To setup an S3 IonFS repository, you'll need:
- An S3 bucket
- AWS, or other API credentials to access the bucket.
Setting up the S3 bucket
AWS Console
To create an S3 bucket for use as an IonFS metadata repository, navigate to the S3 console on AWS and select Create Bucket.
Choose a name for your S3 bucket. Bear in mind, S3 bucket names have to be globally unique.
Select the region you would like the S3 bucket to be created in.
Next, configure S3 to block all public access to the bucket. IonFS never requires public access for an S3 bucket metadata repository.
Ensure the following settings are set:
Disable Bucket Versioning
Next, enable Default Encryption.
Enabling default encryption provides an extra layer of protection for IonFS metadata stored in S3. The Amazon S3 key is more than sufficient for this use case, but an AWS KMS key can also be used.
Please note, using an AWS KMS key will incur additional cost.
Ensure Object Lock is disabled.
Finally, you can now select Create bucket.
Please now verify that the following settings are visible on your bucket from the S3 console:
- The bucket is set in the desired region, and
- Access to the buckets and objects are marked as not public.
Terraform
The following Terraform config can be used to create an S3 bucket on AWS:
Setting up the IAM user
AWS Console
To access your configured S3 bucket, you will need an AWS IAM account and corresponding access policy.
IAM Policy
First, we need to create an IAM policy. Navigate to the IAM console on AWS and click the Policies heading under Access Management. Click Create Policy
Select the JSON tab on the Create Policy screen.
Delete the default content and paste the following into the JSON tab.
You can either replace change-me with your bucket name, or open the S3 console, select your desired bucket and click Copy ARN. Paste this value into each of the Resource blocks.
Ensure that the ARN specified in the first Resource block also has the trailing /*
, so the policy applies to objects stored within the S3 bucket.
Your policy should now look like the following.
You are now ready to click Review policy.
You can now provide a name for your policy (required) and description (optional).
Once ready, click Create policy.
Your policy will now be visible in the IAM Policies list.
Terraform
The following Terraform config can also be used to setup a suitably secured IAM policy to attach to the IAM user:
IAM User
You can now create an IAM user that can use the policy. Click the Users heading under Access Management. Click Add User.
Pick a name for your IAM user, and select Programmatic access only.
Click Next: Permissions to proceed to the policy selection.
To attached the created IonFS policy to your user, click the Attach existing policies directly tab, and select the IonFS policy. The list of policies can be filtered by name to make it easy to find.
You can continue to the next stage by clicking Next: Tags.
You can now verify the user before creation, and once ready click Create user.
Your user will now be created and ready to use. You can now either click Download .csv to obtain a copy of your user's IAM credentials, or copy them directly from the table.
Please note, your IAM user's Secret access key is only available on this page. If you fail to download the csv file or take a copy of the credentials, you will need to generate a new set.
You are now ready to configure IonFS with your S3 bucket!
MongoDB
Coming soon!