Skip to content

Connect a Simple Storage Service (S3) System

Simple Storage Service (S3) is used with Amazon Web Services (AWS) for example. In addition to or as an alternative filestore to a pure MongoDB-based setup, the files uploaded by SEAL Operator can be stored in S3. MongoDB is still required for metadata and internal information.

Here, the configuration settings in AWS S3 concerning SEAL Operator are described in brief. For other S3 systems, refer to the corresponding documentation.

Afterwards, the configuration in SEAL Operator is described.

Set Up and Configure an AWS S3 System for SEAL Operator

To configure the storage for the file upload the following items are required:

  • an S3 bucket where the files will be stored
  • an identity and access management (IAM) service user with access to the S3 bucket, its access key and secret

Hint - AWS account

If you do not have an AWS account, go to https://aws.amazon.com/ and create one. This will be the root (admin) of AWS. Using its access keys is not recommended. Create a specific IAM service user instead as described below.

Create the S3 Bucket

  1. Search for the S3 service and create a bucket with the following settings:

    • Name: This will be the root of the filestore. Therefore, specify a meaningful name, for example, seal-operator-fileupload.

    • Region: Select one closest to you, for example, EU (Frankfurt) eu-central-1.

    • Default encryption: Enable Amazon S3 key (SSE-S3).

Create the Access Policy

First, create a policy that gives access to only the S3 bucket created before.

  1. Search for the IAM service.

  2. Open the Policies tab and create a new one.

  3. Copy & paste the following policy JSON structure. Replace <bucket_name> by the name specified for the S3 bucket above, for example, seal-operator-fileupload:

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:CreateBucket",
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:DeleteBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket_name>",
        "arn:aws:s3:::<bucket_name>/*"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    }
  ]
}

  4. Save the policy under a recognizable name, for example, SealFilestoreS3Policy.

Hint - reuse

The permissions policy can be reused for other users.

Create an IAM user and access key

To connect the SEAL Operator to an Amazon S3 bucket, you need an IAM user with an access key.

  1. Open the AWS Management Console and navigate to IAM.

  2. Go to Users and create a new user (for example seal-operator-s3).

  3. Assign the required permissions, either:

  4. by attaching a policy that grants access to the target S3 bucket (recommended: least privilege), or

  5. by adding the user to a group that already has the required S3 permissions.

  6. After the user has been created, open the user details and go to Security credentials.

  7. In the Access keys section, create a new access key (for programmatic access).
    Save the Access key ID and Secret access key – you will need both for the Operator configuration.

Note

The AWS console UI changes from time to time. The important part is that you create an IAM user with an access key that can read/write the S3 bucket used by the Operator.

Configure the S3 Connector

In SEAL Operator, activate the connector and specify the keys for the connection:

  1. Open a Command Prompt or PowerShell.

  2. Export the complete configuration of SEAL Operator from Consul to a YAML file with the following command. So you're making sure the current configuration settings are being used.

    operator config export <filename>.yml --insecure

  3. Edit the exported file <filename>.yml.

  4. In the section for the S3 connector, set cstatus to on.

    operator:
  connectors:
    s3:
      cstatus: 'on'
      serviceName: operator-fileupload

  5. In the env section, specify the following keys for the operator-fileupload service:

    • FILESTORE_TYPE: type how the content of the uploaded files is stored, here s3

    • S3_ACCESS_KEY_ID: ID of the access key to the S3 system as configured in the S3 system

    • S3_SECRET_ACCESS_KEY: secret of the access key to the S3 system as configured in the S3 system

    • S3_BUCKET: name of the S3 bucket as configured in the S3 system

    • S3_REGION: S3 region as configured in the S3 system

    • MONGO_FILEUPLOAD_URL: URL of the MongoDB used by the fileupload service to store metadata and internal information about uploaded files. This is required even when S3 is used as filestore (FILESTORE_TYPE = s3).

Note

The file content can be stored in S3 (FILESTORE_TYPE = s3), but the fileupload service still needs MongoDB for metadata, job information and housekeeping. Therefore, MONGO_FILEUPLOAD_URL must always be configured.

The S3 connection is configured via the operator-fileupload service. There is no dedicated operator-s3 service.
In the Operator configuration:

  • the s3 connector points to operator-fileupload
  • the S3‑specific environment variables are set under env.service.operator-fileupload.tag.any.
env:
  service:
    operator-fileupload:
      tag:
        any:
          FILESTORE_TYPE: s3
          S3_ACCESS_KEY_ID: '<s3_access_key_id>'
          S3_SECRET_ACCESS_KEY: '<s3_secret_access_key>'
          S3_BUCKET: 'seal-operator-fileupload'
          S3_REGION: 'eu-central-1'
          MONGO_FILEUPLOAD_URL: 'mongodb://<mongodb_server>:27017/operator-fileupload'
          DEFAULT_FILEUPLOAD_PANEL: '/code/lib/defaultConfig/s3-panel.json'
Back to top