I have had to remember the tricky bits of making this work at least 3 times now, so I figured it was about time to post something on this for posterity.

AWS’s Security Token Service (STS) is a lesser known AWS service by which an IAM user can generate a temporary token with a limited access policy. This is handy if you want to issue tokens to a client accessing services via an API, like if you want to give direct access to an S3 bucket.

The problem is that the boto library does not support using said tokens in a simple, consistent way. Here are some steps to assign tokens for access to an S3 bucket

Ensure your IAM user has STS access

In order to issue tokens, you need a user that has permission to access STS resources. Your user should also have access to the resource it wants to grant in the token. An example policy for a user would look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt14162015000",
      "Effect": "Allow",
      "Action": [
        "sts:GetFederationToken"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "Stmt14162010000",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::testing-testing-test-test/*",
        "arn:aws:s3:::testing-testing-test-test"
      ]
    }
  ]
}

This policy grants STS permissions to generate the token and then S3 read and write permission for the testing-testing-test-test bucket.

Generate the token

Now that we have a base set of credentials, we are now going to use it to issue an STS token. You will need the AWS access key ID and secret key associated with the user that has the above access policy. You can get this from the IAM console if you don’t already have it. Then getting the token is relatively straightforward:

1
2
3
4
5
6
7
8
9
10
11
12

from boto import sts
from json import dumps

policy_to_grant = {'Statement': [{'Action': ['s3:GetObject', 's3:PutObject'],
                                  'Effect': 'Allow',
                                  'Resource': ['arn:aws:s3:::testing-testing-test-test/*']}]}
stsconn = sts.connect_to_region('us-east-1',
                                aws_access_key_id='<access id>',
                                aws_secret_access_key='<secret key>')
token = stsconn.get_federation_token(name='testy123',
                                     duration=900,
                                     policy=dumps(policy_to_grant))

Notice that we created a new policy here called policy_to_grant. This is the policy that is associated with the token you just created. Also notice that we specify a duration for the token of 900 seconds. After 15 minutes, the credentials associated with this token will no longer have any access anymore.

Use the token

Now you can distribute this token to whomever you want to grant access for the next 15 minutes (i.e. some client application.) Here is how that client would use it:

1
2
3
4
5
6
7
8
9
10
11
12
13

from boto import s3

s3conn = s3.connect_to_region('us-east-1',
                              aws_access_key_id=token.credentials.access_key,
                              aws_secret_access_key=token.credentials.secret_key,
                              security_token=token.credentials.session_token)
b = s3conn.get_bucket('testing-testing-test-test',
                      validate=False,
                      headers={'x-amz-security-token': str(token.credentials.session_token)})
k = s3.key.Key()
k.name = 'hi'
k.bucket = b
k.set_contents_from_file(open('/tmp/hi'))

Here the client opens a new connection to S3 using the credentials from the token and then uploads a file. There are 3 tricky bits here:

The first tricky bit is the extra third parameter to connect_to_region with the session_token. The access credentials are not complete without the session token. It is unfortunate that you cannot just pass the token itself to connect_to_region and have it deal with all the parts. Oh well.

The second tricky bit is the validate=False parameter passed to get_bucket, this is necessary if your access does not include ListBuckets, since get_bucket tries to verify the bucket exists in your account by default.

The last tricky bit is the worst. Even though you have already specified the token in connect_to_region, you have to specify it again in get_bucket in a totally bizarre fashion as an extra header called x-amz-security-token!

After that bit of obtuse configuration, you are basically home free, you can put the data in that bucket with a given key.