I have had to remember the tricky bits of making this work at least 3 times now, so I figured it was about time to post something on this for posterity.
AWS’s Security Token Service (STS) is a lesser known AWS service by which an IAM user can generate a temporary token with a limited access policy. This is handy if you want to issue tokens to a client accessing services via an API, like if you want to give direct access to an S3 bucket.
The problem is that the boto library does not support using said tokens in a simple, consistent way. Here are some steps to assign tokens for access to an S3 bucket
Ensure your IAM user has STS access
In order to issue tokens, you need a user that has permission to access STS resources. Your user should also have access to the resource it wants to grant in the token. An example policy for a user would look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
This policy grants STS permissions to generate the token and then S3 read and write permission for the testing-testing-test-test bucket.
Generate the token
Now that we have a base set of credentials, we are now going to use it to issue an STS token. You will need the AWS access key ID and secret key associated with the user that has the above access policy. You can get this from the IAM console if you don’t already have it. Then getting the token is relatively straightforward:
1 2 3 4 5 6 7 8 9 10 11 12
Notice that we created a new policy here called
is the policy that is associated with the token you just created.
Also notice that we specify a duration for the token of 900 seconds.
After 15 minutes, the credentials associated with this token will no
longer have any access anymore.
Use the token
Now you can distribute this token to whomever you want to grant access for the next 15 minutes (i.e. some client application.) Here is how that client would use it:
1 2 3 4 5 6 7 8 9 10 11 12 13
Here the client opens a new connection to S3 using the credentials from the token and then uploads a file. There are 3 tricky bits here:
The first tricky bit is the extra third parameter to
session_token. The access credentials are not complete
without the session token. It is unfortunate that you cannot just
pass the token itself to connect_to_region and have it deal with all
the parts. Oh well.
The second tricky bit is the
validate=False parameter passed to
get_bucket, this is necessary if your access does not include
get_bucket tries to verify the bucket exists in
your account by default.
The last tricky bit is the worst. Even though you have already
specified the token in
connect_to_region, you have to specify it
get_bucket in a totally bizarre fashion as an extra header
After that bit of obtuse configuration, you are basically home free, you can put the data in that bucket with a given key.