S3 Cleanup Script

Getting an early jump on spring cleaning in the new year? As you may well know you cannot simply delete a non-empty S3 bucket. I’ve created a handy little python script to help. Simply save the below script as s3Deleter.py and execute with s3Deleter.py <bucketname> <list || delete>

import boto3
import sys

try:
    bucketname = sys.argv[1]
except:
    print ("you failed to enter a bucket name, please enter a valid bucket name as the first argument")
    sys.exit(1)

try:
    action = sys.argv[2].lower()
except:
    print ("you failed to enter a bucket action, please enter a valid bucket action (list or delete) as the second argument")
    sys.exit(1)

allowed_actions = ['list','delete']

if action not in allowed_actions:
    print("you provided an invalid action argument. Please enter either list or delete as the argument")
    sys.exit(1)

s3 = boto3.resource('s3')
bucket = s3.Bucket(bucketname)

def bucket_list():
    print("listing all files in the following S3 Bucket: " + str(bucketname))
    for key in bucket.objects.all():
        print(key.key)
    return

def delete_bucket_contents():
    print("deleting all files in the following S3 Bucket: " + str(bucketname))
    bucket.objects.all().delete()
    return

def delete_bucket():
    print("Deleting the bucket: " + str(bucketname))
    bucket.delete()
    return

def main():
    if action == "list":
        bucket_list()
    if action == "delete":
        delete_bucket_contents()
        delete_bucket()

main()

Python 2.7 EOL

Well it’s finally happened. Python 2.7 has gone end of life. This may not mean anything very immediately for you on your local servers, however this can post a problem with AWS Lambdas or other cloud platform functions.

Fortunately Python has a remarkably easy solution for making this transition seamless. 2to3 is a python code translation program that allows you to convert your python 2.x code the 3.x code.

To view the incompatible code and what the updated code would look like in a diff like sort of statement run the following:

python 2to3 <yourPathToFile.py>

If you would like to write the changes to the file you can run the same command with a -w flag:

python 2to3 -w <yourPathToFile.py>

This made my AWS Python lambdas quick and easy to update, simply updating my cloudformation templates with the updated code after running 2to3 and changing the runtime in the Yaml file to Runtime: “python3.6”

A Better Way To Parse CloudWatch Logs

Let’s face it, as much as I love AWS the experience of viewing logs in cloudwatch is moderately awful and annoying. That is until I stumbled upon a fantastic github project called awslogs.

awslogs can be installed as a pip package with a simple pip install awslogs. You will also need to have the AWS CLI installed and configured with a key and secret key in order to make use of it. Once it’s installed it’s a pleasure to work with. I won’t waste time describing is usage as it’s laid out rather nicely in the README on the github project. Check it out here! https://github.com/jorgebastida/awslogs