Image source: Pixabay
Guest Post by Gilad David Maayan
Whether you’ve recently adopted cloud-based services or you’ve been using them for a while, you’re likely interested in reducing the amount of manual work that managing a cloud system requires. Luckily, AWS includes a variety of automation tools and integration options that you can put to use to accomplish just that.
We’ll look at three of your options here, one simple to implement and two a bit more challenging, to give you a better idea of how they work and how you can use them to maximum benefit.
EBS Snapshots
The Automation of EBS snapshots is one of the easiest places to begin reducing the manual work that your cloud configuration entails because AWS has already done the programming work for you. AWS Lifecycle Manager, available through the EC2 dashboard, is designed specifically for managing backups in EBS.
To use Lifecycle Manager, you must tag the volumes you wish to automate with and define a schedule to be applied to those tags, including how frequently backups should be created, how many should be kept, and when the process should begin. If used in combination with AWS Config, you can ensure that you do not miss tagging any volumes and get the added benefit of being able to prove that backup processes are in place should you be audited for compliance.
A similar utility, called AWS Backup, was recently introduced if you need to manage backup data from other AWS services as well. If this is the case, it is likely a better option as it can be used to centralize your backup management. If you are only concerned with EBS, though, there is not necessarily a reason to switch as it doesn’t seem to provide additional functionality.
It is also possible to automate the backup of EBS snapshots using AWS CLI if you prefer. This is done using custom scripts and Chron jobs and is the way that snapshots were typically automated before Lifecycle Manager or AWS Backup were released. This method is not as simple to manage or initiate as the built-in service but can provide greater flexibility.
AWS Lambda
AWS Lambda is a service that allows you to run code functions on Amazon servers, allowing you the ability to run code while freeing you from the complications of provisioning or managing servers on which to run it. With Lambda, you upload scripts or compiled programs with the functions you wish to run and manage them through API, the command line, or SDK. This code can then be run according to provided runtimes or set to trigger automatically from AWS services.
Lambda enables you to do everything from setting automatic responses to data changes in S3 buckets to deploying serverless applications through CodePipeline. For an in-depth look at its capabilities, you should take a look at the Lambda documentation but the integrations listed below can give you some idea of how it might be used.
Lambda can read events and invoke your functions from the following AWS services:
- Kinesis—used to collect, process, and analyze streaming data
- DynamoDB—AWS’ fully-managed NoSQL database service
- Simple Queue Service (SQS)—used to manage microservices and applications
Functions can also be invoked directly from the following services as well as a few others not listed here:
- Elastic Load Balancing—distributes application traffic across available resources
- Simple Storage Solution (S3)—scalable object storage
- CloudFormation—facilitates management and creation of cloud infrastructure
- CloudWatch—used to monitor cloud metrics and alert to operational issues
- CodeCommit—fully-managed source control service with Git-based repositories
- Config—used to audit and evaluate AWS resource configurations
- API Gateway—fully-managed API development and maintenance service
To get the most out of Lambda you need to have at least some level of programming knowledge but there are pre-built functions out there that you can consider using as well as many third-party services that employ Lambda functionality.
AWS Systems Manager
AWS Systems Manager (SSM) is an agent-based service used to automate common maintenance and deployment tasks for AWS resources and services. It operates through automation documents, written in JSON or YAML, that define services, actions, and parameters according to your needs, and is managed through the SSM Console.
To use SSM, you must first configure roles to be used for automation tasks through CloudFormation or Identity and Access Management (IAM) and provide them the correct permissions for accessing and modifying the services you wish to automate with.
When you create your automation documents, you can specify the specific action you want to run, how many times it should be attempted, a timeout period, what to do if the action fails, and what to do when the action finishes. You can customize actions by specifying that they depend on the approval of a specific role, that they are triggered by specific resource states, or that they utilize branching logic.
SSM can be used to automate copying, creating, and deleting machine images, creating and deleting CloudFormation stacks, creating tags, invoking Lambda functions, calling and running API actions, and launching new instances. This automation can be applied to most of SSM’s other functionality as well.
Conclusion
Including automation in your cloud management and workflows can save you significant time and effort, allowing you to focus on maximizing cloud performance and productivity. As more businesses move to the cloud, AWS continues to offer more features to help you with this process, increasing the options for automating with programmatic skills and decreasing barriers for those without such skills.
Regardless of which applies to you, the methods covered here are a great place to get started with automating your cloud system and increasing the return on your cloud investment.
Gilad David Maayan is a technology writer who has worked with over 150 technology companies including SAP, Imperva, Samsung NEXT, NetApp and Ixia, producing technical and thought leadership content that elucidates technical solutions for developers and IT leadership. Today he heads Agile SEO, the leading marketing agency in the technology industry.
