According to the industry report, tech companies waste up to 35% of their cloud budget inefficiently. This inefficiency is particularly evident in development, staging, or QA environments. Specifically for teams working in a single time zone, running cloud servers beyond working hours is an unreasonable spending.
This article focuses on the best practices for managing and reducing cloud infrastructure costs, particularly in auxiliary environments that don't require 24/7 operation. We will discuss various strategies for cost optimization, such as adopting savings plans, using spot instances, and setting up cost management systems.
One of the key benefits of cloud solutions is the pay-as-you-go model, allowing you to pay only for what you use. The AWS Instance Scheduler helps optimize this process, reducing unnecessary expenses and enhancing efficiency.
A real-world example of Cutting down the AWS bill on the development and Staging resources by up to 70%
Thanks to the outstanding work of all teams on the project, our project entered the list of the 50 fastest-growing companies in America. This success significantly accelerated the project's development and led to the team's expansion. The team grew faster than the project's architecture changed, and soon, to meet the needs of the business and developers, we needed to run 13 testing environments. This definitely impacted our hosting expenses.
Upon analysis, we discovered that our testing environments were actively used only during working hours, while they remained idle in the evenings and on weekends. This led us to the idea of using AWS Instance Scheduler to optimize expenses. We configured the operation of the testing environments for 40 working hours a week, ensuring their functionality only when necessary. Thanks to automation, the startup and shutdown of environments occur automatically, without manual intervention, at the set time.
Furthermore, we found that our staging environment is primarily used for demonstrations and as the final stage before deployment in production. To maintain its operation, we decided to keep the staging environment on 12 hours a day on weekdays. This decision allowed us to reduce expenses by $40,000 a year. After implementing the AWS Instance Scheduler, expenses for EC2 and RDS for testing and staging environments were reduced by 70%. This example demonstrates how a simple yet effective tool can significantly reduce your company's excessive expenses.
What is AWS Instance Scheduler?
AWS Instance Scheduler is an AWS service that allows you to start and stop EC2 and RDS instances automatically in the AWS cloud. This tool is handy for managing costs as it helps avoid unnecessary expenses on resources not used outside of working hours or specific periods
Key features and benefits of AWS Instance Scheduler
Automation:
- Saves time and effort on manual resource management.
- Increases operational efficiency by minimizing human errors.
Flexible schedule settings:
- Organizations can customize instance operation schedules to fit their needs.
- Supports various time zones and individual requirements for each instance.
Cost reduction:
- Significantly cuts expenses by turning off instances when they're not in use.
- Optimizes cloud resource utilization.
Scalability and versatility:
- Suitable for organizations of any size.
- Scales to accommodate growing or changing business needs.
Monitoring and reporting:
- Enables tracking and analysis of resource usage.
- Assists in making informed decisions about resource management.
Compliance:
- Maintains a high level of security and adherence to confidentiality standards.
- Minimizes risks associated with idle instances.
Potential savings with Instance Scheduler
Let's conduct calculations and familiarize ourselves with the methodology for determining potential savings from our solution. In this assessment, we'll consider the cost of using a single instance as an example. However, you can apply similar calculation methods to assess savings in your project, using our approach as a basis.
Methodology for estimating savings
- Current Usage Analysis: First, analyze the current use of EC2 and RDS, considering hours of operation, workload, and performance. Determine which instances are used continuously and which can be turned off during specific periods. When calculating, consider only the use of EC2 and RDS instances, as at the time of writing this article, Instance Scheduler works only for these AWS services.
- Scheduling Determination: Based on the analysis of current usage, develop a schedule for each instance, determining when instances should be active and when stopped. Consider working hours, time zones of teams, and specific application requirements.
- Current Usage Cost Calculation: Using AWS Pricing or AWS Cost Explorer data, calculate the current costs of the instances planned to be managed through Instance Scheduler.
- Forecasting Savings: Estimate how much costs you can reduce by using an Instance Scheduler to stop instances during non-working hours. Consider different types of instances and regions, as service costs can vary significantly.
Calculations and examples of potential savings
Imagine you have a set of EC2 instances used for a Dev environment. Assume they are used only during working hours (approximately 40 hours a week); the rest of the time, they are not used, as your developers only work within business hours.
- Current Costs: If an EC2 instance runs continuously (24/7), that's 168 hours a week. Suppose one of your instances costs $0.1 per hour, equating to $16.8 per week.
- Costs with Instance Scheduler: If the instance operates only during working hours, that's 40 hours a week. Thus, the cost will be reduced to $4 per week.
- Savings: The savings would be $12.8 per week, $51.2 per month, or $614.4 per year, just on one instance. The total savings can be significant if you have many such instances or if they are more expensive. Implementing such automation reduces costs and increases resource usage efficiency, making the cloud infrastructure more resilient and adaptable to business needs.
Considering the cost of the solution
While leveraging the cloud's capabilities, it's important to remember that efficiency often comes with its price tag. This is also true for the AWS Instance Scheduler solution. As you factor this into your overall cost calculations, it's essential to account for the approximate expense of utilizing AWS Instance Scheduler, which is around $2 per month. This cost encompasses using AWS Lambda, CloudWatch, and DynamoDB, integral components of the Instance Scheduler's functionality.
Despite these additional expenses, it's crucial to highlight that the investment in AWS Instance Scheduler is typically offset by the substantial savings it generates. By optimizing instance run times and reducing unnecessary operational hours, the costs saved on your cloud infrastructure can significantly outweigh the operating costs of the solution itself. Thus, while not free, this solution is a strategic move towards a more cost-efficient and resource-optimized cloud environment.
Practical examples of savings
Let's look at specific scenarios of using AWS Instance Scheduler and determine how this simple yet effective tool can reduce your expenses on cloud infrastructure and preserve your budget for increased profitability.
Optimizing costs for Development, QA, and Test environments
- Scenario: Your team of developers and QA engineers typically works during business hours, for example, from 10 AM to 6 PM, averaging an 8-hour workday. After the end of the workday, their environments utilizing EC2 and RDS instances are no longer needed, but without automation, they continue to run and generate costs. Thus, your environments are unused for longer than used, and you pay for what you don't use.
- Solution: With Instance Scheduler, you can automatically stop these resources at 7 PM (just in case) and spin them up at 9 AM. This approach can reduce costs for each instance by up to 65%, considering they are not used overnight and on weekends.
Optimizing costs for staging environments
- Scenario: Staging environments are typically used for pre-production releases and as an exact copy of the production environment for emergency cases. In such cases, your staging environments may be inactive most of the time.
- Solution: You can significantly reduce costs by configuring Instance Scheduler to operate staging environments only during active testing (e.g., on weekdays or during scheduled testing). The exact savings depend on your specific case and how you utilize the environment, but reductions can range from 20% to 50%.
Additional scenarios for using AWS Instance Scheduler
- Training or demonstration environments and resources
For training or demo environments, it is needed only during educational sessions or presentations. For instance, if you have training courses every Wednesday, you can configure instances to operate only on that day. - Internal projects and services
For products and applications used internally by your employees, for example, to maintain workflow. These services can be automatically turned off during non-working hours and on weekends. - Seasonal applications
For applications with seasonal demand, such as retail websites for holiday decorations. Likely, such sites are not profitable during the off-season. Your instances can be activated automatically during the necessary period.
Using AWS Instance Scheduler in these and many other cases allows organizations to manage their cloud resources flexibly, optimizing costs and enhancing the overall efficiency of their cloud infrastructure.
How does the AWS Instance Scheduler work?
- Rule-Based: Instance Scheduler allows users to create customizable rules for starting and stopping instances. These rules can be configured to operate on a specific schedule, for instance, during working hours or on particular days of the week, depending on your objectives.
- Tag Usage: Instance Scheduler uses a tagging system to apply rules to specific instances. Users can assign tags to EC2 and RDS instances that are to be managed by the Scheduler.
- Automation: After setting up the rules and tags, the Instance Scheduler automatically starts and stops instances according to your schedule. This feature reduces the need for manual intervention, eliminates human error, and helps optimize resource usage.
- Flexible Configuration: Instance Scheduler allows organizations to account for various requirements and workflows. For example, you can set up a more aggressive stopping policy for test environments while opting for a more conservative approach for staging environments.
- Monitoring and Reporting: Besides automating the starting and stopping of instances, Instance Scheduler also offers monitoring and reporting capabilities, helping track resource usage and the effectiveness of your scheduling rules.
How to set up AWS Instance Scheduler
To configure automation, which will give you the opportunity to automatically run and stop your EC2 and RDS resources according to the schedule, you must go through the following steps:
- Step 1: Preparation - Ensure that your IAM user has the appropriate access rights.
- Step 2: Setting Up IAM Role
- Step 3: Preparing and Uploading Lambda Code
- Step 4: Configuring and Activating Lambda Function
- Step 5: Create EventBridge rules
- Step 6: Testing and Monitoring the Function
- Step 7: Setting Up Alerts and Notifications
- Step 8: Regular Maintenance and Optimization
Cloud services provide a significant advantage by automating internal cloud processes and setting up cloud services. The Infrastructure as Code (IaC) approach allows us to manage and configure infrastructure with the same level of flexibility as regular code. At ITSyndicate, we often utilize AWS CloudFormation, Terragrunt, or Terraform to automate the setup process, including configuring AWS Instance Scheduler. Our team has created automation using Terraform and made it publicly accessible.
Using this script, you can significantly speed up and simplify setting up AWS Instance Scheduler, making it more manageable, repeatable, and error-free. This approach lets your teams respond more quickly to changes and focus on creating value rather than routine infrastructure management.
Concluding insights
Optimizing costs for cloud resources reduces your company's overall expenses and increases your resource usage efficiency. Using Instance Scheduler allows you to meet the needs of your business precisely, automatically adapting the infrastructure to the actual workload. This is particularly relevant for environments and projects with variable loads, where the cost of unused resources can negatively impact the budget.
In concluding our discussion on automating and optimizing cloud resources with AWS Instance Scheduler, we'd like to highlight a few key points and provide final recommendations from ITSyndicate.
Thorough Planning: Before implementing Instance Scheduler, analyze current resource usage thoroughly. Understanding peak and off-peak periods will help you configure the schedule more accurately and effectively.
Principle of Least Privilege: When setting up IAM roles and policies for Instance Scheduler, ensure only necessary permissions are granted, minimizing security risks.
Regular Updating and Testing: Regularly test your Instance Scheduler configuration to ensure it meets current security and functionality requirements.
Integration with Business Policy: Ensure that your strategy for using Instance Scheduler aligns with company business processes and policies, including approaches to budgeting and reporting.
Using Metrics to Evaluate Efficiency: Regularly analyze cost and resource usage reports to assess the effectiveness of implemented changes and identify further opportunities for optimization.
At ITSyndicate, we recommend using this tool as part of a comprehensive approach to managing cloud resources, combining it with other practices and technologies for the best results. We hope our recommendations and strategies will help you use AWS Instance Scheduler most effectively and achieve new heights in managing cloud infrastructure.
Discover how our services can benefit your business. Leave your contact information and our team will reach out to provide you with detailed information tailored to your specific needs. Take the next step towards achieving your business goals.
How AI and DevOps are transforming software development, addressing new challenges in autonomy and ethics
Achieving 99.9% uptime involves redundancy, continuous monitoring, and robust CI/CD processes to optimize application performance and ensure security
The article discusses overcoming challenges in cloud infrastructure development for a FinTech startup with limited regional options and strict regulations