# Optimize over time Optimize over time **Topics** + [ # COST 10. How do you evaluate new services? ](cost-10.md) + [ # COST 11. How do you evaluate the cost of effort? ](cost-11.md) # COST 10. How do you evaluate new services? As AWS releases new services and features, it's a best practice to review your existing architectural decisions to verify they continue to be the most cost effective. **Topics** + [ # COST10-BP01 Develop a workload review process ](cost_evaluate_new_services_review_process.md) + [ # COST10-BP02 Review and analyze this workload regularly ](cost_evaluate_new_services_review_workload.md) # COST10-BP01 Develop a workload review process COST10-BP01 Develop a workload review process Develop a process that defines the criteria and process for workload review. The review effort should reflect potential benefit. For example, core workloads or workloads with a value of over ten percent of the bill are reviewed quarterly or every six months, while workloads below ten percent are reviewed annually. **Level of risk exposed if this best practice is not established:** High ## Implementation guidance Implementation guidance To have the most cost-efficient workload, you must regularly review the workload to know if there are opportunities to implement new services, features, and components. To achieve overall lower costs the process must be proportional to the potential amount of savings. For example, workloads that are 50% of your overall spend should be reviewed more regularly, and more thoroughly, than workloads that are five percent of your overall spend. Factor in any external factors or volatility. If the workload services a specific geography or market segment, and change in that area is predicted, more frequent reviews could lead to cost savings. Another factor in review is the effort to implement changes. If there are significant costs in testing and validating changes, reviews should be less frequent. Factor in the long-term cost of maintaining outdated and legacy, components and resources and the inability to implement new features into them. The current cost of testing and validation may exceed the proposed benefit. However, over time, the cost of making the change may significantly increase as the gap between the workload and the current technologies increases, resulting in even larger costs. For example, the cost of moving to a new programming language may not currently be cost effective. However, in five years time, the cost of people skilled in that language may increase, and due to workload growth, you would be moving an even larger system to the new language, requiring even more effort than previously. Break down your workload into components, assign the cost of the component (an estimate is sufficient), and then list the factors (for example, effort and external markets) next to each component. Use these indicators to determine a review frequency for each workload. For example, you may have webservers as a high cost, low change effort, and high external factors, resulting in high frequency of review. A central database may be medium cost, high change effort, and low external factors, resulting in a medium frequency of review. Define a process to evaluate new services, design patterns, resource types, and configurations to optimize your workload cost as they become available. Similar to [performance pillar review](https://docs.aws.amazon.com/wellarchitected/latest/framework/perf-06.html) and [reliability pillar review](https://docs.aws.amazon.com/wellarchitected/latest/framework/rel_monitor_aws_resources_review_monitoring.html) processes, identify, validate, and prioritize optimization and improvement activities and issue remediation and incorporate this into your backlog. **Implementation steps** + **Define review frequency: **Define how frequently the workload and its components should be reviewed. Allocate time and resources to continual improvement and review frequency to improve the efficiency and optimization of your workload. This is a combination of factors and may differ from workload to workload within your organization and between components in the workload. Common factors include the importance to the organization measured in terms of revenue or brand, the total cost of running the workload (including operation and resource costs), the complexity of the workload, how easy is it to implement a change, any software licensing agreements, and if a change would incur significant increases in licensing costs due to punitive licensing. Components can be defined functionally or technically, such as web servers and databases, or compute and storage resources. Balance the factors accordingly and develop a period for the workload and its components. You may decide to review the full workload every 18 months, review the web servers every six months, the database every 12 months, compute and short-term storage every six months, and long-term storage every 12 months. + ** Define review thoroughness: **Define how much effort is spent on the review of the workload or workload components. Similar to the review frequency, this is a balance of multiple factors. Evaluate and prioritize opportunities for improvement to focus efforts where they provide the greatest benefits while estimating how much effort is required for these activities. If the expected outcomes do not satisfy the goals, and required effort costs more, then iterate using alternative courses of action. Your review processes should include dedicated time and resources to make continuous incremental improvements possible. As an example, you may decide to spend one week of analysis on the database component, one week of analysis for compute resources, and four hours for storage reviews. ## Resources Resources **Related documents:** + [AWS News Blog](https://aws.amazon.com/blogs/aws/) + [Types of Cloud Computing](https://aws.amazon.com/types-of-cloud-computing/) + [What's New with AWS](https://aws.amazon.com/new/) **Related examples:** + [AWS Support Proactive Services ](https://aws.amazon.com/premiumsupport/technology-and-programs/proactive-services/) + [ Regular workload reviews for SAP workloads ](https://docs.aws.amazon.com/wellarchitected/latest/sap-lens/best-practice-4-4.html) # COST10-BP02 Review and analyze this workload regularly COST10-BP02 Review and analyze this workload regularly Existing workloads are regularly reviewed based on each defined process to find out if new services can be adopted, existing services can be replaced, or workloads can be re-architected. **Level of risk exposed if this best practice is not established:** Medium ## Implementation guidance Implementation guidance AWS is constantly adding new features so you can experiment and innovate faster with the latest technology. [AWS What's New](https://aws.amazon.com/new/) details how AWS is doing this and provides a quick overview of AWS services, features, and Regional expansion announcements as they are released. You can dive deeper into the launches that have been announced and use them for your review and analyze of your existing workloads. To realize the benefits of new AWS services and features, you review on your workloads and implement new services and features as required. This means you may need to replace existing services you use for your workload, or modernize your workload to adopt these new AWS services. For example, you might review your workloads and replace the messaging component with Amazon Simple Email Service. This removes the cost of operating and maintaining a fleet of instances, while providing all the functionality at a reduced cost. To analyze your workload and highlight potential opportunities, you should consider not only new services but also new ways of building solutions. Review the [This is My Architecture](https://aws.amazon.com/architecture/this-is-my-architecture) videos on AWS to learn about other customers’ architecture designs, their challenges and their solutions. Check the [All-In series](https://aws.amazon.com/architecture/all-in-series/) to find out real world applications of AWS services and customer stories. You can also watch the [Back to Basics](https://aws.amazon.com/architecture/back-to-basics/) video series that explains, examines, and breaks down basic cloud architecture pattern best practices. Another source is [How to Build This](https://aws.amazon.com/architecture/how-to-build-this/) videos, which are designed to assist people with big ideas on how to bring their minimum viable product (MVP) to life using AWS services. It is a way for builders from all over the world who have a strong idea to gain architectural guidance from experienced AWS Solutions Architects. Finally, you can review the [Getting Started](https://aws.amazon.com/getting-started/) resource materials, which has step by step tutorials. Before starting your review process, follow your business’ requirements for the workload, security and data privacy requirements in order to use specific service or Region and performance requirements while following your agreed review process. **Implementation steps** + ** Regularly review the workload: **Using your defined process, perform reviews with the frequency specified. Verify that you spend the correct amount of effort on each component. This process would be similar to the initial design process where you selected services for cost optimization. Analyze the services and the benefits they would bring, this time factor in the cost of making the change, not just the long-term benefits. + ** Implement new services:** If the outcome of the analysis is to implement changes, first perform a baseline of the workload to know the current cost for each output. Implement the changes, then perform an analysis to confirm the new cost for each output. ## Resources Resources **Related documents:** + [AWS News Blog](https://aws.amazon.com/blogs/aws/) + [What's New with AWS](https://aws.amazon.com/new/) + [AWS Documentation ](https://docs.aws.amazon.com/) + [AWS Getting Started ](https://aws.amazon.com/getting-started/) + [AWS General Resources ](https://docs.aws.amazon.com/#general_resources) **Related videos:** + [AWS - This is My Architecture](https://aws.amazon.com/architecture/this-is-my-architecture) + [AWS - Back to Basics](https://aws.amazon.com/architecture/back-to-basics/) + [AWS - All-In series](https://aws.amazon.com/architecture/all-in-series/) + [How to Build This](https://aws.amazon.com/architecture/how-to-build-this/) # COST 11. How do you evaluate the cost of effort? **Topics** + [ # COST11-BP01 Perform automation for operations ](cost_evaluate_cost_effort_automations_operations.md) # COST11-BP01 Perform automation for operations COST11-BP01 Perform automation for operations Evaluate the operational costs on the cloud, focusing on quantifying the time and effort savings in administrative tasks, deployments, mitigating the risk of human errors, compliance, and other operations through automation. Assess the time and associated costs required for operational efforts and implement automation for administrative tasks to minimize manual effort wherever feasible. **Level of risk exposed if this best practice is not established:** Low ## Implementation guidance Implementation guidance Automating operations reduces the frequency of manual tasks, improves efficiency, and benefits customers by delivering a consistent and reliable experience when deploying, administering, or operating workloads. You can free up infrastructure resources from manual operational tasks and use them for higher value tasks and innovations, which improves business value. Enterprises require a proven, tested way to manage their workloads in the cloud. That solution must be secure, fast, and cost effective, with minimum risk and maximum reliability. Start by prioritizing your operational activities based on required effort by looking at overall operations cost. For example, how long does it take to deploy new resources in the cloud, make optimization changes to existing ones, or implement necessary configurations? Look at the total cost of human actions by factoring in cost of operations and management. Prioritize automations for admin tasks to reduce the human effort. Review effort should reflect the potential benefit. For example, examine time spent performing tasks manually as opposed to automatically. Prioritize automating repetitive, high value, time consuming and complex activities. Activities that pose a high value or high risk of human error are typically the better place to start automating as the risk often poses an unwanted additional operational cost (like operations team working extra hours). Use automation tools like AWS Systems Manager or AWS Config to streamline operations, compliance, monitoring, lifecycle, and termination processes. With AWS services, tools, and third-party products, you can customize the automations you implement to meet your specific requirement. Following table shows some of the core operation functions and capabilities you can achieve with AWS services to automate administration and operation: + [AWS Audit Manager](https://aws.amazon.com/audit-manager/): Continually audit your AWS usage to simplify risk and compliance assessment + [AWS Backup](https://aws.amazon.com/backup/): Centrally manage and automate data protection. + [AWS Config:](https://aws.amazon.com/config/) Configure compute resources, asses, audit, evaluate configurations and resource inventory. + [AWS CloudFormation](https://aws.amazon.com/cloudformation/): Launch highly available resources with Infrastructure as Code. + [AWS CloudTrail](https://aws.amazon.com/cloudtrail/): IT change management, compliance, and control. + [Amazon EventBridge](https://aws.amazon.com/eventbridge/) Schedule events and trigger AWS Lambda to take action. + [AWS Lambda](https://aws.amazon.com/lambda/): Automate repetitive processes by triggering them with events or by running them on a fixed schedule with AWS EventBridge. + [AWS Systems Manager](https://aws.amazon.com/systems-manager/): Start and stop workloads, patch operating systems, automate configuration, and ongoing management. + [AWS Step Functions](https://aws.amazon.com/step-functions/): Schedule jobs and automate workflows. + [AWS Service Catalog](https://aws.amazon.com/servicecatalog/): Template consumption, infrastructure as code with compliance and control. If you would like to adopt automations immediately with using AWS products and service and if don't have skills in your organization, reach out to [AWS Managed Services (AMS)](https://aws.amazon.com/managed-services/), [AWS Professional Services](https://aws.amazon.com/professional-services/), or [AWS Partners](https://aws.amazon.com/partners/work-with-partners/?nc2=h_ql_pa_wwap_cp) to increase adoption of automation and improve your operational excellence in the cloud. AWS Managed Services (AMS) is a service that operates AWS infrastructure on behalf of enterprise customers and partners. It provides a secure and compliant environment that you can deploy your workloads onto. AMS uses enterprise cloud operating models with automation to allow you to meet your organization requirements, move into the cloud faster, and reduce your on-going management costs. AWS Professional Services can also help you achieve your desired business outcomes and automate operations with AWS. They help customers to deploy automated, robust, agile IT operations, and governance capabilities optimized for the cloud. For detailed monitoring examples and recommended best practices, see Operational Excellence Pillar whitepaper. ### Implementation steps Implementation steps + **Build once and deploy many**: Use infrastructure-as-code such as CloudFormation, AWS SDK, or AWS CLI to deploy once and use many times for similar environments or for disaster recovery scenarios. Tag while deploying to track your consumption as defined in other best practices. Use [AWS Launch Wizard](https://aws.amazon.com/launchwizard/) to reduce the time to deploy many popular enterprise workloads. AWS Launch Wizard guides you through the sizing, configuration, and deployment of enterprise workloads following AWS best practices. You can also use the [Service Catalog](https://aws.amazon.com/servicecatalog/), which helps you create and manage infrastructure-as-code approved templates for use on AWS so anyone can discover approved, self-service cloud resources. + **Automate continuous compliance:** Consider automating assessment and remediation of recorded configurations against predefined standards. When you combine AWS Organizations with the capabilities of AWS Config and [AWS CloudFormation](https://aws.amazon.com/cloudformation/), you can efficiently manage and automate configuration compliance at scale for hundreds of member accounts. You can review changes in configurations and relationships between AWS resources and dive into the history of a resource configuration. + **Automate monitoring tasks** AWS provides various tools that you can use to monitor services. You can configure these tools to automate monitoring tasks. Create and implement a monitoring plan that collects monitoring data from all the parts in your workload so that you can more easily debug a multi-point failure if one occurs. For example, you can use the automated monitoring tools to observe Amazon EC2 and report back to you when something is wrong for system status checks, instance status checks, and Amazon CloudWatch alarms. + **Automate maintenance and operations**: Run routine operations automatically without human intervention. Using AWS services and tools, you can choose which AWS automations to implement and customize for your specific requirements. For example, use [EC2 Image Builder](https://aws.amazon.com/image-builder/) for building, testing, and deployment of virtual machine and container images for use on AWS or on-premises or patching your EC2 instances with AWS SSM. If your desired action cannot be done with AWS services or you need more complex actions with filtering resources, then automate your operations with using [AWS Command Line Interface](https://docs.aws.amazon.com/cli/index.html) (AWS CLI) or AWS SDK tools. AWS CLI provides the ability to automate the entire process of controlling and managing AWS services with scripts without using the AWS Management Console. Select your preferred AWS SDKs to interact with AWS services. For other code examples, see AWS SDK Code [examples repository](https://github.com/awsdocs/aws-doc-sdk-examples). + **Create a continual lifecycle with automations:** It is important that you establish and preserve mature lifecycle policies not only for regulations or redundancy but also for cost optimization. You can use AWS Backup to centrally manage and automate data protection of data stores, such as your buckets, volumes, databases, and file systems. You can also use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of EBS snapshots and EBS-backed AMIs. + **Delete unnecessary resources:** It's quite common to accumulate unused resources in sandbox or development AWS accounts. Developers create and experiment with various services and resources as part of the normal development cycle, and then they don't delete those resources when they're no longer needed. Unused resources can incur unnecessary and sometimes high costs for the organization. Deleting these resources can reduce the costs of operating these environments. Make sure your data is not needed or backed up if you are not sure. You can use AWS CloudFormation to clean up deployed stacks, which automatically deletes most resources defined in the template. Alternatively, you can create an automation for the deletion of AWS resources using tools like [aws-nuke](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/automate-deletion-of-aws-resources-by-using-aws-nuke.html). ## Resources Resources **Related documents:** + [Modernizing operations in the AWS Cloud](https://docs.aws.amazon.com/prescriptive-guidance/latest/migration-operations-integration) + [AWS Services for Automation](https://docs.aws.amazon.com/prescriptive-guidance/latest/migration-operations-integration/aws-services-for-automation.html) + [Infrastructure and automation](https://aws.amazon.com/blogs/infrastructure-and-automation/) + [AWS Systems Manager Automation](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-automation.html) + [Automated and manual monitoring](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring_automated_manual.html) + [AWS automations for SAP administration and operations](https://docs.aws.amazon.com/prescriptive-guidance/latest/strategy-sap-automation/automations.html) + [AWS Managed Services](https://docs.aws.amazon.com/managedservices/index.html) + [AWS Professional Services](https://aws.amazon.com/professional-services/) **Related videos:** + [Automate Continuous Compliance at Scale in AWS](https://www.youtube.com/watch?v=5WOL8Njvx48) + [AWS Backup Demo: Cross-Account & Cross-Region Backup](https://www.youtube.com/watch?v=dCy7ixko3tE) + [Patching for your Amazon EC2 Instances](https://www.youtube.com/watch?v=ABtwRb9BFY4) **Related examples:** + [Reinventing automated operations (Part I)](https://aws.amazon.com/blogs/mt/reinventing-automated-operations-part-i/) + [Reinventing automated operations (Part II)](https://aws.amazon.com/blogs/mt/reinventing-automated-operations-part-ii/) + [Automate deletion of AWS resources by using aws-nuke](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/automate-deletion-of-aws-resources-by-using-aws-nuke.html) + [Delete unused Amazon EBS volumes by using AWS Config and AWS SSM](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/delete-unused-amazon-elastic-block-store-amazon-ebs-volumes-by-using-aws-config-and-aws-systems-manager.html) + [Automate continuous compliance at scale in AWS](https://aws.amazon.com/blogs/mt/automate-cloud-foundational-services-for-compliance-in-aws/) + [IT Automations with AWS Lambda](https://aws.amazon.com/lambda/it-automation/)