# Performance efficiency The performance efficiency pillar focuses on the allocation of AWS resources to meet the requirements of SAP workloads supporting your business. Performance optimization should be a data driven process of monitoring and measuring performance, and adjusting allocated resources to your requirements in order to maintain efficiency as demand changes and technologies evolve. # 13 – Select the optimal compute solution **How do you select the optimal compute solution for your SAP workload?** Evaluate and estimate the performance requirements using metrics from the SAP tools and existing workloads. Map the compute requirements to the SAP supported instances best suited to your workload. Consider any specific storage or network requirements for the instance types as well as the availability of the required instance types in your chosen AWS Region and Availability Zones. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/design-principle-13.html) For more details, see the following information: + AWS Documentation: [Amazon EC2 Instance Types for SAP](https://aws.amazon.com/sap/instance-types/) + SAP Documentation: [Certified and Supported SAP HANA Hardware](https://www.sap.com/dmc/exp/2014-09-02-hana-hardware/enEN/#/solutions?filters=v:deCertified;ve:23) + SAP Note: [1656099 - SAP Applications on AWS: Supported DB/OS and Amazon EC2 products](https://launchpad.support.sap.com/#/notes/1656099) [Requires SAP Portal Access] + SAP Note: [1656250 - SAP on AWS: Support prerequisites](https://launchpad.support.sap.com/#/notes/1656250) [Requires SAP Portal Access] # Best Practice 13.1 – Evaluate or estimate performance requirements Future hardware requirements can be estimated by examining the capacity and usage patterns of existing SAP systems. SAP provides several tools for sizing hardware for new and existing systems. To further validate sizing estimates, proof of concept (POC) deployments and performance testing can be used. **Suggestion 13.1.1 – Reference SAPS performance metrics of source hardware** SAP benchmarks hardware using [SAP Application Performance Standard (SAPS)](https://www.sap.com/about/benchmark/measuring.html), which is a hardware-independent unit of measurement that describes the performance of a system configuration in the SAP environment. Consult your existing hardware vendor and the SAP benchmark directory to obtain SAPS values for your on-premises server hardware. A sizing based on SAPS is appropriate for a migration that introduces minimal changes to the underlying capacity requirements, often referred to as a lift and shift migration. **Suggestion 13.1.2 – Reference SAP EarlyWatch Alert reports and monitoring tools for historical usage details** [SAP EarlyWatch Alert](https://support.sap.com/en/offerings-programs/support-services/earlywatch-alert.html) reports provide utilization information of your SAP application, such as peak memory and CPU usage. A holistic analysis of these reports spanning several peak events, such as month-end closing and large batch loads, can provide valuable insights into system usage. In addition to EarlyWatch alert reports, infrastructure level monitoring tools can provide more granularity and further insights. **Suggestion 13.1.3 – Use SAP HANA sizing reports to estimate compute requirements** When migrating to SAP HANA, use SAP provided tools for estimating the size of the target compute. The output generated by these tools details the hardware sizing requirements for your SAP HANA database. + SAP Documentation: [SAP HANA Administration Guide for HANA Platform](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.05/en-US/bdf26308bb571014b7bcd3bcd586aecd.html) + AWS Documentation: [SAP HANA Sizing](https://docs.aws.amazon.com/sap/latest/sap-hana/migrating-hana--.html) + SAP Note: [1793345 – Sizing for SAP Suite on HANA](https://launchpad.support.sap.com/#/notes/1793345) [Requires SAP Portal Access] + SAP Note: [1872170 – ABAP on HANA sizing report (S/4HANA, Suite on HANA...)](https://launchpad.support.sap.com/#/notes/1872170) [Requires SAP Portal Access] + SAP Note: [2296290 – New Sizing Report for SAP BW/4HANA](https://launchpad.support.sap.com/#/notes/2296290) [Requires SAP Portal Access] + SAP Note: [1958910 - EarlyWatch Alert For HANA Database](https://launchpad.support.sap.com/#/notes/1958910) [Requires SAP Portal Access] **Suggestion 13.1.4 – Use SAP Quick Sizer for greenfield implementations and functional changes** SAP Quick Sizer can be used for sizing new SAP implementations or for those undergoing changes (for example, increased user base, new functionality or modules). The tool helps you to translate your application’s requirements into hardware specifications. For best results, technical and functional teams should collaborate to input values into the Quick Sizer tool. We recommend the use of SAP expert sizing to validate the sizing of complex implementations. For more information on SAP tools and services, refer to the following: + SAP Documentation: [SAP: Sizing Benchmarks](https://www.sap.com/about/benchmark/sizing.html) **Suggestion 13.1.5 – Use proof of concept deployments for sizing accuracy** You can take advantage of the flexibility of AWS services to right-size your SAP workloads and scale as business demands change. Use proofs of concept (POCs) to test migrations to cloud and analyze the performance requirements. This can help right-size the workloads for both cost and performance. # Best Practice 13.2 - Select EC2 instances suitable for SAP workloads AWS works with SAP to ensure that AWS services are suitable to implement and operate SAP software across a wide selection of instance types. Use guidance from the relevant SAP notes and documentation to identify suitable instances. EC2 instance families offer different ratios of CPU and memory, as well as storage and network throughput characteristics suitable for running SAP workloads. Map your requirements to the appropriate instance type using performance metrics, SAPS figures, and compute estimates. Confirm availability of these instances in your selected Region and Availability Zone. **Suggestion 13.2.1 – Follow SAP guidance on supported databases, operating systems, and AWS services** AWS offers services that can be used for the deployment of SAP products. SAP Note: [1656099 - SAP Applications on AWS: Supported DB/OS and Amazon EC2 products](https://launchpad.support.sap.com/#/notes/1656099) describes which SAP products, database and operating system combinations and Amazon EC2 instance types are currently supported. You can determine the availability of individual instance types within a specific AZ using the AWS CLI [to describe instance type offerings](https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instance-type-offerings.html). + AWS Documentation: [Amazon EC2 Instance Types for SAP](https://aws.amazon.com/sap/instance-types/) + SAP Documentation: [SAP NetWeaver benchmarks](https://www.sap.com/dmc/exp/2018-benchmark-directory/#/sd?filters=v:4a9e824336e2837bf9081e423d576dba) **Suggestion 13.2.2 – Use hardware metrics and SAPS to guide selection** Each SAP supported Amazon EC2 instance family provides a specific vCPU to memory ratio. You should evaluate each instance family based on your requirements to understand the performance profile. The current generation of Amazon EC2 instances (based on [AWS Nitro](https://aws.amazon.com/ec2/nitro/) ) offers the best performance and should be used if available and certified for the deployment scenario. SAP application servers can use either the general purpose (`m*`) or memory optimized (`r*`) instances. Where there is a requirement for a higher vCPU to memory ratio, consider using compute optimized (`c*`) instances. For AnyDB database servers, memory optimized (`r*`) instances are a good fit for the required core to memory ratio, but additional analysis should be done to validate the sizing, especially if your deployment is subject to per-CPU licensing. For SAP HANA databases that run in memory, memory optimized (`r*`, `x*`, `u*`) are your only options. **Suggestion 13.2.3 – Use SAP HANA hardware directory and memory requirement to select EC2 instances for SAP HANA** AWS has SAP HANA certification for a subset of Amazon EC2 instances to run SAP HANA workloads. Details of these instances, and the IaaS application types supported (OLAP, OLTP, SAP Business One, Scale-Out) can be found in [Certified and Supported SAP HANA Hardware](https://www.sap.com/dmc/exp/2014-09-02-hana-hardware/enEN/#/solutions?filters=iaas;ve:23) and [Amazon EC2 Instance Types for SAP](https://aws.amazon.com/sap/instance-types/). Database size and actual working memory usage will determine the memory requirement and instance selection. For non-production workloads, additional options exist. Refer to the blog: + SAP on AWS Blog: [Smaller X1e instances for SAP HANA non-production workloads](https://aws.amazon.com/blogs/awsforsap/smaller-x1e-instances-for-sap-hana-non-production-workloads/) **Suggestion 13.2.4 – Be aware of EC2 instance features and throughput characteristics** Amazon EC2 instances have different features and throughput characteristics which should be evaluated based on your use case, particularly for workloads with high I/O and throughput requirements. These include enhanced networking capabilities through the [Elastic Network Adapter (ENA)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html#ena-performance), I/O performance, Amazon EBS optimization, and suitability for placement groups. For a full list of features, see: + AWS Documentation: [General Purpose Instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/general-purpose-instances.html) + AWS Documentation: [Memory Optimized Instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/memory-optimized-instances.html) + AWS Documentation: [Compute Optimized Instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/compute-optimized-instances.html) # Best Practice 13.3 – Select architectures which allow for independent scaling of systems or components SAP systems and components should have the flexibility to scale without being constrained. This might be accomplished within the allocated hardware or by using horizontal scaling of some components. Consider which architectures allow for this scaling and evaluate any associated trade-offs. **Suggestion 13.3.1 – Consider cross-system or cross-component performance impact** Isolate individual systems or components (for example, Central Services, application servers, and database) to avoid negative performance impact between components. Deploying multiple smaller instance sizes can provide options for instance reuse, workload-based scaling, and capacity on-demand. There are exceptions when trying to optimize the use of resources for cost reasons. Refer to the cost pillar for more details. **Suggestion 13.3.2 – Consider capacity flexibility for peak performance** By selecting architectures which allow for scaling of components, such as the application servers, it will be possible to adapt your capacity to match with performance requirements. This allows your SAP systems to scale for exceptional demand including month end processing or seasonal peaks. # Best Practice 13.4 – Choose location to minimize latency Deploy your SAP instances in Regions and Availability Zones that minimize latency for key business processes impacting end users, critical interfaces, and intra-system traffic. **Suggestion 13.4.1 – Select Region and cloud connectivity to optimize performance** Choose a Region based on proximity to your SAP end users and corporate data center. Select and size any cloud connectivity options (such as Direct Connect and VPN) to accommodate your data transfer requirements. Use SAP performance tools to understand the breakdown of user response time (such as network, GUI, application, and database) and evaluate the impact of any changes to the network round trip time as a result of increased latency. We recommend you focus on high frequency, low latency interfaces between systems in different locations. If increased latency impacts certain end user groups, consider the use of end user compute services and accelerators. + AWS Documentation: [AWS Direct Connect](https://aws.amazon.com/directconnect/) + AWS Documentation: [What is AWS Global Accelerator? - AWS Global Accelerator](https://docs.aws.amazon.com/global-accelerator/latest/dg/what-is-global-accelerator.html) + SAP on AWS Blog: [Deploying SAP GUI on Amazon AppStream 2.0](https://aws.amazon.com/blogs/desktop-and-application-streaming/deploying-sap-gui-on-amazon-appstream-2-0/) **Suggestion 13.4.2 – Be aware of SAP guidelines for intra-system latency** SAP provides guidance for acceptable network latency for two traffic patterns: between SAP application servers and the database, and between SAP HANA database servers (primary and secondary) for the purposes of data replication. AWS regional networks generally meet or exceed these requirements. **Network Latency between SAP application servers and database (Pattern 1)** The following SAP guidance for database to application server connectivity is based on systems running in a single data center, which does not reflect the resiliency benefits of a Multi-AZ deployment. An Availability Zone is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region separated by a meaningful distance (at least 10km). Resilient SAP architectures in AWS typically involve deploying infrastructure across multiple AZs, including SAP application server and database instances. If you have SAP transactions or batch jobs with time-critical performance requirements and that make a significant number of database calls, we recommend that you run these jobs on SAP application servers located in the same AZ as the database. This can be achieved by using SAP Logon Groups (transaction SMLG) for end users and a batch server group (transaction SM61) for background processing jobs. This helps ensure that the latency sensitive parts of the SAP workload run on application servers with the lowest latency to the database. Use tools such as NIPING to measure latency. + SAP Note: [1100926 - FAQ: Network performance](https://launchpad.support.sap.com/#/notes/1100926) [Requires SAP Portal Access] + SAP Note: [2543171 - Latency issue between application server and database](https://launchpad.support.sap.com/#/notes/2543171) [Requires SAP Portal Access] **Network Latency between SAP HANA primary and secondary servers (Pattern 2)** The network latency between the primary and secondary instances will be a factor in the redo log shipping wait time. For synchronous replication, SAP recommends that this wait time should be in the low single millisecond range. This requirement can be achieved across AWS Availability Zones. Use tools such as NIPING to measure latency. + SAP Documentation: [SAP HANA system replication network requirements](https://www.sap.com/documents/2016/06/18079a1c-767c-0010-82c7-eda71af511fa.html) **Suggestion 13.4.3 – Use placement groups for SAP HANA scale out** To meet the SAP certification for internode communication in an SAP HANA scale-out deployment, it is necessary to use a cluster placement group. + AWS Documentation: [Placement groups - Amazon Elastic Compute Cloud](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html#placement-groups-cluster) # 14 – Select the optimal storage solution **How do you select the optimal storage solutions for your SAP workloads?** How you configure this storage will impact the performance of your system. AWS offers a wide range of services, including block, file, and object storage, to meet the storage needs of your SAP databases, applications, and backups. We recommend following the guidelines that have been benchmarked and certified by SAP. For SAP HANA, there are very specific guidelines. Other databases will require more analysis to match your workload. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/design-principle-14.html) # Best Practice 14.1 – Create mount points and volume associations to align with function SAP filesystems have unique performance and sharing requirements. For example, the performance profile of the database may require the data filesystem to support a high number of read I/O operations, while the log filesystem is more likely to be constrained by throughput. Filesystems such as `sapmnt` and `trans` need to be shared so that all application servers can access logs and transport files. Recognizing these differences, consider the mapping of filesystems to volumes to ensure there are no performance bottlenecks and that access requirements are met. **Suggestion 14.1.1 – Identify the SAP filesystems and directory requirements for each System** SAP filesystems include system directories (root, boot), executables, page or swap, and application-specific requirements. Each of these should be analyzed to consider: + The impact when a file system is at capacity (100% used), particularly for the root directory + Consistency of build, including whether it is included in an AMI, or deployment patterns + Resilience requirements + Sharing requirements + Performance profile The core SAP filesystem requirements are listed in the SAP documentation. Use these as a baseline and include other requirements specific to your organization. + SAP Documentation: [SAP Required Filesystems and Directories](https://help.sap.com/docs/SLTOOLSET/910828cec5d14d6685da380aec1dc4ae/de6cad1446a743d3853dbcae48bddfba.html?version=CURRENT_VERSION) **Suggestion 14.1.2 – Map the appropriate AWS storage service to match with filesystem function** A filesystem can either be local or shared (NFS / SMB). For shared filesystems consider using AWS services, such as Amazon EFS and Amazon FSx, which provide reliability and availability benefits when compared with a hosted NFS server. Amazon EC2 instance store is another filesystem option which provides temporary block-level storage for your instance. We do not recommend its use due to lack of persistence, availability across instance types, and because it prevents the use of instance recovery. **Suggestion 14.1.3 – Use supported filesystem types** The SAP-supported Linux distributions recommend a number of different file system types. Later versions are standardizing on XFS, but support should be reviewed to ensure there is no performance or functionality impact for your operating system and database version. + SAP Note: [405827 - Linux: Recommended file systems](https://launchpad.support.sap.com/#/notes/405827) [Requires SAP Portal Access] + SAP Note: [2972496 - SAP HANA Filesystem Types](https://launchpad.support.sap.com/#/notes/2972496) [Requires SAP Portal Access] # Best Practice 14.2 – Select and configure EBS types aligned with performance requirements For each filesystem function and storage service, evaluate storage layout guidelines and tuning options to ensure that IOPS and throughput performance are optimized. **Suggestion 14.2.1 – Evaluate storage characteristics and options for EBS volume types** AWS has a range of volume types with unique characteristics to suit different performance requirements of SAP workloads. Use historical data, or sizing to evaluate the IOPS and throughput requirements. Select your volume type by considering performance, durability, flexibility, and cost. IOPS and throughput of the `gp3` , `io1`, and `io2` Block Express volume types are independent of the volume size. IOPS and throughput of the `gp2` volume type is aligned to the volume size. Oversizing the volume may be required to ensure the required IOPS and throughput is available. If choosing Amazon EBS io2 Block Express, ensure that it is available for the selected instance types. + AWS documentation: [Amazon EBS volume types](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html) **Suggestion 14.2.2 – Scale linearly using LVM striping mechanisms** When the performance requirement cannot be met by a single EBS volume, consider striping using Logical Volume Management (LVM). For example, if a single volume has 250 MiB/s throughput capacity, having a stripe set across four volumes can deliver 1000 MiB/s throughput. Volumes should be of the same size and performance characteristics. In SAP HANA benchmark testing, the best performance has been achieved using a 256 KB stripe size for data volumes and a 64 KB stripe size for log volumes. Be aware of instance limits for throughput, I/O, and number of attached volumes. + AWS Documentation: [Create an LVM Logical Volume on an EBS Volume](https://aws.amazon.com/premiumsupport/knowledge-center/create-lv-on-ebs-volume/) + SAP Note: [2931808 - Usage of Logical Volume Manager (LVM) with SAP HANA](https://launchpad.support.sap.com/#/notes/2931808) [Requires SAP Portal Access] + AWS Documentation: [Operating System and Storage Configuration - SAP HANA on AWS](https://docs.aws.amazon.com/sap/latest/sap-hana/operating-system-and-storage-configuration.html#:~:text=Configure%20Storage%20for%20SAP%20HANA) **Suggestion 14.2.3 – To ensure SAP HANA performance, follow AWS provided storage guidelines** AWS works with SAP to certify storage for SAP HANA workloads according to defined performance benchmarks. The configuration provided by AWS balances performance, cost, and durability within the framework of the SAP TDI 5 Storage KPIs. A compliant storage layout is detailed in the documentation and used in Launch Wizard and quick start deployments. + AWS Documentation: [Storage Configuration for SAP HANA - SAP HANA on AWS](https://docs.aws.amazon.com/sap/latest/sap-hana/hana-ops-storage-config.html) If you deviate from the AWS configuration, it is recommended that you run the SAP HANA Hardware and Cloud Measurement Tools. + SAP Note: [2493172 - SAP HANA Hardware and Cloud Measurement Tools](https://launchpad.support.sap.com/#/notes/2493172) [Requires SAP Portal Access] When deciding between general purpose and Provisioned IOPS EBS types, it should be noted that general purpose types meet the SAP HANA Storage KPIs. An in-memory database, such as SAP HANA, has to load data from disk to memory upon database startup. Using Provisioned IOPS EBS types can significantly improve startup times and also speed up tasks such as backups and restores, which are dependent on storage performance. ** Suggestion 14.2.4 – For performant local backups at a low cost, use `st1` storage ** When SAP solutions need local storage for storing backups, consider using a `st1` volume type for its low cost and high throughput. `st1` is a low-cost block storage type designed for frequently accessed, throughput-intensive workloads. For SAP HANA, consider using AWS Backint Agent for SAP HANA to avoid the performance and cost impact of a two-stage backup. # Best Practice 14.3 - Evaluate Amazon EFS and Amazon FSx performance suitability for your SAP use case Amazon EFS (Linux) and Amazon FSx (Windows) provide highly durable and available file systems that can span multiple Availability Zones. Both solutions are designed to deliver high performance, however, when choosing to use network file systems consider the access patterns. For example, many small files, highly parallel writes or high write/read ratios might not be suitable. For SAP workloads, this might apply to SAP HANA XSA, Java executables, or large numbers of job and spool logs. Amazon FSx for NetApp ONTAP is also a SAP-certified storage type for workloads including S/4HANA, Business Suite on HANA, BW/4HANA, Business Warehouse on HANA, and Data Mart Solutions on HANA. FSx for ONTAP allows you to easily create application-consistent snapshots, space-efficient database clones in seconds and automatic replication of your database across AWS Regions. **Suggestion 14.3.1 – Evaluate scale and performance options** Amazon EFS has two modes for performance (general purpose and max I/O) and two different performance modes (bursting mode and provisioned). For SAP applications, general purpose performance mode usually provides sufficient I/O. There may be scenarios in which provisioned throughput should be considered, such as when the amount of data in your file system is low relative to throughput demands. + AWS Documentation: [Amazon Elastic File System (EFS) \$1 FAQs - Scale and Performance](https://aws.amazon.com/efs/faq/#Scale_and_performance) + AWS Documentation: [Amazon FSx for Windows File Server Features \$1 Scale and Performance](https://aws.amazon.com/fsx/windows/features/#Performance_and_scale) + AWS Documentation: [SAP HANA on AWS with Amazon FSx for NetApp ONTAP](https://docs.aws.amazon.com/sap/latest/sap-hana/sap-hana-amazon-fsx.html) **Suggestion 14.3.2 - Consider temporary provisioning for short-term requirements** Use cases related to migrations or one-off activities might benefit from a temporary file system where performance characteristics can be adjusted for the duration of the event. # Best Practice 14.4 – Consider memory as an alternative to storage Consider the performance advantages of using memory for supported scenarios in the database or application layer. SAP HANA uses memory by default, but might benefit from options to optimize load or offload static data. Relational databases should take advantage of caching, and application servers should consider if swap is a requirement. **Suggestion 14.4.1 – Optimize memory usage for SAP HANA** Seek to understand the correlation between SAP HANA memory requirements and operating system memory indicators to help ensure that memory bottlenecks do not impact performance. + SAP Documentation: [SAP HANA Memory Usage and the Operating System](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.05/en-US/ca10596b37f04909a96614553cb8ab1d.html#:~:text=Virtual%2C%20Physical%2C%20and%20Resident%20Memory&text=On%20most%20SAP%20HANA%20hosts,operational%20use%20by%20a%20process.) + SAP Note: [1999997 - FAQ: SAP HANA Memory](https://launchpad.support.sap.com/#/notes/1999997) [Requires SAP Portal Access] To improve database startup performance in scenarios that do not involve a host restart, consider the use of the SAP HANA Fast Restart option. The SAP HANA Fast Restart option dedicates a portion of RAM as a temporary file system (`tempfs`), which is treated by the operating system as persistent memory (until operating system restart) and allows placement of column store main portion in that `tempfs`, which remains there through an index server restart or crash. Thus, no reloading from storage (using I/O) is necessary. + SAP Documentation: [HANA fast restart documentation](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.05/en-US/ce158d28135147f099b761f8b1ee43fc.html) **Suggestion 14.4.2 – Use database caching for relational databases** For a relational database with high read IOPS requirements, database caching allows you to significantly increase throughput and lower the data retrieval latency. The cache acts as an adjacent data access layer to your database, to improve read performance. The following documentation provides information about caching use-cases, but as most of this detail is relevant to AWS databases, consult SAP Notes for information specific to your relational database configuration. + AWS Documentation: [Caching](https://aws.amazon.com/caching/) (including [database caching](https://aws.amazon.com/caching/database-caching/) ) **Suggestion 14.4.3 – Evaluate swap space requirements for SAP applications** When physical memory resources are exhausted, SAP uses swap to move inactive pages to a dedicated disk-based storage area. Although having swap can prevent the application from crashing due to memory insufficient memory, we recommend applying configuration parameters and memory sizing so that swap is used infrequently. If swap usage is expected, evaluate the characteristics of the allocated volume to avoid further performance issues. Swap can prevent out of memory situations for SAP applications, when the host runs out of physical memory. + SAP Note: [153641 - Swap space requirement for R/3 64-bit kernel](https://launchpad.support.sap.com/#/notes/153641) [Requires SAP Portal Access] + SAP Note: [2999334 - SWAP Utilization](https://launchpad.support.sap.com/#/notes/2999334) (HANA related) [Requires SAP Portal Access] + SAP Note: [2488097 - FAQ: Memory usage for the ABAP Server on Windows](https://launchpad.support.sap.com/#/notes/2488097) [Requires SAP Portal Access] # Best Practice 14.5 – Choose appropriate backup solutions and schedule Depending on the backup method, there is the potential to dramatically increase both read and write operations on your storage, which can negatively impact the performance of your application. This is particularly true for database level backups which might be large in volume and lengthy in duration. **Suggestion 14.5.1 – Determine a suitable backup window** Define what is the most appropriate window for the running of backup operations aligned to your business requirements. Consider key dependencies such as the overnight batch schedule and the acceptable runtime. **Suggestion 14.5.2 – Consider options to minimize the performance impact of backups** Analyze any storage or network constraints and evaluate options to minimize the impact of the backup. This may include reducing the duration by using delta change backups either at a database or storage level. Refer to the Reliability Pillar to ensure this does not negatively impact the consistency of backups or the overall restoration time. + SAP Lens [Reliability]: [Best Practice 12.1 - Establish a method for consistent recovery of business data](best-practice-12-1.md) # 15 – Evaluate tuning options for the operating system, database, and SAP application **How do you understand and weigh the effects of different tuning options on your SAP system performance?** The great variance in performance recommendations for different combinations of SAP software offerings, supported operating systems and databases, and versions prohibit an exhaustive list of recommendations for performance excellence in a single document. With that in mind, the following guidance should be applicable to the majority of SAP use cases, and we will call out specific focus areas where applicable. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/design-principle-15.html) # Best Practice 15.1 – Follow operating system guidelines for SAP performance SAP provides specific guidance on how best to tune for optimal performance for each of the operating systems that are supported for the SAP software you are deploying. Be sure to read all of the relevant SAP documentation on the operating system on which you are deploying both to understand the relevant tuning parameters and to take advantage of any operating system-specific options to make performance tuning easier and more dynamic. **Suggestion 15.1.1 – Review operating system-related SAP notes prior to installation, version update, or infrastructure change** When building or updating your operating system (through automation or manually) confirm that the appropriate performance settings specific to your combination of SAP software and operating system version are applied. **Suggestion 15.1.2 – Evaluate operating system vendor-supplied SAP tuning** Red Hat and SUSE provide images and repositories which contain tools and configuration optimized for running SAP. These are available in the AWS Marketplace or in a bring-your-own-subscription (BYOS) model. Vendors are invested in ensuring that their operating systems are optimised for the SAP application. Using vendor-supplied tuning tools such as `saptune` or the (Ansible) system roles for Red Hat Enterprise Linux can assist in defining a known baseline for performance tuning. While this does not preclude tuning the operating system to best accommodate your specific SAP workload, these tools can reduce the effort associated with researching, calculating and applying the most common requirements. Configuration associated with the `tuned` daemon can also adjust dynamically using information it gathers from the system, including CPU count and available memory. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/best-practice-15-1.html) **Suggestion 15.1.3 – Apply relevant network parameters to the operating system** SAP system performance can be seriously impacted by network misconfiguration, particularly in SAP HANA scale-out database designs as well as in communication between different application server instances and the database instance in a system environment. While in many cases in AWS, the maximum network throughput of an instance is dictated by the instance family and size, tuning of the network settings at the operating system level and in the SAP software itself can have an impact. Refer to the following AWS and SAP recommendations: + AWS Documentation: [Benchmarking Network Throughput between Amazon EC2 Linux instances in the same Amazon VPC](https://aws.amazon.com/premiumsupport/knowledge-center/network-throughput-benchmark-linux-ec2/) + AWS Documentation: [Elastic Network Adapter – High Performance Network Interface for Amazon EC2](https://aws.amazon.com/blogs/aws/elastic-network-adapter-high-performance-network-interface-for-amazon-ec2/) + AWS Documentation: [Cluster Placement Groups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html#placement-groups-cluster) + SAP Note: [2198693 - Key Monitoring Metrics for SAP on Amazon Web Services (AWS)](https://launchpad.support.sap.com/#/notes/2198693) [Requires SAP Portal Access] + SAP Note: [1612283 - Hardware Configuration Standards and Guidance](https://launchpad.support.sap.com/#/notes/1612283) [Requires SAP Portal Access] + SAP Note: [2081065 - Troubleshooting SAP HANA Network](https://launchpad.support.sap.com/#/notes/2081065) [Requires SAP Portal Access] + SAP Note: [1100926 - FAQ: Network performance](https://launchpad.support.sap.com/#/notes/1100926) [Requires SAP Portal Access] # Best Practice 15.2 – Modify database parameters to align with hardware selection SAP provides specific guidance to optimize performance of an SAP system by modifying certain parameters of the underlying database. These parameters are specific to database type and can vary based on whether it’s supporting an analytical or a transactional type application. **Suggestion 15.2.1 – Review SAP HANA-specific tuning parameters, if applicable** Operating System and SAP HANA Database parameters can significantly impact performance. Follow SAP on AWS recommendations for Operating system and storage configuration. + AWS Documentation: [SAP HANA on AWS – Operating System and Storage Configuration](https://docs.aws.amazon.com/sap/latest/sap-hana/operating-system-and-storage-configuration.html) Refer to SAP notes and documentation for guidance on SAP HANA parameters including memory allocation. + SAP Note: [2000000 - FAQ: SAP HANA Performance Optimization](https://launchpad.support.sap.com/#/notes/2000000) [Requires SAP Portal Access] + SAP Documentation: [HANA Parameter: global\$1allocation\$1limit](https://help.sap.com/viewer/009e68bc5f3c440cb31823a3ec4bb95b/2.0.05/en-US/514ab38a2e574c85a70ebba80ff16d99.html#loio514ab38a2e574c85a70ebba80ff16d99__configSPS05_id_805) + SAP Note: [1999997 - FAQ: SAP HANA Memory](https://launchpad.support.sap.com/#/notes/1999997) [Requires SAP Portal Access] + SAP Note: [2926166 - How to limit the overall SAP HANA memory allocation](https://launchpad.support.sap.com/#/notes/2926166) [Requires SAP Portal Access] **Suggestion 15.2.2 – Review database tuning guidance for non-SAP HANA databases** Regardless of the underlying database for your SAP system, performance of the system is in part dependent on how the database is tuned. Each database has specific recommendations for tuning based on available compute, memory, and disk storage. Certain database parameters are dependent on your choice of underlying EC2 instance size; for example, the physical memory available will limit the `db_cache_size` for an Oracle database. For information relevant to your database, refer to the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/best-practice-15-2.html) # Best Practice 15.3 – Modify SAP parameters to align with hardware selection Tuning SAP application parameters can help improve the performance of the application. These parameters are often dependent on the underlying hardware configuration and operating system type. ** Suggestion 15.3.1 – Allow SAP to self-tune according to `PHYS_MEMSIZE` ** For recent versions of SAP software, using kernel release 7.40 or higher, self-tuning of certain parameters is possible and recommended. For instance, many parameters are derived via formulas related to the main memory available on an instance (PHYS\$1MEMSIZE). This allows for automatic tuning of memory parameters when resizing an EC2 instance underlying the SAP software to meet changing performance requirements. + SAP Documentation: [SAP Memory Management: Parameter Reference](https://help.sap.com/viewer/f146e75588924fa4987b6c8f1a7a8c7e/LATEST/en-US/493431b15cce5717e10000000a42189b.html) + SAP Note: [2085980 – New features in memory management as of Kernel Release 7.40](https://launchpad.support.sap.com/#/notes/2085980) [Requires SAP Portal Access] **Suggestion 15.3.2 – Review SAP swap space and maximum memory use** When running SAP on AWS, overutilized swap space on disk can cause I/O credit exhaustion on Amazon EBS and lead to performance degradation. Evaluate the different [EBS storage options](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html) available on AWS and configure swap space to meet your performance needs. + SAP Note: [1597355 - Swap-space recommendation for Linux](https://launchpad.support.sap.com/#/notes/1597355) [Requires SAP Portal Access] + SAP Documentation: [Swap Space Requirements](https://help.sap.com/saphelp_nw73/helpdata/en/49/325e42e93934ffe10000000a421937/frameset.htm) # Best Practice 15.4 – Consider performance tuning for recovery and availability options In alignment with both the Well-Architected Reliability and Operational Excellence pillars, tuning of the SAP system given your chosen recovery and resiliency requirements should be evaluated to minimize any performance impact. Take into consideration items such as system performance during a backup, clustering options for the chosen database (for example, synchronous vs. asynchronous SAP HANA System Replication), and the distribution of load across multiple SAP application server instances. **Suggestion 15.4.1 – Review performance recommendations for backup and recovery solutions** Each supported database has different recommendations to optimize the performance of backups and recovery operations, and these often work in conjunction with your chosen software solution for managing backups and restores, including third-party offerings. In general, following the guidelines for improving throughput between an EC2 instance and the storage target of your backup (such as, EBS volumes, S3 buckets, and EFS file systems) can improve the performance of your backup and recovery. For example, when using Amazon S3 as a repository for backups and AWS Backint Agent for SAP HANA, you can enable performance improvements via changing configuration parameters, such as the maximum concurrency settings. Another common way to improve backup performance is to increase EBS volume performance characteristics, such as maximizing GP3 throughput configuration to 1,000 MB/s. For more information, refer to the following: + AWS Documentation: [AWS Backint Agent for SAP HANA](https://docs.aws.amazon.com/sap/latest/sap-hana/aws-backint-agent-installing-configuring.html#aws-backint-agent-performance-tuning) + AWS Documentation: [SAP NetWeaver on AWS – Backup and Recovery](https://docs.aws.amazon.com/sap/latest/sap-netweaver/backup-and-recovery.html) + AWS Documentation: [S3 Configuration Parameters](https://awscli.amazonaws.com/v2/documentation/api/latest/topic/s3-config.html) + SAP on AWS Blog: [Build for availability and reliability](https://aws.amazon.com/blogs/awsforsap/sap-on-aws-build-for-availability-and-reliability/) [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/best-practice-15-4.html) **Suggestion 15.4.2 – Review configuration of clustering parameters** Clustering options for SAP HANA and other databases often rely on a confirmed connection in a cluster (that is, a heartbeat) between the primary instance and the failover instance. SAP administrators must balance the speed with which an action can occur in the system with the potential for any failover side effects that may occur if there is a false negative interruption in communication. Follow recommendations for timeout parameters and related settings. + AWS Documentation: [SAP HANA on AWS: High Availability Configuration Guide for SLES and RHEL](https://docs.aws.amazon.com/sap/latest/sap-hana/sap-hana-on-aws-ha-configuration.html) + AWS Documentation: [SAP HANA on AWS Operations Guide: Networking](https://docs.aws.amazon.com/sap/latest/sap-hana/hana-ops-networking.html) + AWS Documentation: [SAP on AWS – IBM Db2 HADR with Pacemaker](https://docs.aws.amazon.com/sap/latest/sap-AnyDB/sap-ibm-pacemaker.html) + SAP Note: 1612105 - [DB6: FAQ on Db2 High Availability Disaster Recovery (HADR)](https://launchpad.support.sap.com/#/notes/1612105) [Requires SAP Portal Access] + Operating System-specific Documentation: [SUSE Linux SAP HSR Scale-up Performance Optimized Scenario](https://documentation.suse.com/sbp/all/html/SLES4SAP-hana-sr-guide-PerfOpt-12/index.html) + Operating System-specific Documentation: [Automated SAP HANA System Replication in Scale-Up in pacemaker cluster](https://access.redhat.com/articles/3004101) # 16 – Understand ongoing performance and optimization options **What processes and procedures do you put in place to measure performance changes and opportunities for optimization?** Baseline your applications performance requirement from its historical monitoring data and set relevant alerts to inform system administrators when deviations occur. Have procedures in place for system admins to remediate such issues with manual or automated actions. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/wellarchitected/latest/sap-lens/design-principle-16.html) # Best Practice 16.1 – Have data to evaluate performance To evaluate the performance of an SAP system and take action in the event performance is suboptimal, monitoring data must be collected for compute, memory, storage, and networking as described in the Well-Architected Framework Performance Excellence guidelines concerning monitoring your resources. As stated in the Well-Architected Framework Operational Excellence pillar, understanding the current state of the system, putting in place key performance indicators, and collecting metrics in a timely manner for diagnosis are crucial for investigating performance issues. + Well-Architected Framework [Performance Efficiency]: [Monitor Your Resources to Ensure That They Are Performing as Expected](https://docs.aws.amazon.com/wellarchitected/latest/performance-efficiency-pillar/monitor-your-resources-to-ensure-that-they-are-performing-as-expected.html) + Well-Architected Framework [Operational Excellence]: [Understanding Workload Health](https://docs.aws.amazon.com/wellarchitected/latest/operational-excellence-pillar/understanding-workload-health.html) **Suggestion 16.1.1 – Gather and store data relevant to performance metrics** To collect and view relevant SAP monitoring data, you should install and configure the AWS Data Provider for SAP and set up metrics in your chosen monitoring tools that support your SAP workload. Further details on monitoring and additional recommendations are available in the Operational Excellence pillar. + AWS Documentation: [AWS Data Provider for SAP](https://docs.aws.amazon.com/sap/latest/general/aws-data-provider.html) + SAP Lens [Operational Excellence]: [Best Practice 1.1 - Implement prerequisites for monitoring SAP on AWS](best-practice-1-1.md) + SAP Lens [Operational Excellence]: [Best Practice 1.2 - Implement infrastructure monitoring for SAP](best-practice-1-2.md) + SAP Lens [Operational Excellence]: [Best Practice 1.3 - Implement application and database monitoring for SAP](best-practice-1-3.md) # Best Practice 16.2 – Establish baseline performance requirements Every SAP application has unique performance requirements. Using historical monitoring data helps SAP administration teams understand the baseline performance of these applications, enabling them to identify and understand the extent of any performance changes. Relevant alerts can be put in place to detect anomalies, such as unintended CPU spikes, storage throughput deltas, memory consumption increases, and more complex performance decrements. This monitoring data can be used to further fine-tune performance. **Suggestion 16.2.1 – Collect and evaluate data that reflects SAP-specific KPIs** This suggestion aligns closely with additional suggestions in the Well-Architected Framework Performance Efficiency Pillar discussion regarding [resource monitoring](https://docs.aws.amazon.com/wellarchitected/latest/performance-efficiency-pillar/monitor-your-resources-to-ensure-that-they-are-performing-as-expected.html). In addition to this general guidance, SAP-specific KPIs include dialog response time, buffer swaps, used memory. These KPIs might differ based on the type of SAP software and version you are running on. Further detail on KPI and monitoring recommendations is available in this document in the Operational Excellence pillar: + SAP Lens [Operational Excellence]: [Best Practice 1.2 - Implement infrastructure monitoring for SAP](best-practice-1-2.md) + SAP Lens [Operational Excellence]: [Best Practice 1.3 - Implement application and database monitoring for SAP](best-practice-1-3.md) # Best practice 16.3 – Identify performance trends using data After baselines for performance are established, system administrators must monitor trends over time to see if KPIs remain stable within preferred norms. If the performance data indicates a trend toward unacceptable values of the KPI, system administrators can then take steps to avoid or mitigate performance impacts. **Suggestion 16.3.1 – Conduct regular reviews of SAP system performance** Periodic reviews of KPIs by system administrators can help identify trends in performance-related data as well as determine which alerts might be most beneficial. These alerts can then be used to automate notifications should the trend continue as well as put in place auto-remediation measures to address the potential performance issue (for example, dynamically changing SAP parameters in response to performance indicators). Examples of KPIs and related trends can be found in SAP EarlyWatch Alert reports, which in some cases can be customized with additional useful metrics. SAP service level reporting can also be useful if you have Service Level Agreements (SLAs) in place for your SAP workloads. + SAP Documentation: [Service Level Reporting](http://support.sap.com/slr) + SAP Note: [1040343 - SAP EarlyWatch Alert](https://launchpad.support.sap.com/#/notes/1040343) [Requires SAP Portal Access] + SAP Note: [1829914 - Customize EWA Reports](https://launchpad.support.sap.com/#/notes/1829914) [Requires SAP Portal Access] **Suggestion 16.3.2 – Retain historical data to identify trends** You should retain performance data and associated logs for a predetermined period of time to understand trends in system behavior. Performance tuning of any SAP system will depend on the ability to look back over historical periods of days, weeks, and months to understand what constitutes a performance trend or cyclical performance event. Common events that require retention of data to observe performance impacts include: + Month-end and year-end financial processing + Increased reporting requirements around business milestones (for example, after a large semi-annual sales kick-off) + On-boarding of a large new SAP user population within the business + Technology changes, such as infrastructure sizing, database patches, operating system version updates, or SAP software upgrades # Best Practice 16.4 – Identify and triage performance issues When key metrics indicate performance is degrading, have a process in place to remediate the underlying cause. Using automation (see the following best practice on dynamic scaling) can reduce the need for manual intervention, but when that is not possible, having an automated alerting process for administrators is vital. **Suggestion 16.4.1 – Configure performance alerts appropriately** Follow the guidelines as mentioned in the Well-Architected Framework Performance Efficiency pillar regarding monitoring and alerts, and make use of SAP alerting capabilities where they provide additional capabilities. Additional details are also available in [Operational Excellence] [1 - Design SAP workload to allow understanding and reaction to its state](design-principle-1.md). + Well-Architected Framework [Performance Efficiency]: [Monitoring](https://docs.aws.amazon.com/wellarchitected/latest/performance-efficiency-pillar/monitoring.html) + SAP Documentation: [SAP NetWeaver Alert Monitor](https://help.sap.com/doc/7a827019728810148a4b1a83b0e91070/1610 001/en-US/frameset.htm?frameset.htm) **Suggestion 16.4.2 – Automatic remediation of performance incidents** While the management of performance incidents involves the best practices on operations detailed in the Well-Architected Framework Operational Excellence pillar, the proactive detection and automated remediation of potential performance impairment can prevent deepening a performance problem and can improve the end-user experience. When automated processes for mitigating a performance issue are not possible, having a detailed runbook in place on how the operational team should respond to a performance issue can accelerate the response to a performance incident. + SAP Lens [Operational Excellence]: [Best Practice 1.8 Use automated response and recovery techniques to react to monitoring alerts](best-practice-1-8.md) + Well-Architected Framework [Operational Excellence]: [Best Practices: Operate](https://docs.aws.amazon.com/wellarchitected/latest/framework/oe-operate.html) # Best Practice 16.5 – Scale to meet performance demands One of the primary benefits of operating workloads in AWS is the ability to increase or decrease the compute capacity and change the storage performance characteristics to match the performance required for the use case. For SAP workloads, use dynamic scaling where applicable to avoid performance bottlenecks. Scenarios where dynamic scaling is not possible, such as scaling out an SAP HANA database cluster, use a manual deployment process. **Suggestion 16.5.1 – Reactively scale SAP workloads** In response to dynamic changes in workload performance requirements, scale your SAP resources accordingly. Where possible, use automation to scale in or out, but when that is not an option (such as scaling up a database instance), have a process in place to do so manually. Consider: + Adding or removing application server capacity or changing instance sizes as required to meet demand + Changing SAP parameters to redistribute virtual resources programmatically + [Modifying storage type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-modify-volume.html) (for example, Amazon EBS `gp3` to `io2` or vice versa in AWS), where applicable, to optimize storage performance + AWS Documentation: [Request modifications to your EBS volumes](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/requesting-ebs-volume-modifications.html) **Suggestion 16.5.2 – Schedule scaling for predictable SAP workloads** Whether in an automated or manual fashion, scaling an SAP workload up or down based on predictable performance patterns is advisable. For instance, when month-end financial processing on an SAP ECC system leads to a predictable 20% increase in processing requirements on application server instances, system administrators can proactively increase the number or size of application servers, then scale-in the number of instances when the usage predictably decreases. # Best Practice 16.6 – Develop mechanisms for simulating production load for analysis purposes Having a clone of production data in a test system allows system administrators to simulate production SAP workloads and conduct vital performance tests, such as stress and volume testing. This type of testing can help identify potential performance bottlenecks and prevent performance issues from occurring in a live production environment. **Suggestion 16.6.1 – Define performance sensitive activities** Evaluate which transactions, reports, and operational activities could have an impact on your business if they do not meet peak load requirements or time-critical thresholds. For example, an overnight batch job which must complete in five hours, or a customer-facing transaction accessed concurrently by thousands of users during a quarterly business peak. Document and agree on the measurement approach, KPIs, and success criteria for these workload activities. **Suggestion 16.6.2 – Create an automated test approach for key activities** If required, develop a test strategy to confirm that your SAP workload performance benchmarks are met. Evaluate how test landscapes and tools can enable a repeatable suite of tests to measure the impact of operational activities, change releases and major patching on the performance of your workload. # Best Practice 16.7 – Continually optimize sizing and configuration based on performance data Review performance metrics on a regular basis outside of your incident response process. By doing so, you can discover system components that are undersized, oversized, or no longer used at all. A regular performance optimization cadence should be established for your SAP workloads, focusing on right sizing your system components for real user load. This activity will improve user experience, eliminate unneeded aspects of your architecture, and help improve both the cost effectiveness and resilience of your workload. **Suggestion 16.7.1 – Perform regular architecture right sizing with historical performance metrics as a guide** Regularly review your SAP workload for opportunities to right-size its components. Consider whether storage, compute, network, and supporting services need to be increased or decreased to better fit your business performance requirements. For further information, refer to the following resources: + SAP Lens [Cost Optimization]: [Best Practice 20.5 - Review usage for opportunities to optimize](best-practice-20-5.md) + SAP Lens [Operational Excellence]: [Best Practice 4.4 - Perform regular workload reviews to optimize for resiliency, performance, agility, and cost](best-practice-4-4.md) + AWS Documentation: [Right-Sizing](https://aws.amazon.com/aws-cost-management/aws-cost-optimization/right-sizing/)