FSIPERF07: How do you make trade-offs in your architecture?
Financial services workloads often have to make trade-offs in their architecture to meet their most important goals and KPIs, where performance of the system is deemed more important than other factors, or vice-versa.
FSIPERF07-BP01 Understand your priorities and architect to meet them
For example, a low-latency trading system needs to preserve the performance of the system above all other factors, and be prepared to compromise on the cost of infrastructure to meet their goals. In this situation it is still important not to compromise on availability, and this may require significant investment in parallel, independent, deployments for example an independent deployment of the application stack in multiple AWS Availability Zones or Regions rather than a failover architecture.
Within the workload it may be necessary to trade-off between persistent capacity and elasticity to make sure that the application always has the ability to handle peak workloads without needing timed or reactive scaling up. Consider how much of your peak workload you need to be able to service at any time.
When choosing services consider performance determinism. AWS serverless services like AWS Lambda and AWS Fargate can bring significant performance benefits due to their ability to scale elastically on demand, without intervention, but this is often coupled with less fine control over the underlying environment, for example CPU clock speed, and this can introduce an element of variability into workload performance. Where the workload performance must be as consistent as possible, consider using Amazon EC2, where you get the widest choice, and greatest level of control, over the production environment. For example, using Amazon EC2 directly enables the use of ENA Express, to increase network throughput and reduce latency, but brings restrictions on the Amazon EC2 instances that support this feature.
Consider trade-offs in your application architecture. For example, to preserve
network latency you may choose to use certain services and configurations that are more
complex to implement and maintain, but offer better performance, such as using VPC Peering instead of AWS Transit Gateway
FSIPERF07-BP02 Balance AI model complexity with performance requirements
For financial services applications utilizing generative AI, carefully evaluate the trade-offs between model complexity, response quality, and performance. For time-sensitive financial applications like real-time fraud detection or trading analysis, consider using smaller, specialized models that can provide faster response times. For less time-sensitive tasks like regulatory documentation analysis, larger models with higher accuracy might be more appropriate.
Implementation steps:
-
Evaluate different model sizes and architectures against your ground truth dataset.
-
Consider using model distillation to create smaller, faster models for time-sensitive financial applications.
-
Test prompt caching for common financial queries to reduce latency.
-
Implement streaming responses for improved perceived latency in user-facing applications.
-
Consider using model routers to direct different types of financial queries to the appropriate model based on complexity and time sensitivity.
FSIPERF07-BP03 Optimize cost-performance trade-offs for AI infrastructure
Financial services organizations must carefully balance the costs of high-performance AI infrastructure with the business value generated by AI applications, considering regulatory requirements and competitive advantages.
Cost optimization strategies:
-
Use EC2 Spot instances for AI training workloads that can tolerate interruptions.
-
Commit to reserved capacity for predictable AI inference workloads to reduce costs.
-
Deploy multiple models on shared infrastructure to improve resource utilization.
-
Implement time-based scaling for AI workloads with predictable usage pattern.
Performance and cost considerations:
-
Balance model accuracy requirements with inference costs and latency constraints.
-
Evaluate GPU costs against performance benefits for specific financial AI use cases.
-
Consider trade-offs between real-time processing costs and batch processing latency.
-
Balance performance benefits of edge deployment with infrastructure costs.
Implementation guidance:
-
Use AWS Cost Explorer and AWS Trusted Advisor to identify AI infrastructure optimization opportunities.
-
Implement budget alerts and cost allocation tags for AI workload cost management.
-
Configure SageMaker AI inference endpoints with appropriate auto scaling policies to balance cost and performance.
-
Use Savings Plans and Reserved Instances for predictable AI workloads to reduce infrastructure costs.
