This whitepaper is for historical reference only. Some content might be outdated and some links might not be available.
Streaming
This stage is only applicable for real-time processing. This stage is primarily responsible for ingesting the unbounded stream of data and providing guaranteed delivery for downstream processing.
Cost factors
The primary costs of this stage are:
-
Data transfer – These are the costs you pay for the rate at which data is consumed by the data streaming service.
-
Streaming service costs – These are the costs you pay (usually per second or per hour) for AWS management of Amazon Kinesis or Amazon MSK service that is being used, including instance cost of Amazon MSK.
-
Storage cost – This is the cost of storing data in streaming service until data is consumed by its consumers and processed.
Cost optimization factors
Ingesting and processing real-time streaming data requires the infrastructure to support the aggregation of the source events, processing of streams, and making the data available for consumption. The AWS streaming ETL services such as Kinesis Data Streams, Kinesis Firehose, and Amazon Managed Kafka Services (MSK), reduces the administration cost. AWS manages the infrastructure, storage, networking, and configuration in your streaming ETL pipeline.
We recommend that you consider the following actions to reduce the cost when using the following services:
