Send high availability data to Amazon Managed Service for Prometheus with Prometheus
To set up a high availability configuration with Prometheus, you must apply
external labels on all instances of a high availability group, so Amazon Managed Service for Prometheus can
identify them. Use the cluster label to identify a Prometheus instance
agent as part of a high availability group. Use the __replica__ label
to identify each replica in the group separately. You need to apply both
__replica__ and cluster labels for de-duplication to
work.
Note
The __replica__ label is formatted with two underscore symbols
before and after the word replica.
Example: code snippets
In the following code snippets, the cluster label identifies the
Prometheus instance agent prom-team1, and the _replica_
label identifies the replicas replica1 and
replica2.
cluster: prom-team1 __replica__: replica1
cluster: prom-team1 __replica__: replica2
As Amazon Managed Service for Prometheus stores data samples from high availability replicas with these
labels, it strips the replica label when the samples are accepted. This
means that you will only have a 1:1 series mapping for your current series instead
of a series per replica. The cluster label is kept.
Note
Certain versions of Kubernetes (1.28 and 1.29) may emit their own metric with
a cluster label. This can cause issues with Amazon Managed Service for Prometheus
deduplication. See the High availability
FAQ for more information.
