# ElastiCache in-transit encryption (TLS)
To help keep your data secure, Amazon ElastiCache and Amazon EC2 provide mechanisms to guard against unauthorized access of your data on the server. By providing in-transit encryption capability, ElastiCache gives you a tool you can use to help protect your data when it is moving from one location to another.
All Valkey or Redis OSS serverless caches have in-transit encryption enabled. For node-based clusters, you can enable in-transit encryption on a replication group by setting the parameter `TransitEncryptionEnabled` to `true` (CLI: `--transit-encryption-enabled`) when you create the replication group. You can do this whether you are creating the replication group using the AWS Management Console, the AWS CLI, or the ElastiCache API.
All serverless caches have in-transit encryption enabled. For node-based clusters, you can enable in-transit encryption on a cluster by setting the parameter `TransitEncryptionEnabled` to `true` (CLI: `--transit-encryption-enabled`) when you create the cluster using the `CreateCacheCluster` (CLI: `create-cache-cluster`) operation.
**Topics**
+ [In-transit encryption overview](#in-transit-encryption-overview)
+ [In-transit encryption conditions (Valkey and Redis OSS)](#in-transit-encryption-constraints)
+ [In-transit encryption conditions (Memcached)](#in-transit-encryption-constraints)
+ [In-transit encryption best practices](#in-transit-encryption-best-practices)
+ [Further Valkey and Redis OSS options](#in-transit-encryption-see-also)
+ [Enabling in-transit encryption for Memcached](#in-transit-encryption-enable-existing-mc)
+ [Enabling in-transit encryption](in-transit-encryption-enable.md)
+ [Connecting to ElastiCache (Valkey) or Amazon ElastiCache for Redis OSS with in-transit encryption using valkey-cli](connect-tls.md)
+ [Enabling in-transit encryption on a node-based Redis OSS cluster using Python](in-transit-encryption-enable-python.md)
+ [Best practices when enabling in-transit encryption](enable-python-best-practices.md)
+ [Connecting to nodes enabled with in-transit encryption using Openssl (Memcached)](#in-transit-encryption-connect-mc)
+ [Creating a TLS Memcached client using Java](#in-transit-encryption-connect-java)
+ [Creating a TLS Memcached client using PHP](#in-transit-encryption-connect-php-mc)
## In-transit encryption overview
Amazon ElastiCache in-transit encryption is a feature that allows you to increase the security of your data at its most vulnerable points—when it is in transit from one location to another. Because there is some processing needed to encrypt and decrypt the data at the endpoints, enabling in-transit encryption can have some performance impact. You should benchmark your data with and without in-transit encryption to determine the performance impact for your use cases.
ElastiCache in-transit encryption implements the following features:
+ **Encrypted client connections**—client connections to cache nodes are TLS encrypted.
+ **Encrypted server connections—**data moving between nodes in a cluster is encrypted.
+ **Server authentication**—clients can authenticate that they are connecting to the right server.
+ **Client authentication**—using the Valkey and Redis OSS AUTH feature, the server can authenticate the clients.
**Note**
ElastiCache does not support mTLS (mutual TLS).
## In-transit encryption conditions (Valkey and Redis OSS)
The following constraints on Amazon ElastiCache in-transit encryption should be kept in mind when you plan your node-based cluster implementation:
+ In-transit encryption is supported on replication groups running Valkey and Redis OSS.
+ Modifying the in-transit encryption setting, for an existing cluster, is supported on replication groups running Valkey 7.2 and later, and Redis OSS version 7 and later.
+ In-transit encryption is supported only for replication groups running in an Amazon VPC.
+ In-transit encryption is not supported for replication groups running the following node types: M1, M2.
For more information, see [Supported node types](CacheNodes.SupportedTypes.md).
+ In-transit encryption is enabled by explicitly setting the parameter `TransitEncryptionEnabled` to `true`.
+ Ensure that your caching client supports TLS connectivity and that you have enabled it in client configuration.
+ Starting April 28, 2026, AWS will update the minimum supported TLS version to 1.2 on ElastiCache for Valkey version 7.2 and above, and ElastiCache for Redis OSS version 6 and above. Customers must update their client software before that date. This update helps you meet security, compliance, and regulatory needs.
## In-transit encryption conditions (Memcached)
The following constraints on Amazon ElastiCache in-transit encryption should be kept in mind when you plan your node-based cluster implementation:
+ In-transit encryption is supported on clusters running Memcached versions 1.6.12 and later.
+ In-transit encryption supports Transport Layer Security (TLS) versions 1.2 and 1.3.
+ In-transit encryption is supported only for clusters running in an Amazon VPC.
+ In-transit encryption is not supported for replication groups running the following node types: M1, M2, M3, R3, T2.
For more information, see [Supported node types](CacheNodes.SupportedTypes.md).
+ In-transit encryption is enabled by explicitly setting the parameter `TransitEncryptionEnabled` to `true`.
+ You can enable in-transit encryption on a cluster only when creating the cluster. You cannot toggle in-transit encryption on and off by modifying a cluster.
+ Ensure that your caching client supports TLS connectivity and that you have enabled it in client configuration.
## In-transit encryption best practices
+ Because of the processing required to encrypt and decrypt the data at the endpoints, implementing in-transit encryption can reduce performance. Benchmark in-transit encryption compared to no encryption on your own data to determine its impact on performance for your implementation.
+ Because creating new connections can be expensive, you can reduce the performance impact of in-transit encryption by persisting your TLS connections.
## Further Valkey and Redis OSS options
For further information on options available for Valkey and Redis OSS, see the followling links.
+ [At-Rest Encryption in ElastiCache](at-rest-encryption.md)
+ [Authenticating with the Valkey and Redis OSS AUTH command](auth.md)
+ [Role-Based Access Control (RBAC)](Clusters.RBAC.md)
+ [Amazon VPCs and ElastiCache security](VPCs.md)
+ [Identity and Access Management for Amazon ElastiCache](IAM.md)
## Enabling in-transit encryption for Memcached
To enable in-transit encryption when creating a Memcached cluster using the AWS Management Console, make the following selections:
+ Choose Memcached as your engine.
+ Choose engine version 1.6.12 or later.
+ Under **Encryption in transit**, choose **Enable**.
For the step-by-step process, see [Creating a cluster for Valkey or Redis OSS](Clusters.Create.md).
# Enabling in-transit encryption
All serverless caches have in-transit encryption enabled. On a node-based cluster, you can enable in-transit encryption using the AWS Management Console, the AWS CLI, or the ElastiCache API.
## Enabling in-transit encryption using the AWS Management Console
### Enabling in-transit encryption for a new node-based cluster using the AWS Management Console
When designing your own cluster, 'Dev/Test' and 'Production' configurations with the 'Easy create' method have in-transit encryption enabled. When choosing configuration yourself, make the following selections:
+ Click the checkbox next to **Enable** for the **Encryption in transit** option.
For the step-by-step process, see the following:
+ [Creating a Valkey (cluster mode disabled) cluster (Console)](SubnetGroups.designing-cluster-pre.valkey.md#Clusters.Create.CON.valkey-gs)
+ [Creating a Valkey or Redis OSS (cluster mode enabled) cluster (Console)](Clusters.Create.md#Clusters.Create.CON.RedisCluster)
### Enabling in-transit encryption for an existing node-based cluster using the AWS Management Console
Enabling encryption in transit, is a two-step process, you must first set the transit encryption mode to `preferred`. This mode allows your Valkey or Redis OSS clients to connect using both encrypted and unencrypted connections. After you migrate all your Valkey or Redis OSS clients to use encrypted connections, you can then modify your cluster configuration to set the transit encryption mode to `required`. Setting the transit encryption mode to `required` will drop all unencrypted connections and will allow encrypted connections only.
**Set your **Transit encryption mode** to **Preferred****
1. Sign in to the AWS Management Console and open the Amazon ElastiCache console at [https://console.aws.amazon.com/elasticache/](https://console.aws.amazon.com/elasticache/).
1. Choose **Valkey caches** or **Redis OSS caches** from the ElastiCache **Resources** listed on the navigation pane, present on the left hand.
1. Choose the cache you want to update.
1. Choose the **Actions** dropdown, then choose **Modify**.
1. Choose **Enable** under **Encryption in transit** in the **Security** section.
1. Choose **Preferred** as the **Transit encryption mode**.
1. Choose **Preview changes** and save your changes.
After you migrate all your Valkey or Redis OSS clients to use encrypted connections:
**Set your **Transit encryption mode** to **Required****
1. Sign in to the AWS Management Console and open the Amazon ElastiCache console at [https://console.aws.amazon.com/elasticache/](https://console.aws.amazon.com/elasticache/).
1. Choose **Valkey caches** or **Redis OSS caches** from the ElastiCache **Resources** listed on the navigation pane, present on the left hand.
1. Choose the cache you want to update.
1. Choose the **Actions** dropdown, then choose **Modify**.
1. Choose **Required** as the **Transit encryption mode**, in the **Security** section.
1. Choose **Preview changes** and save your changes.
## Enabling in-transit encryption using the AWS CLI
To enable in-transit encryption when creating a Valkey or Redis OSS replication group using the AWS CLI, use the parameter `transit-encryption-enabled`.
### Enabling in-transit encryption on a new node-based cluster for Valkey or Redis OSS (Cluster Mode Disabled) (CLI)
Use the AWS CLI operation `create-replication-group` and the following parameters to create a Valkey or Redis OSS replication group with replicas that has in-transit encryption enabled:
**Key parameters:**
+ **--engine**—Must be `valkey` or `redis`.
+ **--transit-encryption-enabled**—Required. If you enable in-transit encryption, you must also provide a value for the `--cache-subnet-group` parameter.
+ **--num-cache-clusters**—Must be at least 1. The maximum value for this parameter is six.
For more information, see the following:
+ [Creating a Valkey or Redis OSS (Cluster Mode Disabled) replication group from scratch (AWS CLI)](Replication.CreatingReplGroup.NoExistingCluster.Classic.md#Replication.CreatingReplGroup.NoExistingCluster.Classic.CLI)
+ [create-replication-group](https://docs.aws.amazon.com/cli/latest/reference/elasticache/create-replication-group.html)
### Enabling in-transit encryption on a new node-based cluster for Valkey or Redis OSS (Cluster Mode Enabled) (CLI)
Use the AWS CLI operation `create-replication-group` and the following parameters to create a Valkey or Redis OSS (cluster mode enabled) replication group that has in-transit encryption enabled:
**Key parameters:**
+ **--engine**—Must be `valkey` or `redis`.
+ **--transit-encryption-enabled**—Required. If you enable in-transit encryption you must also provide a value for the `--cache-subnet-group` parameter.
+ Use one of the following parameter sets to specify the configuration of the replication group's node groups:
+ **--num-node-groups**—Specifies the number of shards (node groups) in this replication group. The maximum value of this parameter is 500.
**--replicas-per-node-group**—Specifies the number of replica nodes in each node group. The value specified here is applied to all shards in this replication group. The maximum value of this parameter is 5.
+ **--node-group-configuration**—Specifies the configuration of each shard independently.
For more information, see the following:
+ [Creating a Valkey or Redis OSS (Cluster Mode Enabled) replication group from scratch (AWS CLI)](Replication.CreatingReplGroup.NoExistingCluster.Cluster.md#Replication.CreatingReplGroup.NoExistingCluster.Cluster.CLI)
+ [create-replication-group](https://docs.aws.amazon.com/cli/latest/reference/elasticache/create-replication-group.html)
### Enabling in-transit encryption for an existing cluster using the AWS CLI
Enabling encryption in transit, is a two-step process, you must first set the transit encryption mode to `preferred`. This mode allows your Valkey or Redis OSS clients to connect using both encrypted and unencrypted connections. After you migrate all your Valkey or Redis OSS clients to use encrypted connections, you can then modify your cluster configuration to set the transit encryption mode to `required`. Setting the transit encryption mode to `required` will drop all unencrypted connections and will allow encrypted connections only.
Use the AWS CLI operation `modify-replication-group` and the following parameters to update a Valkey or Redis OSS (cluster mode enabled) replication group that has in-transit encryption disabled.
**To enable in-transit encryption**
1. Set transit-encryption-mode to `preferred`, using the following parameters
+ **--transit-encryption-enabled**—Required.
+ **--transit-encryption-mode**—Must be set to `preferred`.
1. Set transit-encryption-mode to `required`, using the following parameters:
+ **--transit-encryption-enabled**—Required.
+ **--transit-encryption-mode**—Must be set to `required`.
# Connecting to ElastiCache (Valkey) or Amazon ElastiCache for Redis OSS with in-transit encryption using valkey-cli
To access data from ElastiCache for Redis OSS caches enabled with in-transit encryption, you use clients that work with Secure Socket Layer (SSL). You can also use valkey-cli with TLS/SSL on Amazon Linux and Amazon Linux 2. If your client does not support TLS, you can use the `stunnel` command on your client host to create an SSL tunnel to the Redis OSS nodes.
## Encrypted connection with Linux
To use valkey-cli to connect to a Valkey or Redis OSS cluster enabled with in-transit encryption on Amazon Linux 2 or Amazon Linux, follow these steps.
1. Download and compile the valkey-cli utility. This utility is included in the Valkey software distribution.
1. At the command prompt of your EC2 instance, type the appropriate commands for the version of Linux you are using.
**Amazon Linux 2**
If using Amazon Linux 2, enter this:
```
sudo yum -y install openssl-devel gcc
wget -O valkey-7.2.6.tar.gz https://github.com/valkey-io/valkey/archive/refs/tags/7.2.6.tar.gz
tar xvzf valkey-7.2.6.tar.gz
cd valkey-7.2.6
make distclean
make valkey-cli BUILD_TLS=yes
sudo install -m 755 src/valkey-cli /usr/local/bin/
```
**Amazon Linux**
If using Amazon Linux, enter this:
```
sudo yum install gcc jemalloc-devel openssl-devel tcl tcl-devel clang wget
wget -O valkey-8.0.0.tar.gz https://github.com/valkey-io/valkey/archive/refs/tags/8.0.0.tar.gz
tar xvzf valkey-8.0.0.tar.gz
cd valkey-8.0.0
make valkey-cli CC=clang BUILD_TLS=yes
sudo install -m 755 src/valkey-cli /usr/local/bin/
```
On Amazon Linux, you may also need to run the following additional steps:
```
sudo yum install clang
CC=clang make
sudo make install
```
1. After you have downloaded and installed the valkey-cli utility, it is recommended that you run the optional `make-test` command.
1. To connect to a cluster with encryption and authentication enabled, enter this command:
```
valkey-cli -h Primary or Configuration Endpoint --tls -a 'your-password' -p 6379
```
**Note**
If you install redis6 on Amazon Linux 2023, you can now use the command `redis6-cli` instead of `valkey-cli`:
```
redis6-cli -h Primary or Configuration Endpoint --tls -p 6379
```
## Encrypted connection with stunnel
To use valkey-cli to connect to a Redis OSS cluster enabled with in-transit encryption using stunnel, follow these steps.
1. Use SSH to connect to your client and install `stunnel`.
```
sudo yum install stunnel
```
1. Run the following command to create and edit file `'/etc/stunnel/valkey-cli.conf'` simultaneously to add a ElastiCache for Redis OSS cluster endpoint to one or more connection parameters, using the provided output below as template.
```
vi /etc/stunnel/valkey-cli.conf
fips = no
setuid = root
setgid = root
pid = /var/run/stunnel.pid
debug = 7
delay = yes
options = NO_SSLv2
options = NO_SSLv3
[valkey-cli]
client = yes
accept = 127.0.0.1:6379
connect = primary.ssltest.wif01h.use1.cache.amazonaws.com:6379
[valkey-cli-replica]
client = yes
accept = 127.0.0.1:6380
connect = ssltest-02.ssltest.wif01h.use1.cache.amazonaws.com:6379
```
In this example, the config file has two connections, the `valkey-cli` and the `valkey-cli-replica`. The parameters are set as follows:
+ **client** is set to yes to specify this stunnel instance is a client.
+ **accept** is set to the client IP. In this example, the primary is set to the Redis OSS default 127.0.0.1 on port 6379. The replica must call a different port and set to 6380. You can use ephemeral ports 1024–65535. For more information, see [Ephemeral ports](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html#VPC_ACLs_Ephemeral_Ports) in the *Amazon VPC User Guide.*
+ **connect** is set to the Redis OSS server endpoint. For more information, see [Finding connection endpoints in ElastiCache](Endpoints.md).
1. Start `stunnel`.
```
sudo stunnel /etc/stunnel/valkey-cli.conf
```
Use the `netstat` command to confirm that the tunnels started.
```
sudo netstat -tulnp | grep -i stunnel
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 3189/stunnel
tcp 0 0 127.0.0.1:6380 0.0.0.0:* LISTEN 3189/stunnel
```
1. Connect to the encrypted Redis OSS node using the local endpoint of the tunnel.
+ If no AUTH password was used during ElastiCache for Redis OSS cluster creation, this example uses the valkey-cli to connect to the ElastiCache for Redis OSS server using complete path for valkey-cli, on Amazon Linux:
```
/home/ec2-user/redis-7.2.5/src/valkey-cli -h localhost -p 6379
```
If AUTH password was used during Redis OSS cluster creation, this example uses valkey-cli to connect to the Redis OSS server using complete path for valkey-cli, on Amazon Linux:
```
/home/ec2-user/redis-7.2.5/src/valkey-cli -h localhost -p 6379 -a my-secret-password
```
OR
+ Change directory to redis-7.2.5 and do the following:
If no AUTH password was used during ElastiCache for Redis OSS cluster creation, this example uses the valkey-cli to connect to the ElastiCache for Redis OSS server using complete path for valkey-cli, on Amazon Linux:
```
src/valkey-cli -h localhost -p 6379
```
If AUTH password was used during Redis OSS cluster creation, this example uses valkey-cli to connect to the Valkey or Redis OSS server using complete path for valkey-cli, on Amazon Linux:
```
src/valkey-cli -h localhost -p 6379 -a my-secret-password
```
This example uses Telnet to connect to the Valkey Redis OSS server.
```
telnet localhost 6379
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
auth MySecretPassword
+OK
get foo
$3
bar
```
1. To stop and close the SSL tunnels, `pkill` the stunnel process.
```
sudo pkill stunnel
```
# Enabling in-transit encryption on a node-based Redis OSS cluster using Python
The following guide will demonstrate how to enable in-transit encryption on a Redis OSS 7.0 cluster that was originally created with in-transit encryption disabled. TCP and TLS clients will continue communicating with the cluster during this process without downtime.
Boto3 will get the credentials it needs (`aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token`) from the environment variables. Those credentials will be pasted in advance in the same bash terminal where we will run `python3` to process the Python code shown in this guide. The code in the example below was process from an EC2 instance that was launched in the same VPC that will be used to create the ElastiCache Redis OSS Cluster in it.
**Note**
The following examples use the boto3 SDK for ElastiCache management operations (cluster or user creation) and redis-py/redis-py-cluster for data handling.
You must use at least boto3 version (=\$1) 1.26.39 to use the online TLS migration with the cluster modification API.
ElastiCache supports online TLS migration only for clusters with Valkey version 7.2 and above or Redis OSS version 7.0 or above. So if you have a cluster running a Redis OSS version earlier than 7.0, you’ll need to upgrade the Redis OSS version of your cluster. For more information on version differences, see [Major engine version behavior and compatibility differences with Redis OSS](VersionManagementConsiderations.md).
**Topics**
+ [Define the string constants that will launch the ElastiCache Valkey or Redis OSS Cluster](#enable-python-define-constants)
+ [Define the classes for the cluster configuration](#enable-python-define-classes)
+ [Define a class that will represent the cluster itself](#enable-python-define-classes-cluster)
+ [(Optional) Create a wrapper class to demo client connection to Valkey or Redis OSS cluster](#enable-python-create-wrapper)
+ [Create the main function that demos the process of changing in-transit encryption configuration](#enable-python-main-function)
## Define the string constants that will launch the ElastiCache Valkey or Redis OSS Cluster
First, let’s define some simple Python string constants that will hold the names of the AWS entities required to create the ElastiCache cluster such as `security-group`, `Cache Subnet group`, and a `default parameter group`. All of these AWS entities must be created in advance in your AWS account in the Region you are willing to use.
```
#Constants definitions
SECURITY_GROUP = "sg-0492aa0a29c558427"
CLUSTER_DESCRIPTION = "This cluster has been launched as part of the online TLS migration user guide"
EC_SUBNET_GROUP = "client-testing"
DEFAULT_PARAMETER_GROUP_REDIS_7_CLUSTER_MODE_ENABLED = "default.redis7.cluster.on"
```
## Define the classes for the cluster configuration
Now, let’s define some simple Python classes that will represent a configuration of a cluster, which will hold metadata about the cluster such as the Valkey or Redis OSS version, the instance type, and whether in-transit encryption (TLS) is enabled or disabled.
```
#Class definitions
class Config:
def __init__(
self,
instance_type: str = "cache.t4g.small",
version: str = "7.0",
multi_az: bool = True,
TLS: bool = True,
name: str = None,
):
self.instance_type = instance_type
self.version = version
self.multi_az = multi_az
self.TLS = TLS
self.name = name or f"tls-test"
def create_base_launch_request(self):
return {
"ReplicationGroupId": self.name,
"TransitEncryptionEnabled": self.TLS,
"MultiAZEnabled": self.multi_az,
"CacheNodeType": self.instance_type,
"Engine": "redis",
"EngineVersion": self.version,
"CacheSubnetGroupName": EC_SUBNET_GROUP ,
"CacheParameterGroupName": DEFAULT_PARAMETER_GROUP_REDIS_7_CLUSTER_MODE_ENABLED ,
"ReplicationGroupDescription": CLUSTER_DESCRIPTION,
"SecurityGroupIds": [SECURITY_GROUP],
}
class ConfigCME(Config):
def __init__(
self,
instance_type: str = "cache.t4g.small",
version: str = "7.0",
multi_az: bool = True,
TLS: bool = True,
name: str = None,
num_shards: int = 2,
num_replicas_per_shard: int = 1,
):
super().__init__(instance_type, version, multi_az, TLS, name)
self.num_shards = num_shards
self.num_replicas_per_shard = num_replicas_per_shard
def create_launch_request(self) -> dict:
launch_request = self.create_base_launch_request()
launch_request["NumNodeGroups"] = self.num_shards
launch_request["ReplicasPerNodeGroup"] = self.num_replicas_per_shard
return launch_request
```
## Define a class that will represent the cluster itself
Now, let’s define some simple Python classes that will represent the ElastiCache Valkey or Redis OSS Cluster itself. This class will have a client field which will hold a boto3 client for ElastiCache management operations such as creating the cluster and querying the ElastiCache API.
```
import botocore.config
import boto3
# Create boto3 client
def init_client(region: str = "us-east-1"):
config = botocore.config.Config(retries={"max_attempts": 10, "mode": "standard"})
init_request = dict()
init_request["config"] = config
init_request["service_name"] = "elasticache"
init_request["region_name"] = region
return boto3.client(**init_request)
class ElastiCacheClusterBase:
def __init__(self, name: str):
self.name = name
self.elasticache_client = init_client()
def get_first_replication_group(self):
return self.elasticache_client.describe_replication_groups(
ReplicationGroupId=self.name
)["ReplicationGroups"][0]
def get_status(self) -> str:
return self.get_first_replication_group()["Status"]
def get_transit_encryption_enabled(self) -> bool:
return self.get_first_replication_group()["TransitEncryptionEnabled"]
def is_available(self) -> bool:
return self.get_status() == "available"
def is_modifying(self) -> bool:
return self.get_status() == "modifying"
def wait_for_available(self):
while True:
if self.is_available():
break
else:
time.sleep(5)
def wait_for_modifying(self):
while True:
if self.is_modifying():
break
else:
time.sleep(5)
def delete_cluster(self) -> bool:
self.elasticache_client.delete_replication_group(
ReplicationGroupId=self.name, RetainPrimaryCluster=False
)
def modify_transit_encryption_mode(self, new_transit_encryption_mode: str):
# generate api call to migrate the cluster to TLS preffered or to TLS required
self.elasticache_client.modify_replication_group(
ReplicationGroupId=self.name,
TransitEncryptionMode=new_transit_encryption_mode,
TransitEncryptionEnabled=True,
ApplyImmediately=True,
)
self.wait_for_modifying()
class ElastiCacheClusterCME(ElastiCacheClusterBase):
def __init__(self, name: str):
super().__init__(name)
@classmethod
def launch(cls, config: ConfigCME = None) -> ElastiCacheClusterCME:
config = config or ConfigCME()
print(config)
new_cluster = ElastiCacheClusterCME(config.name)
launch_request = config.create_launch_request()
new_cluster.elasticache_client.create_replication_group(**launch_request)
new_cluster.wait_for_available()
return new_cluster
def get_configuration_endpoint(self) -> str:
return self.get_first_replication_group()["ConfigurationEndpoint"]["Address"]
#Since the code can throw exceptions, we define this class to make the code more readable and
#so we won't forget to delete the cluster
class ElastiCacheCMEManager:
def __init__(self, config: ConfigCME = None):
self.config = config or ConfigCME()
def __enter__(self) -> ElastiCacheClusterCME:
self.cluster = ElastiCacheClusterCME.launch(self.config)
return self.cluster
def __exit__(self, exc_type, exc_val, exc_tb):
self.cluster.delete_cluster()
```
## (Optional) Create a wrapper class to demo client connection to Valkey or Redis OSS cluster
Now, let’s create a wrapper class for the `redis-py-cluster` client. This wrapper class will support pre-filling the cluster with some keys and then doing random repeated `get` commands.
**Note**
This is an optional step but it simplifies the code of the main function that comes in a later step.
```
import redis
improt random
from time import perf_counter_ns, time
class DowntimeTestClient:
def __init__(self, client):
self.client = client
# num of keys prefilled
self.prefilled = 0
# percent of get above prefilled
self.percent_get_above_prefilled = 10 # nil result expected when get hit above prefilled
# total downtime in nano seconds
self.downtime_ns = 0
# num of success and fail operations
self.success_ops = 0
self.fail_ops = 0
self.connection_errors = 0
self.timeout_errors = 0
def replace_client(self, client):
self.client = client
def prefill_data(self, timelimit_sec=60):
end_time = time() + timelimit_sec
while time() < end_time:
self.client.set(self.prefilled, self.prefilled)
self.prefilled += 1
# unsuccesful operations throw exceptions
def _exec(self, func):
try:
start_ns = perf_counter_ns()
func()
self.success_ops += 1
elapsed_ms = (perf_counter_ns() - start_ns) // 10 ** 6
# upon succesful execution of func
# reset random_key to None so that the next command
# will use a new random key
self.random_key = None
except Exception as e:
elapsed_ns = perf_counter_ns() - start_ns
self.downtime_ns += elapsed_ns
# in case of failure- increment the relevant counters so that we will keep track
# of how many connection issues we had while trying to communicate with
# the cluster.
self.fail_ops += 1
if e.__class__ is redis.exceptions.ConnectionError:
self.connection_errors += 1
if e.__class__ is redis.exceptions.TimeoutError:
self.timeout_errors += 1
def _repeat_exec(self, func, seconds):
end_time = time() + seconds
while time() < end_time:
self._exec(func)
def _new_random_key_if_needed(self, percent_above_prefilled):
if self.random_key is None:
max = int((self.prefilled * (100 + percent_above_prefilled)) / 100)
return random.randint(0, max)
return self.random_key
def _random_get(self):
key = self._new_random_key_if_needed(self.percent_get_above_prefilled)
result = self.client.get(key)
# we know the key was set for sure only in the case key < self.prefilled
if key < self.prefilled:
assert result.decode("UTF-8") == str(key)
def repeat_get(self, seconds=60):
self._repeat_exec(self._random_get, seconds)
def get_downtime_ms(self) -> int:
return self.downtime_ns // 10 ** 6
def do_get_until(self, cond_check):
while not cond_check():
self.repeat_get()
# do one more get cycle once condition is met
self.repeat_get()
```
## Create the main function that demos the process of changing in-transit encryption configuration
Now, let’s define the main function, which will do the following:
1. Create the cluster using boto3 ElastiCache client.
1. Initialize the `redis-py-cluster` client that will connect to the cluster with a clear TCP connection without TLS.
1. The `redis-py-cluster` client prefills the cluster with some data.
1. The boto3 client will trigger TLS migration from no-TLS to TLS preferred.
1. While the cluster is being migrated to TLS `Preferred`, the `redis-py-cluster` TCP client will send repeated `get` operations to the cluster until the migration is finished.
1. After the migration to TLS `Preferred` is finished, we will assert that the cluster supports in-transit encryption. Afterwards, we will create a `redis-py-cluster` client that will connect to the cluster with TLS.
1. We will send some `get` commands using the new TLS client and the old TCP client.
1. The boto3 client will trigger TLS migration from TLS `Preferred` to TLS required.
1. While the cluster is being migrated to TLS required, the redis-py-cluster TLS client will send repeated `get` operations to the cluster until the migration is finished.
```
import redis
def init_cluster_client(
cluster: ElastiCacheClusterCME, prefill_data: bool, TLS: bool = True) -> DowntimeTestClient:
# we must use for the host name the cluster configuration endpoint.
redis_client = redis.RedisCluster(
host=cluster.get_configuration_endpoint(), ssl=TLS, socket_timeout=0.25, socket_connect_timeout=0.1
)
test_client = DowntimeTestClient(redis_client)
if prefill_data:
test_client.prefill_data()
return test_client
if __name__ == '__main__':
config = ConfigCME(TLS=False, instance_type="cache.m5.large")
with ElastiCacheCMEManager(config) as cluster:
# create a client that will connect to the cluster with clear tcp connection
test_client_tcp = init_cluster_client(cluster, prefill_data=True, TLS=False)
# migrate the cluster to TLS Preferred
cluster.modify_transit_encryption_mode(new_transit_encryption_mode="preferred")
# do repeated get commands until the cluster finishes the migration to TLS Preferred
test_client_tcp.do_get_until(cluster.is_available)
# verify that in transit encryption is enabled so that clients will be able to connect to the cluster with TLS
assert cluster.get_transit_encryption_enabled() == True
# create a client that will connect to the cluster with TLS connection.
# we must first make sure that the cluster indeed supports TLS
test_client_tls = init_cluster_client(cluster, prefill_data=True, TLS=True)
# by doing get commands with the tcp client for 60 more seconds
# we can verify that the existing tcp connection to the cluster still works
test_client_tcp.repeat_get(seconds=60)
# do get commands with the new TLS client for 60 more seconds
test_client_tcp.repeat_get(seconds=60)
# migrate the cluster to TLS required
cluster.modify_transit_encryption_mode(new_transit_encryption_mode="required")
# from this point the tcp clients will be disconnected and we must not use them anymore.
# do get commands with the TLS client until the cluster finishes migartion to TLS required mode.
test_client_tls.do_get_until(cluster.is_available)
```
# Best practices when enabling in-transit encryption
## Before enabling in-transit encryption: make sure you have proper DNS records handling
**Note**
We are changing and deleting old endpoints during this process. Incorrect usage of the endpoints can result in the Valkey or Redis OSS client using old and deleted endpoints that will prevent it from connecting to the cluster.
While the cluster is being migrated from no-TLS to TLS-preferred, the old cluster configuration endpoint DNS record is kept and the new cluster configuration endpoint DNS records are being generated in a different format. TLS-enabled clusters use a different format of DNS records than TLS-disabled clusters. ElastiCache will keep both DNS records when a cluster is configured in `encryption mode: Preferred` so that Applications and other Valkey or Redis OSS clients can switch between them. The following changes in the DNS records take place during the TLS-migration process:
### Description of the changes in the DNS records that take place when enabling in-transit encryption
**For CME clusters**
When a cluster is set to ‘transit encryption mode: preferred’:
+ The original cluster configuration endpoint for no-TLS cluster will remain active. There will be no downtime when cluster is re-configured form TLS encryption mode ‘none’ to ‘preferred’.
+ New TLS Valkey or Redis OSS endpoints will be generated when cluster is set to TLS-preferred mode. These new endpoints will resolve to the same IPs as the old ones (non-TLS).
+ The new TLS Valkey or Redis OSS configuration endpoint will be exposed in the ElastiCache Console and in the response to `describe-replication-group` API.
When a cluster is set to ‘transit encryption mode: required’:
+ Old non-TLS enabled endpoints will be deleted. There will be no downtime of TLS cluster endpoints.
+ You can retrieve a new `cluster-configuration-endpoint` from ElastiCache Console or from the `describe-replication-group` API.
**For CMD clusters with Automatic Failover enabled or Automatic Failover disabled**
When replication group is set to ‘transit encryption mode: preferred’:
+ The original primary endpoint and reader endpoint for non-TLS enabled cluster will remain active.
+ New TLS primary and reader endpoints will be generated when cluster is set to TLS `Preferred` mode. This new endpoints will resolve to the same IP(s) as the old ones (non-TLS).
+ The new primary endpoint and reader endpoint will be exposed in the ElastiCache Console and in the response to the `describe-replication-group` API.
When replication group is set to ‘transit encryption mode: required’:
+ Old non-TLS primary and reader endpoints will be deleted. There will be no downtime of TLS cluster endpoints.
+ You can retrieve new primary and reader endpoints from ElastiCache Console or from the `describe-replication-group` API.
### The suggested usage of the DNS records
**For CME clusters **
+ Use the cluster configuration endpoint instead of per-node DNS records in your application’s code. Using per-node DNS names directly is not recommended because during migration they will change and the application code will break connection to the cluster.
+ Don't hardcode a cluster configuration endpoint in your application, as it will change during this process.
+ Having the cluster configuration endpoint hardcoded in your application is a bad practice, since it can be changed during this process. After the in-transit encryption is completed, query the cluster configuration endpoint with the `describe-replication-group` API (as demonstrated above (in bold)) and use the DNS you get in response from that point on.
**For CMD clusters with Automatic Failover enabled **
+ Use the primary endpoint and reader endpoint instead of the per-node DNS names in your application’s code since the old per-node DNS names are deleted and new ones are generated when migrating the cluster from no-TLS to TLS-preferred. Using per-node DNS names directly is not recommended because you might add replicas to your cluster in the future. Also, when Automatic Failover is enabled, the roles of the primary cluster and replicas are changed automatically by the ElastiCache service, using the primary endpoint and reader endpoint is suggested to help you keep track of those changes. Lastly, using the reader endpoint will help you distribute your reads from the replicas equally between the replicas in the cluster.
+ Having the primary endpoint and reader endpoint hardcoded in your application is a bad practice since it can be changed during the TLS migration process. After the migration change to TLS-preferred is completed, query the primary endpoint and reader endpoint endpoint with the describe-replication-group API and use the DNS you get in response from this point on. This way you will be able to keep track of changes in endpoints in a dynamic way.
**For CMD cluster with Automatic Failover disabled **
+ Use the primary endpoint and reader endpoint instead of the per-node DNS names in your application’s code. When Automatic Failover is disabled, scaling, patching, failover, and other procedures that are managed automatically by the ElastiCache service when Automatic Failover is enabled are done by you instead. This makes it easier for you to manually keep track of the different endpoints. Since the old per-node DNS names are deleted and new ones are generated when migrating the cluster from no-TLS to TLS-preferred, do not use the per-node DNS names directly. This is mandatory so that clients will be able to connect to the cluster during the TLS-migration. Also, you’ll benefit from evenly spreading the reads between the replicas when using the reader endpoint and keep track of the DNS-records when adding or deleting replicas form the cluster.
+ Having the cluster configuration endpoint hardcoded in your application is a bad practice since it can be changed during the TLS migration process.
## During the in-transit encryption: pay attention to when the migration process finishes
Change of transit encryption mode is not immediate and can take some time. This is especially true for large clusters. Only when the cluster finishes the migration to TLS-preferred is it able to accept and serve both TCP and TLS connections. Therefore, you should not create clients that will try to establish TLS connections to the cluster until the in-transit encryption is completed.
There are several ways to get notified when the in-transit encryption is completed successfully or failed: (Not shown in the code example above):
+ Using the SNS service to get a notification when the encryption is completed
+ Using the `describe-events` API that will emit an event when the encryption is completed
+ Seeing a message in the ElastiCache Console that the encryption is completed
You can also implement logic in your application to know if the encryption is completed. In the example above, we saw several ways to ensure the cluster finishes the migration:
+ Waiting until the migration process starts (the cluster status changes to “modifying“), and waiting until the modification is finished (the cluster status changes back to “available“)
+ Asserting that the cluster has `transit_encryption_enabled` set to True by querying the `describe-replication-group` API.
### After enabling in-transit encryption: make sure the clients you use are configured properly
While the cluster is in TLS-preferred mode, your application should open TLS connections to the cluster and only use those connections. This way your application will not experience downtime when enabling in-transit encryption. You can make sure that there are no clearer TCP connections to the Valkey or Redis OSS engine using the info command under the SSL section.
```
# SSL
ssl_enabled:yes
ssl_current_certificate_not_before_date:Mar 20 23:27:07 2017 GMT
ssl_current_certificate_not_after_date:Feb 24 23:27:07 2117 GMT
ssl_current_certificate_serial:D8C7DEA91E684163
tls_mode_connected_tcp_clients:0 (should be zero)
tls_mode_connected_tls_clients:100
```
## Connecting to nodes enabled with in-transit encryption using Openssl (Memcached)
To access data from ElastiCache for Memcached nodes enabled with in-transit encryption, you need to use clients that work with Secure Socket Layer (SSL). You can also use Openssl s\$1client on Amazon Linux and Amazon Linux 2.
To use Openssl s\$1client to connect to a Memcached cluster enabled with in-transit encryption on Amazon Linux 2 or Amazon Linux:
```
/usr/bin/openssl s_client -connect memcached-node-endpoint:memcached-port
```
## Creating a TLS Memcached client using Java
To create a client in TLS mode, do the following to initialize the client with the appropriate SSLContext:
```
import java.security.KeyStore;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.ConnectionFactoryBuilder;
import net.spy.memcached.MemcachedClient;
public class TLSDemo {
public static void main(String[] args) throws Exception {
ConnectionFactoryBuilder connectionFactoryBuilder = new ConnectionFactoryBuilder();
// Build SSLContext
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init((KeyStore) null);
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, tmf.getTrustManagers(), null);
// Create the client in TLS mode
connectionFactoryBuilder.setSSLContext(sslContext);
MemcachedClient client = new MemcachedClient(connectionFactoryBuilder.build(), AddrUtil.getAddresses("mycluster.fnjyzo.cfg.use1.cache.amazonaws.com:11211"));
// Store a data item for an hour.
client.set("theKey", 3600, "This is the data value");
}
}
```
## Creating a TLS Memcached client using PHP
To create a client in TLS mode, do the following to initialize the client with the appropriate SSLContext:
```
setOption(Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
/* Add the memcached's cluster server/s */
$tls_client->addServer($server_endpoint, $server_port);
/* Configure the client to use TLS */
if(!$tls_client->setOption(Memcached::OPT_USE_TLS, 1)) {
echo $tls_client->getLastErrorMessage(), "\n";
exit(1);
}
/* Set your TLS context configurations values.
* See MemcachedTLSContextConfig in memcached-api.php for all configurations */
$tls_config = new MemcachedTLSContextConfig();
$tls_config->hostname = '*.mycluster.fnjyzo.use1.cache.amazonaws.com';
$tls_config->skip_cert_verify = false;
$tls_config->skip_hostname_verify = false;
/* Use the created TLS context configuration object to create OpenSSL's SSL_CTX and set it to your client.
* Note: These TLS context configurations will be applied to all the servers connected to this client. */
$tls_client->createAndSetTLSContext((array)$tls_config);
/* test the TLS connection with set-get scenario: */
/* store the data for 60 seconds in the cluster.
* The client will decide which cache host will store this item.
*/
if($tls_client->set('key', 'value', 60)) {
print "Successfully stored key\n";
} else {
echo "Failed to set key: ", $tls_client->getLastErrorMessage(), "\n";
exit(1);
}
/* retrieve the key */
if ($tls_client->get('key') === 'value') {
print "Successfully retrieved key\n";
} else {
echo "Failed to get key: ", $tls_client->getLastErrorMessage(), "\n";
exit(1);
}
```
For more information on using the PHP client, see [Installing the ElastiCache cluster client for PHP](Appendix.PHPAutoDiscoverySetup.md).