# Security in AWS ParallelCluster
Cloud security at AWS is the highest priority. As an AWS customer, you benefit from a data center and network architecture that is built to meet the requirements of the most security sensitive organizations.
Security is a shared responsibility between AWS and you. The [shared responsibility model](https://aws.amazon.com/compliance/shared-responsibility-model/) describes this as security *of* the cloud and security *in* the cloud:
+ **Security of the cloud** – AWS is responsible for protecting the infrastructure that runs AWS services in the AWS Cloud. AWS also provides you with services that you can use securely. Third-party auditors regularly test and verify the effectiveness of our security as part of the [AWS Compliance Programs](https://aws.amazon.com/compliance/programs/). To learn about the compliance programs that apply to AWS ParallelCluster, see [AWS Services in Scope by Compliance Program](https://aws.amazon.com/compliance/services-in-scope/).
+ **Security in the cloud** – Your responsibility is determined by the specific AWS service or services that you use. You are also responsible for several other related factors including the sensitivity of your data, your company’s requirements, and applicable laws and regulations.
This documentation describes how you should apply the shared responsibility model when using AWS ParallelCluster. The following topics show you how to configure AWS ParallelCluster to meet your security and compliance objectives. You also learn how to use AWS ParallelCluster in a way that helps you to monitor and secure your AWS resources.
**Topics**
+ [Security information for services used by AWS ParallelCluster](#security-seealso)
+ [Data protection in AWS ParallelCluster](data-protection.md)
+ [Identity and Access Management for AWS ParallelCluster](security-iam.md)
+ [Compliance validation for AWS ParallelCluster](security-compliance-validation.md)
+ [Enforcing a Minimum Version of TLS 1.2](security-enforcing-tls.md)
+ [Configuring security groups for restricted environments](security-groups-configuration.md)
## Security information for services used by AWS ParallelCluster
+ [Security in Amazon EC2 ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security.html)
+ [Security in Amazon API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/security.html)
+ [Security in AWS Batch](https://docs.aws.amazon.com/batch/latest/userguide/security.html)
+ [Security in CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/security.html)
+ [Security in Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/security.html)
+ [Security in AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/security.html)
+ [Security in Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/security.html)
+ [Security in Amazon ECR](https://docs.aws.amazon.com/AmazonECR/latest/userguide/security.html)
+ [Security in Amazon ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security.html)
+ [Security in Amazon EFS](https://docs.aws.amazon.com/efs/latest/ug/security-considerations.html)
+ [Security in FSx for Lustre](https://docs.aws.amazon.com/fsx/latest/LustreGuide/security.html)
+ [Security in AWS Identity and Access Management (IAM) ](https://docs.aws.amazon.com/IAM/latest/UserGuide/security.html)
+ [Security in EC2 Image Builder](https://docs.aws.amazon.com/imagebuilder/latest/userguide/image-builder-security.html)
+ [Security in AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/lambda-security.html)
+ [Security in Amazon Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/security.html)
+ [Security in Amazon SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-security.html)
+ [Security in Amazon SQS (For AWS ParallelCluster version 2.x.)](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-security.html)
+ [Security in Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/security.html)
+ [Security in Amazon VPC](https://docs.aws.amazon.com/vpc/latest/userguide/security.html)
# Data protection in AWS ParallelCluster
The AWS [shared responsibility model](https://aws.amazon.com/compliance/shared-responsibility-model/) applies to data protection in . As described in this model, AWS is responsible for protecting the global infrastructure that runs all of the AWS Cloud. You are responsible for maintaining control over your content that is hosted on this infrastructure. You are also responsible for the security configuration and management tasks for the AWS services that you use. For more information about data privacy, see the [Data Privacy FAQ](https://aws.amazon.com/compliance/data-privacy-faq/). For information about data protection in Europe, see the [AWS Shared Responsibility Model and GDPR](https://aws.amazon.com/blogs/security/the-aws-shared-responsibility-model-and-gdpr/) blog post on the *AWS Security Blog*.
For data protection purposes, we recommend that you protect AWS account credentials and set up individual users with AWS IAM Identity Center or AWS Identity and Access Management (IAM). That way, each user is given only the permissions necessary to fulfill their job duties. We also recommend that you secure your data in the following ways:
+ Use multi-factor authentication (MFA) with each account.
+ Use SSL/TLS to communicate with AWS resources. We require TLS 1.2 and recommend TLS 1.3.
+ Set up API and user activity logging with AWS CloudTrail. For information about using CloudTrail trails to capture AWS activities, see [Working with CloudTrail trails](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-trails.html) in the *AWS CloudTrail User Guide*.
+ Use AWS encryption solutions, along with all default security controls within AWS services.
+ Use advanced managed security services such as Amazon Macie, which assists in discovering and securing sensitive data that is stored in Amazon S3.
+ If you require FIPS 140-3 validated cryptographic modules when accessing AWS through a command line interface or an API, use a FIPS endpoint. For more information about the available FIPS endpoints, see [Federal Information Processing Standard (FIPS) 140-3](https://aws.amazon.com/compliance/fips/).
We strongly recommend that you never put confidential or sensitive information, such as your customers' email addresses, into tags or free-form text fields such as a **Name** field. This includes when you work with or other AWS services using the console, API, AWS CLI, or AWS SDKs. Any data that you enter into tags or free-form text fields used for names may be used for billing or diagnostic logs. If you provide a URL to an external server, we strongly recommend that you do not include credentials information in the URL to validate your request to that server.
## Data encryption
A key feature of any secure service is that information is encrypted when it is not being actively used.
### Encryption at rest
AWS ParallelCluster does not itself store any customer data other than the credentials it needs to interact with the AWS services on the user's behalf.
For data on the nodes in the cluster, data can be encrypted at rest.
For Amazon EBS volumes, encryption is configured using the [`EbsSettings`](SharedStorage-v3.md#SharedStorage-v3-EbsSettings)/`Encrypted` and [`EbsSettings`](SharedStorage-v3.md#SharedStorage-v3-EbsSettings)/`KmsKeyId` settings in the [`EbsSettings`](SharedStorage-v3.md#SharedStorage-v3-EbsSettings) section. For more information, see [Amazon EBS encryption](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html) in the Amazon EC2 User Guide.
For Amazon EFS volumes, encryption is configured using the [`EfsSettings`](SharedStorage-v3.md#SharedStorage-v3-EfsSettings)/`Encrypted` and [`EfsSettings`](SharedStorage-v3.md#SharedStorage-v3-EfsSettings)/`KmsKeyId` settings in the [`EfsSettings`](SharedStorage-v3.md#SharedStorage-v3-EfsSettings) section. For more information, see [How encryption at rest works](https://docs.aws.amazon.com/efs/latest/ug/encryption-at-rest.html#howencrypt) in the *Amazon Elastic File System User Guide.*
For FSx for Lustre file systems, encryption of data at rest is automatically enabled when creating an Amazon FSx file system. For more information, see [Encrypting data at rest](https://docs.aws.amazon.com/fsx/latest/LustreGuide/encryption-at-rest.html) in the *Amazon FSx for Lustre User Guide*.
For instance types with NVMe volumes, the data on NVMe instance store volumes is encrypted using an XTS-AES-256 cipher implemented on a hardware module on the instance. The encryption keys are generated using the hardware module and are unique to each NVMe instance storage device. All encryption keys are destroyed when the instance is stopped or terminated and cannot be recovered. You cannot disable this encryption and you cannot provide your own encryption key. For more information, see [Encryption at rest](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/data-protection.html#encryption-rest) in the *Amazon EC2 User Guide*.
If you use AWS ParallelCluster to invoke an AWS service that transmits customer data to your local computer for storage, then refer to the Security and Compliance chapter in that service's User Guide for information on how that data is stored, protected, and encrypted.
### Encryption in transit
By default, all data transmitted from the client computer running AWS ParallelCluster and AWS service endpoints is encrypted by sending everything through a HTTPS/TLS connection. Traffic between the nodes in the cluster can be automatically encrypted, depending on the instance types selected. For more information, see [Encryption in transit](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/data-protection.html#encryption-transit) in the *Amazon EC2 User Guide*.
## See also
+ [Data protection in Amazon EC2 ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/data-protection.html)
+ [Data protection in EC2 Image Builder](https://docs.aws.amazon.com/imagebuilder/latest/userguide/data-protection.html)
+ [Data protection in CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/security-data-protection.html)
+ [Data protection in Amazon EFS](https://docs.aws.amazon.com/efs/latest/ug/efs-backup-solutions.html)
+ [Data protection in Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurability.html)
+ [Data protection in FSx for Lustre](https://docs.aws.amazon.com/fsx/latest/LustreGuide/data-protection.html)
# Identity and Access Management for AWS ParallelCluster
AWS ParallelCluster uses roles to access your AWS resources and their services. The instance and user policies that AWS ParallelCluster uses to grant permissions are documented at [AWS Identity and Access Management permissions in AWS ParallelCluster](iam-roles-in-parallelcluster-v3.md).
The only major difference is how you authenticate when using a standard user and long-term credentials. Although an user requires a password to access an AWS service's console, that same user requires an access key pair to perform the same operations using AWS ParallelCluster. All other short-term credentials are used in the same way they are used with the console.
The credentials used by AWS ParallelCluster are stored in plaintext files and are ***not*** encrypted.
+ The `$HOME/.aws/credentials` file stores long-term credentials required to access your AWS resources. These include your access key ID and secret access key.
+ Short-term credentials, such as those for roles that you assume, or that are for AWS IAM Identity Center services, are also stored in the `$HOME/.aws/cli/cache` and `$HOME/.aws/sso/cache` folders, respectively.
**Mitigation of Risk**
+ We strongly recommend that you configure your file system permissions on the `$HOME/.aws` folder and its child folders and files to restrict access to only authorized users.
+ Use roles with temporary credentials wherever possible to reduce the opportunity for damage if the credentials are compromised. Use long-term credentials only to request and refresh short-term role credentials.
# Compliance validation for AWS ParallelCluster
Third-party auditors assess the security and compliance of AWS services as part of multiple AWS compliance programs. Using AWS ParallelCluster to access a service does not alter that service's compliance.
For a list of AWS services in scope of specific compliance programs, see [AWS services in scope by compliance program](https://aws.amazon.com/compliance/services-in-scope/). For general information, see [AWS compliance programs](https://aws.amazon.com/compliance/programs/).
You can download third-party audit reports using the AWS Artifact. For more information, see [Downloading reports in AWS Artifact](https://docs.aws.amazon.com/artifact/latest/ug/downloading-documents.html).
Your compliance responsibility when using AWS ParallelCluster is determined by the sensitivity of your data, your company's compliance objectives, and applicable laws and regulations. AWS provides the following resources to help with compliance:
+ [Security and compliance quick start guides](https://aws.amazon.com/quickstart/?awsf.quickstart-homepage-filter=categories%23security-identity-compliance) – These deployment guides discuss architectural considerations and provide steps for deploying security- and compliance-focused baseline environments on AWS.
+ [Architecting for HIPAA security and Compliance on Amazon Web Services AWS Whitepaper ](https://docs.aws.amazon.com/pdfs/whitepapers/latest/architecting-hipaa-security-and-compliance-on-aws/architecting-hipaa-security-and-compliance-on-aws.pdf) – This whitepaper describes how companies can use AWS to create HIPAA-compliant applications.
+ [AWS compliance resources](https://aws.amazon.com/compliance/resources/) – This collection of workbooks and guides might apply to your industry and location.
+ [Evaluating resources with rules](https://docs.aws.amazon.com/config/latest/developerguide/evaluate-config.html) in the *AWS Config Developer Guide* – The AWS Config service assesses how well your resource configurations comply with internal practices, industry guidelines, and regulations.
+ [AWS Security Hub CSPM](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub.html) – This AWS service provides a comprehensive view of your security state within AWS that helps you check your compliance with security industry standards and best practices.
# Enforcing a Minimum Version of TLS 1.2
To add increased security when communicating with AWS services, you should configure your AWS ParallelCluster to use TLS 1.2 or later. When you use AWS ParallelCluster, Python is used to set the TLS version.
To ensure AWS ParallelCluster uses no TLS version earlier than TLS 1.2, you might need to recompile OpenSSL to enforce this minimum and then recompile Python to use the newly built OpenSSL.
## Determine Your Currently Supported Protocols
First, create a self-signed certificate to use for the test server and the Python SDK using OpenSSL.
```
$ openssl req -subj '/CN=localhost' -x509 -newkey rsa:4096 -nodes -keyout key.pem -out cert.pem -days 365
```
Then spin up a test server using OpenSSL.
```
$ openssl s_server -key key.pem -cert cert.pem -www
```
In a new terminal window, create a virtual environment and install the Python SDK.
```
$ python3 -m venv test-env
source test-env/bin/activate
pip install botocore
```
Create a new Python script named `check.py` that uses the SDK’s underlying HTTP library.
```
$ import urllib3
URL = 'https://localhost:4433/'
http = urllib3.PoolManager(
ca_certs='cert.pem',
cert_reqs='CERT_REQUIRED',
)
r = http.request('GET', URL)
print(r.data.decode('utf-8'))
```
Run your new script.
```
$ python check.py
```
This displays details about the connection made. Search for "Protocol : " in the output. If the output is "TLSv1.2" or later, the SDK defaults to TLS v1.2 or later. If it's an earlier version, you need to recompile OpenSSL and recompile Python.
However, even if your installation of Python defaults to TLS v1.2 or later, it's still possible for Python to renegotiate to a version earlier than TLS v1.2 if the server doesn't support TLS v1.2 or later. To check that Python doesn't automatically renegotiate to earlier versions, restart the test server with the following.
```
$ openssl s_server -key key.pem -cert cert.pem -no_tls1_3 -no_tls1_2 -www
```
If you're using an earlier version of OpenSSL, you might not have the `-no_tls_3` flag available. If this is the case, remove the flag because the version of OpenSSL you're using doesn't support TLS v1.3. Then rerun the Python script.
```
$ python check.py
```
If your installation of Python correctly doesn't renegotiate for versions earlier than TLS 1.2, you should receive an SSL error.
```
$ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=4433): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1108)')))
```
If you're able to make a connection, you need to recompile OpenSSL and Python to disable negotiation of protocols earlier than TLS v1.2.
## Compile OpenSSL and Python
To ensure that AWS ParallelCluster doesn't negotiate for anything earlier than TLS 1.2, you need to recompile OpenSSL and Python. To do this, copy the following content to create a script and run it.
```
#!/usr/bin/env bash
set -e
OPENSSL_VERSION="1.1.1d"
OPENSSL_PREFIX="/opt/openssl-with-min-tls1_2"
PYTHON_VERSION="3.8.1"
PYTHON_PREFIX="/opt/python-with-min-tls1_2"
curl -O "https://www.openssl.org/source/openssl-$OPENSSL_VERSION.tar.gz"
tar -xzf "openssl-$OPENSSL_VERSION.tar.gz"
cd openssl-$OPENSSL_VERSION
./config --prefix=$OPENSSL_PREFIX no-ssl3 no-tls1 no-tls1_1 no-shared
make > /dev/null
sudo make install_sw > /dev/null
cd /tmp
curl -O "https://www.python.org/ftp/python/$PYTHON_VERSION/Python-$PYTHON_VERSION.tgz"
tar -xzf "Python-$PYTHON_VERSION.tgz"
cd Python-$PYTHON_VERSION
./configure --prefix=$PYTHON_PREFIX --with-openssl=$OPENSSL_PREFIX --disable-shared > /dev/null
make > /dev/null
sudo make install > /dev/null
```
This compiles a version of Python that has a statically linked OpenSSL that doesn't automatically negotiate anything earlier than TLS 1.2. This also installs OpenSSL in the `/opt/openssl-with-min-tls1_2` directory and installs Python in the `/opt/python-with-min-tls1_2` directory. After you run this script, confirm installation of the new version of Python.
```
$ /opt/python-with-min-tls1_2/bin/python3 --version
```
This should print out the following.
```
Python 3.8.1
```
To confirm this new version of Python doesn't negotiate a version earlier than TLS 1.2, rerun the steps from [Determine Your Currently Supported Protocols](#enforcing-tls-supported) using the newly installed Python version (that is, `/opt/python-with-min-tls1_2/bin/python3`).
# Configuring security groups for restricted environments
By default, AWS ParallelCluster creates and configures security groups that allow all traffic between cluster nodes. In highly restricted environments, you might need to limit network access to only the ports required for cluster operation. This section describes how to configure custom security groups with restricted access for your AWS ParallelCluster deployment.
## Security groups overview
AWS ParallelCluster uses security groups to control network traffic between the head node, compute nodes, and login nodes (if configured). By default, when AWS ParallelCluster creates a cluster, it creates security groups that allow all traffic between nodes within the cluster. In environments with strict security requirements, you can provide custom security groups that limit traffic to only the necessary ports.
Security groups can be configured in the following sections of your cluster configuration:
+ [`HeadNode` / `Networking`](HeadNode-v3.md#HeadNode-v3-Networking) - Controls access to and from the head node
+ [`Scheduling` / `SlurmQueues` / `Networking`](Scheduling-v3.md#Scheduling-v3-SlurmQueues-Networking) - Controls access to and from compute nodes
+ [`LoginNodes`](LoginNodes-v3.md) - Controls access to and from login nodes (if configured)
For each of these sections, you can specify:
+ `SecurityGroups` - Replaces the default security groups that AWS ParallelCluster would create
+ `AdditionalSecurityGroups` - Adds security groups in addition to the default ones created by AWS ParallelCluster
## Required ports for cluster operation
When configuring custom security groups, you must ensure that the following ports are open between the appropriate nodes:
**Required ports for head node**
| Port | Protocol | Direction | Purpose |
| --- | --- | --- | --- |
| 22 | TCP | Inbound | SSH access to the head node (from allowed IP ranges) |
| 6817-6819 | TCP | Inbound | Slurm controller ports (from compute and login nodes) |
| 6817-6819 | TCP | Outbound | Slurm controller ports (to compute and login nodes) |
| 8443 | TCP | Inbound | NICE DCV (if enabled, from allowed IP ranges) |
| 111, 2049 | TCP/UDP | Inbound | NFS (from compute and login nodes, if using NFS for shared storage) |
| 443 | TCP | Outbound | HTTPS access to AWS services (if not using VPC endpoints) |
**Required ports for compute nodes**
| Port | Protocol | Direction | Purpose |
| --- | --- | --- | --- |
| 22 | TCP | Inbound | SSH access (from head node and login nodes) |
| 6818 | TCP | Inbound | Slurm daemon port (from head node) |
| 6817-6819 | TCP | Outbound | Slurm controller ports (to head node) |
| 111, 2049 | TCP/UDP | Outbound | NFS (to head node, if using NFS for shared storage) |
| 443 | TCP | Outbound | HTTPS access to AWS services (if not using VPC endpoints) |
If you're using EFA (Elastic Fabric Adapter), you must also allow all traffic between compute nodes that have EFA enabled:
+ All TCP and UDP traffic between compute nodes with EFA
+ All traffic on the EFA device between compute nodes with EFA
**Note**
If you're using shared storage systems like FSx for Lustre, Amazon EFS, or other storage solutions, you'll need to ensure that the appropriate ports are open for those services as well.
## Creating custom security groups
To create custom security groups for your AWS ParallelCluster deployment, follow these steps:
1. Create security groups for the head node, compute nodes, and login nodes (if applicable) using the AWS Management Console, AWS CLI, or AWS CloudFormation.
1. Configure the security group rules to allow only the necessary traffic as outlined in the previous section.
1. Reference these security groups in your cluster configuration file.
Here's an example of how to create security groups using the AWS CLI:
```
# Create security group for head node
aws ec2 create-security-group \
--group-name pcluster-head-node-sg \
--description "Security group for ParallelCluster head node" \
--vpc-id vpc-12345678
# Create security group for compute nodes
aws ec2 create-security-group \
--group-name pcluster-compute-node-sg \
--description "Security group for ParallelCluster compute nodes" \
--vpc-id vpc-12345678
# Add rules to allow necessary traffic between head and compute nodes
# (Add specific rules based on the required ports listed above)
```
## Configuring security groups in the cluster configuration
Once you've created your custom security groups, you can reference them in your cluster configuration file:
```
# Example cluster configuration with custom security groups
HeadNode:
...
Networking:
SubnetId: subnet-12345678
SecurityGroups:
- sg-headnode12345 # Custom security group for head node
# Or use AdditionalSecurityGroups if you want to keep the default security groups
# AdditionalSecurityGroups:
# - sg-additional12345
...
Scheduling:
Scheduler: slurm
SlurmQueues:
- Name: queue1
...
Networking:
SubnetIds:
- subnet-12345678
SecurityGroups:
- sg-computenode12345 # Custom security group for compute nodes
# Or use AdditionalSecurityGroups if you want to keep the default security groups
# AdditionalSecurityGroups:
# - sg-additional12345
...
# If using login nodes
LoginNodes:
Pools:
- Name: login-pool
...
Networking:
SubnetIds:
- subnet-12345678
SecurityGroups:
- sg-loginnode12345 # Custom security group for login nodes
# Or use AdditionalSecurityGroups if you want to keep the default security groups
# AdditionalSecurityGroups:
# - sg-additional12345
...
```
When using `SecurityGroups`, AWS ParallelCluster will use only the security groups you specify, replacing the default ones. When using `AdditionalSecurityGroups`, AWS ParallelCluster will use both the default security groups it creates and the additional ones you specify.
**Warning**
If you enable [Elastic Fabric Adapter (EFA)](Scheduling-v3.md#yaml-Scheduling-SlurmQueues-ComputeResources-Efa) for your compute instances, make sure that your EFA-enabled instances are members of a security group that allows all inbound and outbound traffic to itself. This is required for EFA to function properly.
## Using VPC endpoints in restricted environments
In highly restricted environments, you might want to deploy AWS ParallelCluster in a subnet without internet access. In this case, you'll need to configure VPC endpoints to allow the cluster to communicate with AWS services. For detailed instructions, see [AWS ParallelCluster in a single subnet with no internet access](aws-parallelcluster-in-a-single-public-subnet-no-internet-v3.md).
When using VPC endpoints, ensure that your security groups allow traffic to and from the VPC endpoints. You can do this by adding the security groups associated with the VPC endpoints to the `AdditionalSecurityGroups` configuration for your head node and compute nodes.
```
HeadNode:
...
Networking:
SubnetId: subnet-1234567890abcdef0
AdditionalSecurityGroups:
- sg-abcdef01234567890 # Security group that enables communication with VPC endpoints
...
Scheduling:
Scheduler: slurm
SlurmQueues:
- ...
Networking:
SubnetIds:
- subnet-1234567890abcdef0
AdditionalSecurityGroups:
- sg-1abcdef01234567890 # Security group that enables communication with VPC endpoints
```
## Best practices for security group configuration
When configuring security groups for AWS ParallelCluster in restricted environments, consider the following best practices:
+ **Principle of least privilege**: Only open the ports that are necessary for cluster operation.
+ **Use security group references**: When possible, use security group references (allowing traffic from another security group) rather than CIDR blocks to limit traffic between cluster components.
+ **Restrict SSH access**: Limit SSH access to the head node to only the IP ranges that need it using the [`HeadNode` / `Ssh` / `AllowedIps`](HeadNode-v3.md#yaml-HeadNode-Ssh-AllowedIps) configuration.
+ **Restrict DCV access**: If using NICE DCV, limit access to only the IP ranges that need it using the [`HeadNode` / `Dcv` / `AllowedIps`](HeadNode-v3.md#yaml-HeadNode-Dcv-AllowedIps) configuration.
+ **Test thoroughly**: After configuring custom security groups, thoroughly test all cluster functionality to ensure that all required communication paths are working.
+ **Document your configuration**: Maintain documentation of your security group configuration, including which ports are open and why they are needed.
## Troubleshooting security group issues
If you encounter issues after configuring custom security groups, consider the following troubleshooting steps:
+ **Check cluster logs**: Review the cluster logs in CloudWatch Logs for any connection errors.
+ **Verify security group rules**: Ensure that all required ports are open between the appropriate nodes.
+ **Test connectivity**: Use tools like `telnet` or `nc` to test connectivity between nodes on specific ports.
+ **Temporarily expand rules**: If you're having trouble identifying which ports are needed, temporarily allow all traffic between cluster nodes and then gradually restrict it as you identify the required ports.
+ **Check VPC endpoint configuration**: If you're using VPC endpoints, ensure that they are properly configured and that the security groups allow traffic to and from them.
If you continue to experience issues, you can revert to using the default security groups created by AWS ParallelCluster by removing the `SecurityGroups` configuration from your cluster configuration file.