# Working with custom identity providers AWS Transfer Family offers several options for custom identity providers to authenticate and authorize users for secure file transfers. Here are the main approaches: + [Custom identity provider solution](custom-idp-toolkit.md)—This topic describes the Transfer Family custom identity provider solution, using a toolkit hosted in GitHub. **Note** For most use cases, this is the recommended option. Specifically, if you need to support more than 100 Active Directory groups, the custom identity provider solution offers a scalable alternative without group limitations. This solution is described in the blog post, [Simplify Active Directory authentication with a custom identity provider for AWS Transfer Family](https://aws.amazon.com/blogs/storage/simplify-active-directory-authentication-with-a-custom-identity-provider-for-aws-transfer-family/). + [Using Amazon API Gateway to integrate your identity provider](authentication-api-gateway.md)—This topic describes how to use an AWS Lambda function to back an Amazon API Gateway method. You can provide a RESTful interface with a single Amazon API Gateway method. Transfer Family calls this method to connect to your identity provider, which authenticates and authorizes your users for access to Amazon S3 or Amazon EFS. Use this option if you need a RESTful API to integrate your identity provider or if you want to use AWS WAF to leverage its capabilities for geo-blocking or rate-limiting requests. For details, see [Using Amazon API Gateway to integrate your identity provider](authentication-api-gateway.md). + [Dynamic permission management approaches](dynamic-permission-management.md)—This topic describes approaches for managing user permissions dynamically using session policies. To authenticate your users, you can use your existing identity provider with AWS Transfer Family. You integrate your identity provider using an AWS Lambda function, which authenticates and authorizes your users for access to Amazon S3 or Amazon Elastic File System (Amazon EFS). For details, see [Using AWS Lambda to integrate your identity provider](custom-lambda-idp.md). You can also access CloudWatch graphs for metrics such as number of files and bytes transferred in the AWS Transfer Family Management Console, giving you a single pane of glass to monitor file transfers using a centralized dashboard. + Transfer Family provides a blog post and a workshop that walk you through building a file transfer solution. This solution leverages AWS Transfer Family for managed SFTP/FTPS endpoints and Amazon Cognito and DynamoDB for user management. The blog post is available at [Using Amazon Cognito as an identity provider with AWS Transfer Family and Amazon S3](https://aws.amazon.com/blogs/storage/using-amazon-cognito-as-an-identity-provider-with-aws-transfer-family-and-amazon-s3/). You can view the details for the workshop [here](https://catalog.workshops.aws/transfer-family-sftp/en-US). **Note** For custom identity providers, the username must be a minimum of 3 and a maximum of 100 characters. You can use the following characters in the username: a–z, A-Z, 0–9, underscore '\$1', hyphen '-', period '.' and at sign '@'. The username can't start with a hyphen '-', period '.' or at sign '@'. When implementing a custom identity provider, consider the following best practices: + Deploy the solution in the same AWS account and region as your Transfer Family servers. + Implement the principle of least privilege when configuring IAM roles and policies. + Use features like IP allow-listing and standardized logging for enhanced security. + Test your custom identity provider thoroughly in a non-production environment before deployment. **Topics** + [ # Custom identity provider solution ](custom-idp-toolkit.md) + [ # Using AWS Lambda to integrate your identity provider ](custom-lambda-idp.md) + [ # Using Amazon API Gateway to integrate your identity provider ](authentication-api-gateway.md) + [ # Using multiple authentication methods ](custom-idp-mfa.md) + [ # IPv6 support for custom identity providers ](custom-idp-ipv6.md) # Custom identity provider solution The AWS Transfer Family custom identity provider solution is a modular custom identity provider solution that solves for many common authentication and authorization use cases that enterprises have when implementing the service. This solution provides a reusable foundation for implementing custom identity providers with granular per-user session configuration and separates authentication and authorization logic, offering a flexible and easy-to-maintain foundation for various use cases. With the AWS Transfer Family custom identity provider solution, you can address common enterprise authentication and authorization use cases. This modular solution offers: + A reusable foundation for implementing custom identity providers + Granular per-user session configuration + Separated authentication and authorization logic ## Implementation details for the custom identity toolkit The solution provides a flexible and maintainable base for various use cases. To get started, review the toolkit at [https://github.com/aws-samples/toolkit-for-aws-transfer-family](https://github.com/aws-samples/toolkit-for-aws-transfer-family), then follow the deployment instructions in the [Getting started](https://github.com/aws-samples/toolkit-for-aws-transfer-family/tree/main/solutions/custom-idp#getting-started) section. ![\[Architecture diagram for the custom identity provider toolkit available in GitHub.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/custom-idp-solution-high-level-architecture.png) **Note** If you have previously used custom identity provider templates and examples, consider adopting this solution instead. Moving forward, provider-specific modules will standardize on this solution. Ongoing maintenance and feature enhancements will be applied to this solution. This solution contains standard patterns for implementing a custom provider that accounts for details including logging and where to store the additional session metadata needed for AWS Transfer Family, such as the `HomeDirectoryDetails` parameter. This solution provides a reusable foundation for implementing custom identity providers with granular per-user session configuration, and decouples the identity provider authentication logic from the reusable logic that builds a configuration that is returned to Transfer Family to complete authentication and establish settings for the session. The code and supporting resources for this solution are available at [https://github.com/aws-samples/toolkit-for-aws-transfer-family](https://github.com/aws-samples/toolkit-for-aws-transfer-family). The toolkit contains the following features: + An [AWS Serverless Application Model](https://aws.amazon.com/serverless/sam) template that provisions the required resources. Optionally, deploy and configure Amazon API Gateway to incorporate AWS WAF, as described in the blog post [ Securing AWS Transfer Family with AWS Web Application Firewall and Amazon API Gateway](https://aws.amazon.com/blogs/storage/securing-aws-transfer-family-with-aws-web-application-firewall-and-amazon-api-gateway/). + An [Amazon DynamoDB](https://aws.amazon.com/dynamodb) schema to store configuration metadata about identity providers, including user session settings such as `HomeDirectoryDetails`, `Role`, and `Policy`. + A modular approach that enables you to add new identity providers to the solution in the future, as modules. + Attribute retrieval: Optionally retrieve IAM role and POSIX Profile (UID and GID) attributes from supported identity providers, including AD, LDAP, and Okta. + Support for multiple identity providers connected to a single Transfer Family server and multiple Transfer Family servers using the same deployment of the solution. + Built-in IP allow-list checking such as IP allow lists that can optionally be configured on a per-user or per-identity provider basis. + Detailed logging with configurable log-level and tracing support to aid in troubleshooting. Before you begin to deploy the custom identity provider solution, you need to have the following AWS resources. + An Amazon Virtual Private Cloud (VPC) with private subnets, with internet connectivity through either a NAT gateway or a DynamoDB gateway endpoint. + Appropriate IAM permissions to perform the following tasks: + Deploy the `custom-idp.yaml` CloudFormation template, + Create AWS CodePipeline projects + Create AWS CodeBuild projects + Create IAM roles and policies **Important** You must deploy the solution to the same AWS account and AWS Region that contains your target Transfer Family servers. ## Supported identity providers The following list contains details for identity providers that are supported for the custom identity provider solution. | Provider | Password flows | Public key flows | Multi-factor | Attribute retrieval | Details | | --- | --- | --- | --- | --- | --- | | Active Directory and LDAP | Yes | Yes | No | Yes | User verification can be performed as part of public key authentication flow. | | Argon2 (local hash) | Yes | No | No | No | Argon2 hashes are stored in the user record for 'local' password based authentication use cases. | | Amazon Cognito | Yes | No | Yes\$1 | No | Time-based One-Time Password (TOTP)-based multi-factor authentication only. \$1SMS-based MFA is not supported. | | Entra ID (formerly Azure AD) | Yes | No | No | No | | | Okta | Yes | Yes | Yes\$1 | Yes | TOTP-based MFA only. | | Public key | No | Yes | No | No | Public keys are stored in the user record in DynamoDB. | | Secrets Manager | Yes | Yes | No | No | | # Using AWS Lambda to integrate your identity provider This topic describes how to create an AWS Lambda function that connects to your custom identity provider. You can use any custom identity provider, such as Okta, Secrets Manager, OneLogin, or a custom data store that includes authorization and authentication logic. For most use cases, the recommended way to configure a custom identity provider is to use the [Custom identity provider solution](custom-idp-toolkit.md). **Note** Before you create a Transfer Family server that uses Lambda as the identity provider, you must create the function. For an example Lambda function, see [Example Lambda functions](#lambda-auth-examples). Or, you can deploy a CloudFormation stack that uses one of the [Lambda function templates](#lambda-idp-templates). Also, make sure your Lambda function uses a resource-based policy that trusts Transfer Family. For an example policy, see [Lambda resource-based policy](#lambda-resource-policy). 1. Open the [AWS Transfer Family console](https://console.aws.amazon.com/transfer/). 1. Choose **Create server** to open the **Create server** page. For **Choose an identity provider**, choose **Custom Identity Provider**, as shown in the following screenshot. ![\[The Choose an identity provider console section with Custom identity provider selected. Also has the default value selected, which is that users can authenticate using either their password or key.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/custom-lambda-console.png) **Note** The choice of authentication methods is only available if you enable SFTP as one of the protocols for your Transfer Family server. 1. Make sure the default value, **Use AWS Lambda to connect your identity provider**, is selected. 1. For **AWS Lambda function**, choose the name of your Lambda function. 1. Fill in the remaining boxes, and then choose **Create server**. For details on the remaining steps for creating a server, see [Configuring an SFTP, FTPS, or FTP server endpoint](tf-server-endpoint.md). ## Lambda resource-based policy You must have a policy that references the Transfer Family server and Lambda ARNs. For example, you could use the following policy with your Lambda function that connects to your identity provider. The policy is escaped JSON as a string. **** ``` "Policy": "{\"Version\":\"2012-10-17\", \"Id\":\"default\", \"Statement\":[ {\"Sid\":\"AllowTransferInvocation\", \"Effect\":\"Allow\", \"Principal\":{\"Service\":\"transfer.amazonaws.com\"}, \"Action\":\"lambda:InvokeFunction\", \"Resource\":\"arn:aws:lambda:region:123456789012:function:my-lambda-auth-function\", \"Condition\":{\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:transfer:region:123456789012:server/server-id\"}}} ]}" ``` **Note** In the example policy above, replace each *user input placeholder* with your own information. ## Event message structure The event message structure from SFTP server sent to the authorizer Lambda function for a custom IDP is as follows. ``` { "username": "value", "password": "value", "protocol": "SFTP", "serverId": "s-abcd123456", "sourceIp": "192.168.0.100" } ``` Where `username` and `password` are the values for the sign-in credentials that are sent to the server. For example, you enter the following command to connect: ``` sftp bobusa@server_hostname ``` You are then prompted to enter your password: ``` Enter password: mysecretpassword ``` You can check this from your Lambda function by printing the passed event from within the Lambda function. It should look similar to the following text block. ``` { "username": "bobusa", "password": "mysecretpassword", "protocol": "SFTP", "serverId": "s-abcd123456", "sourceIp": "192.168.0.100" } ``` The event structure is similar for FTP and FTPS: the only difference is those values are used for the `protocol` parameter, rather than SFTP. ## Lambda functions for authentication To implement different authentication strategies, edit the Lambda function. To help you meet your application's needs, you can deploy a CloudFormation stack. For more information about Lambda, see the [AWS Lambda Developer Guide](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) or [ Building Lambda functions with Node.js](https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html). **Topics** + [ ### Valid Lambda values ](#lambda-valid-values) + [ ### Example Lambda functions ](#lambda-auth-examples) + [ ### Testing your configuration ](#authentication-test-configuration) + [ ### Lambda function templates ](#lambda-idp-templates) ### Valid Lambda values The following table describes details for the values that Transfer Family accepts for Lambda functions that are used for custom identity providers. | Value | Description | Required | | --- | --- | --- | | `Role` | Specifies the Amazon Resource Name (ARN) of the IAM role that controls your users' access to your Amazon S3 bucket or Amazon EFS file system. The policies attached to this role determine the level of access that you want to provide your users when transferring files into and out of your Amazon S3 or Amazon EFS file system. The IAM role should also contain a trust relationship that allows the server to access your resources when servicing your users' transfer requests. For details on establishing a trust relationship, see [To establish a trust relationship](requirements-roles.md#establish-trust-transfer). | Required | | `PosixProfile` | The full POSIX identity, including user ID (`Uid`), group ID (`Gid`), and any secondary group IDs (`SecondaryGids`), that controls your users' access to your Amazon EFS file systems. The POSIX permissions that are set on files and directories in your file system determine the level of access your users get when transferring files into and out of your Amazon EFS file systems. | Required for Amazon EFS backing storage | | `PublicKeys` | A list of SSH public key values that are valid for this user. An empty list implies that this is not a valid login. Must not be returned during password authentication. | Optional | | `Policy` | A session policy for your user so that you can use the same IAM role across multiple users. This policy scopes down user access to portions of their Amazon S3 bucket. For more information about using session policies with custom identity providers, see the session policy examples in this topic. | Optional | | `HomeDirectoryType` | The type of landing directory (folder) that you want your users' home directory to be when they log in to the server. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transfer/latest/userguide/custom-lambda-idp.html) | Optional | | `HomeDirectoryDetails` | Logical directory mappings that specify which Amazon S3 or Amazon EFS paths and keys should be visible to your user and how you want to make them visible. You must specify the `Entry` and `Target` pair, where `Entry` shows how the path is made visible and `Target` is the actual Amazon S3 or Amazon EFS path. | Required if `HomeDirectoryType` has a value of `LOGICAL` | | `HomeDirectory` | The landing directory for a user when they log in to the server using the client. The format depends on your storage backend: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/transfer/latest/userguide/custom-lambda-idp.html) The bucket name or Amazon EFS file system ID must be included in the path. Omitting this information will result in "File not found" errors during file transfers. | Optional | **Note** `HomeDirectoryDetails` is a string representation of a JSON map. This is in contrast to `PosixProfile`, which is an actual JSON map object, and `PublicKeys` which is a JSON array of strings. See the code examples for the language-specific details. **HomeDirectory Format Requirements** When using the `HomeDirectory` parameter, ensure you include the complete path format: **For Amazon S3 storage:** Always include the bucket name in the format `/bucket-name/path` **For Amazon EFS storage:** Always include the file system ID in the format `/fs-12345/path` A common cause of "File not found" errors is omitting the bucket name or EFS file system ID from the `HomeDirectory` path. Setting `HomeDirectory` to just `/` without the storage identifier will cause authentication to succeed but file operations to fail. ### Example Lambda functions This section presents some example Lambda functions, in both NodeJS and Python. **Note** In these examples, the user, role, POSIX profile, password, and home directory details are all examples, and must be replaced with your actual values. ------ #### [ Logical home directory, NodeJS ] The following NodeJS example function provides the details for a user that has a [logical home directory](https://docs.aws.amazon.com/transfer/latest/userguide/logical-dir-mappings.html). ``` // GetUserConfig Lambda exports.handler = (event, context, callback) => { console.log("Username:", event.username, "ServerId: ", event.serverId); var response; // Check if the username presented for authentication is correct. This doesn't check the value of the server ID, only that it is provided. if (event.serverId !== "" && event.username == 'example-user') { var homeDirectoryDetails = [ { Entry: "/", Target: "/fs-faa1a123" } ]; response = { Role: 'arn:aws:iam::123456789012:role/transfer-access-role', // The user is authenticated if and only if the Role field is not blank PosixProfile: {"Gid": 65534, "Uid": 65534}, // Required for EFS access, but not needed for S3 HomeDirectoryDetails: JSON.stringify(homeDirectoryDetails), HomeDirectoryType: "LOGICAL", }; // Check if password is provided if (!event.password) { // If no password provided, return the user's SSH public key response['PublicKeys'] = [ "ssh-rsa abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789" ]; // Check if password is correct } else if (event.password !== 'Password1234') { // Return HTTP status 200 but with no role in the response to indicate authentication failure response = {}; } } else { // Return HTTP status 200 but with no role in the response to indicate authentication failure response = {}; } callback(null, response); }; ``` ------ #### [ Path-based home directory, NodeJS ] The following NodeJS example function provides the details for a user that has a path-based home directory. ``` // GetUserConfig Lambda exports.handler = (event, context, callback) => { console.log("Username:", event.username, "ServerId: ", event.serverId); var response; // Check if the username presented for authentication is correct. This doesn't check the value of the server ID, only that it is provided. // There is also event.protocol (one of "FTP", "FTPS", "SFTP") and event.sourceIp (e.g., "127.0.0.1") to further restrict logins. if (event.serverId !== "" && event.username == 'example-user') { response = { Role: 'arn:aws:iam::123456789012:role/transfer-access-role', // The user is authenticated if and only if the Role field is not blank Policy: '', // Optional, JSON stringified blob to further restrict this user's permissions // HomeDirectory format depends on your storage backend: // For S3: '/bucket-name/user-home-directory' (e.g., '/my-transfer-bucket/users/john') // For EFS: '/fs-12345/user-home-directory' (e.g., '/fs-faa1a123/users/john') HomeDirectory: '/my-transfer-bucket/users/example-user' // S3 example - replace with your bucket name // HomeDirectory: '/fs-faa1a123/users/example-user' // EFS example - uncomment for EFS }; // Check if password is provided if (!event.password) { // If no password provided, return the user's SSH public key response['PublicKeys'] = [ "ssh-rsa abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789" ]; // Check if password is correct } else if (event.password !== 'Password1234') { // Return HTTP status 200 but with no role in the response to indicate authentication failure response = {}; } } else { // Return HTTP status 200 but with no role in the response to indicate authentication failure response = {}; } callback(null, response); }; ``` ------ #### [ Logical home directory, Python ] The following Python example function provides the details for a user that has a [logical home directory](https://docs.aws.amazon.com/transfer/latest/userguide/logical-dir-mappings.html). ``` # GetUserConfig Python Lambda with LOGICAL HomeDirectoryDetails import json def lambda_handler(event, context): print("Username: {}, ServerId: {}".format(event['username'], event['serverId'])) response = {} # Check if the username presented for authentication is correct. This doesn't check the value of the server ID, only that it is provided. if event['serverId'] != '' and event['username'] == 'example-user': homeDirectoryDetails = [ { 'Entry': '/', 'Target': '/fs-faa1a123' } ] response = { 'Role': 'arn:aws:iam::123456789012:role/transfer-access-role', # The user will be authenticated if and only if the Role field is not blank 'PosixProfile': {"Gid": 65534, "Uid": 65534}, # Required for EFS access, but not needed for S3 'HomeDirectoryDetails': json.dumps(homeDirectoryDetails), 'HomeDirectoryType': "LOGICAL" } # Check if password is provided if event.get('password', '') == '': # If no password provided, return the user's SSH public key response['PublicKeys'] = [ "ssh-rsa abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789" ] # Check if password is correct elif event['password'] != 'Password1234': # Return HTTP status 200 but with no role in the response to indicate authentication failure response = {} else: # Return HTTP status 200 but with no role in the response to indicate authentication failure response = {} return response ``` ------ #### [ Path-based home directory, Python ] The following Python example function provides the details for a user that has a path-based home directory. ``` # GetUserConfig Python Lambda with PATH HomeDirectory def lambda_handler(event, context): print("Username: {}, ServerId: {}".format(event['username'], event['serverId'])) response = {} # Check if the username presented for authentication is correct. This doesn't check the value of the server ID, only that it is provided. # There is also event.protocol (one of "FTP", "FTPS", "SFTP") and event.sourceIp (e.g., "127.0.0.1") to further restrict logins. if event['serverId'] != '' and event['username'] == 'example-user': response = { 'Role': 'arn:aws:iam::123456789012:role/transfer-access-role', # The user will be authenticated if and only if the Role field is not blank 'Policy': '', # Optional, JSON stringified blob to further restrict this user's permissions # HomeDirectory format depends on your storage backend: # For S3: '/bucket-name/user-home-directory' (e.g., '/my-transfer-bucket/users/john') # For EFS: '/fs-12345/user-home-directory' (e.g., '/fs-faa1a123/users/john') 'HomeDirectory': '/my-transfer-bucket/users/example-user', # S3 example - replace with your bucket name # 'HomeDirectory': '/fs-faa1a123/users/example-user', # EFS example - uncomment for EFS 'HomeDirectoryType': "PATH" # Not strictly required, defaults to PATH } # Check if password is provided if event.get('password', '') == '': # If no password provided, return the user's SSH public key response['PublicKeys'] = [ "ssh-rsa abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789" ] # Check if password is correct elif event['password'] != 'Password1234': # Return HTTP status 200 but with no role in the response to indicate authentication failure response = {} else: # Return HTTP status 200 but with no role in the response to indicate authentication failure response = {} return response ``` ------ ### Testing your configuration After you create your custom identity provider, you should test your configuration. ------ #### [ Console ] **To test your configuration by using the AWS Transfer Family console** 1. Open the [AWS Transfer Family console](https://console.aws.amazon.com/transfer/). 1. On the **Servers** page, choose your new server, choose **Actions**, and then choose **Test**. 1. Enter the text for **Username** and **Password** that you set when you deployed the CloudFormation stack. If you kept the default options, the username is `myuser` and the password is `MySuperSecretPassword`. 1. Choose the **Server protocol** and enter the IP address for **Source IP**, if you set them when you deployed the CloudFormation stack. ------ #### [ CLI ] **To test your configuration by using the AWS CLI** 1. Run the [test-identity-provider](https://docs.aws.amazon.com/cli/latest/reference/transfer/test-identity-provider.html) command. Replace each `user input placeholder` with your own information, as described in the subsequent steps. ``` aws transfer test-identity-provider --server-id s-1234abcd5678efgh --user-name myuser --user-password MySuperSecretPassword --server-protocol FTP --source-ip 127.0.0.1 ``` 1. Enter the server ID. 1. Enter the username and password that you set when you deployed the CloudFormation stack. If you kept the default options, the username is `myuser` and the password is `MySuperSecretPassword`. 1. Enter the server protocol and source IP address, if you set them when you deployed the CloudFormation stack. ------ If user authentication succeeds, the test returns a `StatusCode: 200` HTTP response, an empty string `Message: ""` (which would contain a reason for failure otherwise), and a `Response` field. **Note** In the response example below, the `Response` field is a JSON object that has been "stringified" (converted into a flat JSON string that can be used inside a program), and contains the details of the user's roles and permissions. ``` { "Response":"{\"Policy\":\"{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Sid\\\":\\\"ReadAndListAllBuckets\\\",\\\"Effect\\\":\\\"Allow\\\",\\\"Action\\\":[\\\"s3:ListAllMybuckets\\\",\\\"s3:GetBucketLocation\\\",\\\"s3:ListBucket\\\",\\\"s3:GetObjectVersion\\\",\\\"s3:GetObjectVersion\\\"],\\\"Resource\\\":\\\"*\\\"}]}\",\"Role\":\"arn:aws:iam::000000000000:role/MyUserS3AccessRole\",\"HomeDirectory\":\"/\"}", "StatusCode": 200, "Message": "" } ``` ### Lambda function templates You can deploy an CloudFormation stack that uses a Lambda function for authentication. We provide several templates that authenticate and authorize your users using sign-in credentials. You can modify these templates or AWS Lambda code to further customize user access. **Note** You can create a FIPS-enabled AWS Transfer Family server through CloudFormation by specifying a FIPS-enabled security policy in your template. Available security policies are described in [Security policies for AWS Transfer Family servers](security-policies.md) **To create an CloudFormation stack to use for authentication** 1. Open the CloudFormation console at [https://console.aws.amazon.com/cloudformation](https://console.aws.amazon.com/cloudformation/). 1. Follow the instructions for deploying an CloudFormation stack from an existing template in [Selecting a stack template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-console-create-stack-template.html) in the *AWS CloudFormation User Guide*. 1. Use one of the following templates to create a Lambda function to use for authentication in Transfer Family. + [Classic (Amazon Cognito) stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-basic-lambda-cognito-s3.template.yml) A basic template for creating a AWS Lambda for use as a custom identity provider in AWS Transfer Family. It authenticates against Amazon Cognito for password-based authentication and public keys are returned from an Amazon S3 bucket if public key based authentication is used. After deployment, you can modify the Lambda function code to do something different. + [AWS Secrets Manager stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-secrets-manager-lambda.template.yml) A basic template that uses AWS Lambda with an AWS Transfer Family server to integrate Secrets Manager as an identity provider. It authenticates against an entry in AWS Secrets Manager of the format `aws/transfer/server-id/username`. Additionally, the secret must hold the key-value pairs for all user properties returned to Transfer Family. After deployment, you can modify the Lambda function code to do something different. + [Okta stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-okta-lambda.template.yml): A basic template that uses AWS Lambda with an AWS Transfer Family server to integrate Okta as a custom identity provider. + [Okta-mfa stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-okta-mfa-lambda.template.yml): A basic template that uses AWS Lambda with an AWS Transfer Family server to integrate Okta, with Multi Factor Authentication, as a custom identity provider. + [ Azure Active Directory template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-basic-lambda-azure-ad.template.yml): details for this stack are described in the blog post [ Authenticating to AWS Transfer Family with Azure Active Directory and AWS Lambda](https://aws.amazon.com/blogs/storage/authenticating-to-aws-transfer-family-with-azure-active-directory-and-aws-lambda/). After the stack has been deployed, you can view details about it on the **Outputs** tab in the CloudFormation console. Deploying one of these stacks is the easiest way to integrate a custom identity provider into the Transfer Family workflow. # Using Amazon API Gateway to integrate your identity provider This topic describes how to use an AWS Lambda function to back an API Gateway method. Use this option if you need a RESTful API to integrate your identity provider or if you want to use AWS WAF to leverage its capabilities for geo-blocking or rate-limiting requests. For most use cases, the recommended way to configure a custom identity provider is to use the [Custom identity provider solution](custom-idp-toolkit.md). **Limitations if using an API Gateway to integrate your identity provider** + This configuration does not support custom domains. + This configuration does not support a private API Gateway URL. If you need either of these, you can use Lambda as an identity provider, without API Gateway. For details, see [Using AWS Lambda to integrate your identity provider](custom-lambda-idp.md). ## Authenticating using an API Gateway method You can create an API Gateway method for use as an identity provider for Transfer Family. This approach provides a highly secure way for you to create and provide APIs. With API Gateway, you can create an HTTPS endpoint so that all incoming API operations are transmitted with greater security. For more details about the API Gateway service, see the [API Gateway Developer Guide](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html). API Gateway offers an authorization method named `AWS_IAM`, which gives you the same authentication based on AWS Identity and Access Management (IAM) that AWS uses internally. If you enable authentication with `AWS_IAM`, only callers with explicit permissions to call an API can reach that API's API Gateway method. To use your API Gateway method as a custom identity provider for Transfer Family, enable IAM for your API Gateway method. As part of this process, you provide an IAM role with permissions for Transfer Family to use your gateway. **Note** To improve security, you can configure a web application firewall. AWS WAF is a web application firewall that lets you monitor the HTTP and HTTPS requests that are forwarded to an Amazon API Gateway. For details, see [Add a web application firewall](web-application-firewall.md). **Do not enable API Gateway caching** Do not enable caching for your API Gateway method when using it as a custom identity provider for Transfer Family. Caching is inappropriate and invalid for authentication requests because: Each authentication request is unique and requires a live response, not a cached response Caching provides no benefits since Transfer Family never sends duplicate or repeated requests to the API Gateway Enabling caching will cause the API Gateway to respond with mismatched data, resulting in invalid responses to authentication requests **To use your API Gateway method for custom authentication with Transfer Family** 1. Create an CloudFormation stack. To do this: **Note** The stack templates have been updated to use BASE64-encoded passwords: for details, see [Improvements to the CloudFormation templates](#base64-templates). 1. Open the CloudFormation console at [https://console.aws.amazon.com/cloudformation](https://console.aws.amazon.com/cloudformation/). 1. Follow the instructions for deploying an CloudFormation stack from an existing template in [Selecting a stack template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-console-create-stack-template.html) in the *AWS CloudFormation User Guide*. 1. Use one of the following basic templates to create an AWS Lambda-backed API Gateway method for use as a custom identity provider in Transfer Family. + [Basic stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-basic-apig.template.yml) By default, your API Gateway method is used as a custom identity provider to authenticate a single user in a single server using a hard-coded SSH (Secure Shell) key or password. After deployment, you can modify the Lambda function code to do something different. + [AWS Secrets Manager stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-secrets-manager-apig.template.yml) By default, your API Gateway method authenticates against an entry in Secrets Manager of the format `aws/transfer/server-id/username`. Additionally, the secret must hold the key-value pairs for all user properties returned to Transfer Family. After deployment, you can modify the Lambda function code to do something different. For more information, see the blog post[Enable password authentication for AWS Transfer Family using AWS Secrets Manager](https://aws.amazon.com/blogs/storage/enable-password-authentication-for-aws-transfer-family-using-aws-secrets-manager-updated/). + [Okta stack template](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-okta-apig.template.yml) Your API Gateway method integrates with Okta as a custom identity provider in Transfer Family. For more information, see the blog post [Using Okta as an identity provider with AWS Transfer Family](https://aws.amazon.com/blogs/storage/using-okta-as-an-identity-provider-with-aws-transfer-for-sftp/). Deploying one of these stacks is the easiest way to integrate a custom identity provider into the Transfer Family workflow. Each stack uses the Lambda function to support your API method based on API Gateway. You can then use your API method as a custom identity provider in Transfer Family. By default, the Lambda function authenticates a single user called `myuser` with a password of `MySuperSecretPassword`. After deployment, you can edit these credentials or update the Lambda function code to do something different. **Important** We recommend that you edit the default user and password credentials. After the stack has been deployed, you can view details about it on the **Outputs** tab in the CloudFormation console. These details include the stack's Amazon Resource Name (ARN), the ARN of the IAM role that the stack created, and the URL for your new gateway. **Note** If you are using the custom identity provider option to enable password–based authentication for your users, and you enable the request and response logging provided by API Gateway, API Gateway logs your users' passwords to your Amazon CloudWatch Logs. We don't recommend using this log in your production environment. For more information, see [Set up CloudWatch API logging in API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html) in the *API Gateway Developer Guide*. 1. Check the API Gateway method configuration for your server. To do this: 1. Open the API Gateway console at [https://console.aws.amazon.com/apigateway/](https://console.aws.amazon.com/apigateway/). 1. Choose the **Transfer Custom Identity Provider basic template API** that the CloudFormation template generated. You might need to select your region to see your gateways. 1. In the **Resources** pane, choose **GET**. The following screenshot shows the correct method configuration. ![\[API configuration details, showing the method configuration parameters for the Request Paths and the for the URL Query String.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/apig-config-method-fields.png) At this point, your API gateway is ready to be deployed. 1. For **Actions**, choose **Deploy API**. For **Deployment stage**, choose **prod**, and then choose **Deploy**. After the API Gateway method is successfully deployed, view its performance in **Stages** > **Stage details**, as shown in the following screenshot. **Note** Copy the **Invoke URL** address that appears at the top of the screen. You might need it for the next step. ![\[Stage details with the Invoke URL highlighted.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/apig-config-method-invoke.png) 1. Open the AWS Transfer Family console at [https://console.aws.amazon.com/transfer/](https://console.aws.amazon.com/transfer/). 1. A Transfer Family should have been created for you, when you created the stack. If not, configure your server using these steps. 1. Choose **Create server** to open the **Create server** page. For **Choose an identity provider**, choose **Custom**, then select **Use Amazon API Gateway to connect to your identity provider**, as shown in the following screenshot. ![\[The identity provider screen with Custom Identity Provider selected, and with the API Gateway chosen for connecting to your identity provider.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/create-server-choose-idp-custom.png) 1. In the **Provide an Amazon API Gateway URL** text box, paste the **Invoke URL** address of the API Gateway endpoint that you created in step 3 of this procedure. 1. For **Role**, choose the IAM role that was created by the CloudFormation template. This role allows Transfer Family to invoke your API gateway method. The invocation role contains the CloudFormation stack name that you selected for the stack that you created in step 1. It has the following format: `CloudFormation-stack-name-TransferIdentityProviderRole-ABC123DEF456GHI`. 1. Fill in the remaining boxes, and then choose **Create server**. For details on the remaining steps for creating a server, see [Configuring an SFTP, FTPS, or FTP server endpoint](tf-server-endpoint.md). ## Implementing your API Gateway method To create a custom identity provider for Transfer Family, your API Gateway method must implement a single method that has a resource path of `/servers/serverId/users/username/config`. The `serverId` and `username` values come from the RESTful resource path. Also, add `sourceIp` and `protocol` as **URL Query String Parameters** in the **Method Request**, as shown in the following image. ![\[The Resources screen of the API Gateway showing the GET method details.\]](http://docs.aws.amazon.com/transfer/latest/userguide/images/apig-config-method-request.png) **Note** The username must be a minimum of 3 and a maximum of 100 characters. You can use the following characters in the username: a–z, A-Z, 0–9, underscore '\$1', hyphen '-', period '.' and at sign '@'. The username can't start with a hyphen '-', period '.' or at sign '@'. If Transfer Family attempts password authentication for your user, the service supplies a `Password:` header field. In the absence of a `Password:` header, Transfer Family attempts public key authentication to authenticate your user. When you are using an identity provider to authenticate and authorize end users, in addition to validating their credentials, you can allow or deny access requests based on the IP addresses of the clients used by your end users. You can use this feature to ensure that data stored in your S3 buckets or your Amazon EFS file system can be accessed over the supported protocols only from IP addresses that you have specified as trusted. To enable this feature, you must include `sourceIp` in the Query string. If you have multiple protocols enabled for your server and want to provide access using the same username over multiple protocols, you can do so as long as the credentials specific to each protocol have been set up in your identity provider. To enable this feature, you must include the `protocol` value in the RESTful resource path. Your API Gateway method should always return HTTP status code `200`. Any other HTTP status code means that there was an error accessing the API. **Amazon S3 example response** The example response body is a JSON document of the following form for Amazon S3. ``` { "Role": "IAM role with configured S3 permissions", "PublicKeys": [ "ssh-rsa public-key1", "ssh-rsa public-key2" ], "Policy": "STS Assume role session policy", "HomeDirectory": "/amzn-s3-demo-bucket/path/to/home/directory" } ``` **Note** The policy is escaped JSON as a string. For example: **** ``` "Policy": "{ \"Version\": \"2012-10-17\", \"Statement\": [ {\"Condition\": {\"StringLike\": {\"s3:prefix\": [\"user/*\", \"user/\"]}}, \"Resource\": \"arn:aws:s3:::amzn-s3-demo-bucket\", \"Action\": \"s3:ListBucket\", \"Effect\": \"Allow\", \"Sid\": \"ListHomeDir\"}, {\"Resource\": \"arn:aws:s3:::*\", \"Action\": [\"s3:PutObject\", \"s3:GetObject\", \"s3:DeleteObjectVersion\", \"s3:DeleteObject\", \"s3:GetObjectVersion\", \"s3:GetObjectACL\", \"s3:PutObjectACL\"], \"Effect\": \"Allow\", \"Sid\": \"HomeDirObjectAccess\"}] }" ``` The following example response shows that a user has a logical home directory type. ``` { "Role": "arn:aws:iam::123456789012:role/transfer-access-role-s3", "HomeDirectoryType":"LOGICAL", "HomeDirectoryDetails":"[{\"Entry\":\"/\",\"Target\":\"/amzn-s3-demo-bucket1\"}]", "PublicKeys":[""] } ``` **Amazon EFS example response** The example response body is a JSON document of the following form for Amazon EFS. ``` { "Role": "IAM role with configured EFS permissions", "PublicKeys": [ "ssh-rsa public-key1", "ssh-rsa public-key2" ], "PosixProfile": { "Uid": "POSIX user ID", "Gid": "POSIX group ID", "SecondaryGids": [Optional list of secondary Group IDs], }, "HomeDirectory": "/fs-id/path/to/home/directory" } ``` The `Role` field shows that successful authentication occurred. When doing password authentication (when you supply a `Password:` header), you don't need to provide SSH public keys. If a user can't be authenticated, for example, if the password is incorrect, your method should return a response without `Role` set. An example of such a response is an empty JSON object. The following example response shows a user that has a logical home directory type. ``` { "Role": "arn:aws:iam::123456789012:role/transfer-access-role-efs", "HomeDirectoryType": "LOGICAL", "HomeDirectoryDetails":"[{\"Entry\":\"/\",\"Target\":\"/faa1a123\"}]", "PublicKeys":[""], "PosixProfile":{"Uid":65534,"Gid":65534} } ``` You can include user policies in the Lambda function in JSON format. For more information about configuring user policies in Transfer Family, see [Managing access controls](users-policies.md). ## Default Lambda function To implement different authentication strategies, edit the Lambda function that your gateway uses. To help you meet your application's needs, you can use the following example Lambda functions in Node.js. For more information about Lambda, see the [AWS Lambda Developer Guide](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) or [ Building Lambda functions with Node.js](https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html). The following example Lambda function takes your username, password (if you're performing password authentication), server ID, protocol, and client IP address. You can use a combination of these inputs to look up your identity provider and determine if the login should be accepted. **Note** If you have multiple protocols enabled for your server and want to provide access using the same username over multiple protocols, you can do so as long as the credentials specific to the protocol have been set up in your identity provider. For File Transfer Protocol (FTP), we recommend maintaining separate credentials from Secure Shell (SSH) File Transfer Protocol (SFTP) and File Transfer Protocol over SSL (FTPS). We recommend maintaining separate credentials for FTP because, unlike SFTP and FTPS, FTP transmits credentials in clear text. By isolating FTP credentials from SFTP or FTPS, if FTP credentials are shared or exposed, your workloads using SFTP or FTPS remain secure. This example function returns the role and logical home directory details, along with the public keys (if it performs public key authentication). When you create service-managed users, you set their home directory, either logical or physical. Similarly, we need the Lambda function results to convey the desired user physical or logical directory structure. The parameters you set depend on the value for the [https://docs.aws.amazon.com//transfer/latest/APIReference/API_CreateUser.html#TransferFamily-CreateUser-request-HomeDirectoryType](https://docs.aws.amazon.com//transfer/latest/APIReference/API_CreateUser.html#TransferFamily-CreateUser-request-HomeDirectoryType) field. + `HomeDirectoryType` set to `PATH` – the `HomeDirectory` field must then be an absolute Amazon S3 bucket prefix or Amazon EFS absolute path that is visible to your users. + `HomeDirectoryType` set to `LOGICAL` – Do *not* set a `HomeDirectory` field. Instead, we set a `HomeDirectoryDetails` field that provides the desired Entry/Target mappings, similar to the described values in the [https://docs.aws.amazon.com//transfer/latest/APIReference/API_CreateUser.html#TransferFamily-CreateUser-request-HomeDirectoryMappings](https://docs.aws.amazon.com//transfer/latest/APIReference/API_CreateUser.html#TransferFamily-CreateUser-request-HomeDirectoryMappings) parameter for service-managed users. The example functions are listed in [Example Lambda functions](custom-lambda-idp.md#lambda-auth-examples). ## Lambda function for use with AWS Secrets Manager To use AWS Secrets Manager as your identity provider, you can work with the Lambda function in the sample CloudFormation template. The Lambda function queries the Secrets Manager service with your credentials and, if successful, returns a designated secret. For more information about Secrets Manager, see the [AWS Secrets Manager User Guide](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html). To download a sample CloudFormation template that uses this Lambda function, go to the [Amazon S3 bucket provided by AWS Transfer Family](https://s3.amazonaws.com/aws-transfer-resources/custom-idp-templates/aws-transfer-custom-idp-secrets-manager-apig.template.yml). ## Improvements to the CloudFormation templates Improvements to the API Gateway interface have been made to the published CloudFormation templates. The templates now use BASE64-encoded passwords with the API Gateway. Your existing deployments continue to work without this enhancement, but don't allow for passwords with characters outside the basic US-ASCII character set. The changes in the template that enable this capability are as follows: + The `GetUserConfigRequest AWS::ApiGateway::Method` resource has to have this `RequestTemplates` code (the line in italics is the updated line) ``` RequestTemplates: application/json: | { "username": "$util.urlDecode($input.params('username'))", "password": "$util.escapeJavaScript($util.base64Decode($input.params('PasswordBase64'))).replaceAll("\\'","'")", "protocol": "$input.params('protocol')", "serverId": "$input.params('serverId')", "sourceIp": "$input.params('sourceIp')" } ``` + The `RequestParameters` for the `GetUserConfig` resource must change to use the `PasswordBase64` header (the line in italics is the updated line): ``` RequestParameters: method.request.header.PasswordBase64: false method.request.querystring.protocol: false method.request.querystring.sourceIp: false ``` **To check if the template for your stack is the latest** 1. Open the CloudFormation console at [https://console.aws.amazon.com/cloudformation](https://console.aws.amazon.com/cloudformation/). 1. From the list of stacks, choose your stack. 1. From the details panel, choose the **Template** tab. 1. Look for the following: + Search for `RequestTemplates`, and make sure you have this line: ``` "password": "$util.escapeJavaScript($util.base64Decode($input.params('PasswordBase64'))).replaceAll("\\'","'")", ``` + Search for `RequestParameters`, and make sure you have this line: ``` method.request.header.PasswordBase64: false ``` If you don't see the updated lines, edit your stack. For details on how to update your CloudFormation stack, see [Modifying a stack template](https://docs.aws.amazon.com//AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-get-template.html) in the *AWS CloudFormation; User Guide*. # Using multiple authentication methods The Transfer Family server controls the AND logic when you use multiple authentication methods. Transfer Family treats this as two separate requests to your custom identity provider: however, their effect is combined. Both requests must return successfully with the correct response to allow the authentication to complete. Transfer Family requires the two responses to be complete, meaning they contain all of the required elements (role, home directory, policy and the POSIX profile if you're using Amazon EFS for storage). Transfer Family also requires that the password response must not include public keys. The public key request must have a separate response from the identity provider. That behavior is unchanged when using **Password OR Key** or **Password AND Key**. The SSH/SFTP protocol challenges the software client first with a public key authentication, then requests a password authentication. This operation mandates both are successful before the user is allowed to complete the authentication. For custom identity provider options, you can specify any of the following options for how to authenticate. + **Password OR Key** – users can authenticate with either their password or their key. This is the default value. + **Password ONLY** – users must provide their password to connect. + **Key ONLY** – users must provide their private key to connect. + **Password AND Key** – users must provide both their private key and their password to connect. The server checks the key first, and then if the key is valid, the system prompts for a password. If the private key provided does not match the public key that is stored, authentication fails. # IPv6 support for custom identity providers AWS Transfer Family custom identity providers fully support IPv6 connections. When implementing a custom identity provider, your Lambda function can receive and process authentication requests from both IPv4 and IPv6 clients without any additional configuration. The Lambda function receives the client's IP address in the `sourceIp` field of the request, which can be either an IPv4 address (for example, `203.0.113.42`) or an IPv6 address (for example, `2001:db8:85a3:8d3:1319:8a2e:370:7348`). Your custom identity provider implementation should handle both address formats appropriately. **Important** If your custom identity provider performs IP-based validation or logging, ensure your implementation properly handles IPv6 address formats. IPv6 addresses are longer than IPv4 addresses and use a different notation format. **Note** When handling IPv6 addresses in your custom identity provider, ensure you're using proper IPv6 address parsing functions rather than simple string comparisons. IPv6 addresses can be represented in various canonical formats (for example `fd00:b600::ec2` or `fd00:b600:0:0:0:0:0:ec2`). Use appropriate IPv6 address libraries or functions in your implementation language to correctly validate and compare IPv6 addresses. **Example Handling both IPv4 and IPv6 addresses in a custom identity provider** ``` def lambda_handler(event, context): # Extract the source IP address from the request source_ip = event.get('sourceIp', '') # Log the client IP address (works for both IPv4 and IPv6) print(f"Authentication request from: {source_ip}") # Example of IP-based validation that works with both IPv4 and IPv6 if is_ip_allowed(source_ip): # Continue with authentication # ... else: # Reject the authentication request return { "Role": "", "HomeDirectory": "", "Status": "DENIED" } ``` For more information about implementing custom identity providers, see [Using AWS Lambda to integrate your identity provider](custom-lambda-idp.md).