Using logical directories to simplify your Transfer Family directory structures

To simplify your AWS Transfer Family server directory structure, you can use logical directories. With logical directories, you can construct a virtual directory structure that uses user-friendly names that your users navigate when they connect to your Amazon S3 bucket or Amazon EFS file system. When you use logical directories, you can avoid disclosing absolute directory paths, Amazon S3 bucket names, and EFS file system names to your end users.

You can use logical directories to set the user’s root directory to a desired location within your storage hierarchy, by performing what is known as a chroot operation. In this mode, users are not able to navigate to a directory outside of the home or root directory that you've configured for them.

For example, although an Amazon S3 user has been scoped down to access only /amzn-s3-demo-bucket/home/${transfer:UserName}, some clients allow users to traverse up a folder to /amzn-s3-demo-bucket/home. In this situation, the user lands back on their intended home directory only after logging out of and back in to the Transfer Family server again. Performing a chroot operation can prevent this situation from occurring.

You can create your own directory structure across buckets and prefixes. This feature is useful if you have a workflow that is expecting a specific directory structure that you can't replicate through bucket prefixes. You can also link to multiple non-contiguous locations within Amazon S3, similar to creating a symbolic link in a Linux file system where your directory path references a different location in the file system.

Rules for using logical directories

Before you build your logical directory mappings, you should understand the following rules:

You can specify a HomeDirectory value when using the LOGICAL HomeDirectoryType. This enhancement applies to Service Managed users, Active Directory access, and Custom Identity Provider implementations where the HomeDirectoryDetails are provided in the response.

Implementing logical directories and chroot

To use logical directories and chroot features, you must do the following:

Turn on logical directories for each user. Do this by setting the HomeDirectoryType parameter to LOGICAL when you create or update your user.

"HomeDirectoryType": "LOGICAL"

chroot

For chroot, create a directory structure that consists of a single Entry and Target pairing for each user. The root folder is the Entry point, and the Target is the location in your bucket or file system to map to.

You can use an absolute path as in the previous example, or you can use a dynamic substitution for the username with ${transfer:UserName}, as in the following example.

[{"Entry": "/", "Target":
"/amzn-s3-demo-bucket/${transfer:UserName}"}]

In the preceding example, the user is locked to their root directory and cannot traverse up higher in the hierarchy.

Virtual directory structure

For a virtual directory structure, you can create multiple Entry Target pairings, with targets anywhere in your S3 buckets or EFS file systems, including across multiple buckets or file systems, as long as the user’s IAM role mapping has permissions to access them.

In the following virtual structure example, when the user logs into AWS SFTP, they are in the root directory with sub-directories of /pics, /doc, /reporting, and /anotherpath/subpath/financials.

[
{"Entry": "/pics", "Target": "/amzn-s3-demo-bucket1/pics"}, 
{"Entry": "/doc", "Target": "/amzn-s3-demo-bucket1/anotherpath/docs"},
{"Entry": "/reporting", "Target": "/amzn-s3-demo-bucket2/Q1"},
{"Entry": "/anotherpath/subpath/financials", "Target": "/amzn-s3-demo-bucket2/financials"}]

{
   "Entry": "/pics", 
   "Target": "/amzn-s3-demo-bucket/pics"
}, 
{
   "Entry": "/doc", 
   "Target": "/amzn-s3-demo-bucket/mydocs"
}, 
{
   "Entry": "/temp", 
   "Target": "/amzn-s3-demo-bucket"
}

sftp> pwd
Remote working directory: /
sftp> put file
Uploading file to /file
remote open("/file"): No such file or directory

For more information about configuring logical directories and chroot for your users, including an AWS CloudFormation template that you can download and use, see Simplify your AWS SFTP Structure with chroot and logical directories in the AWS Storage Blog.

Configure logical directories examples

In this example, we create a user and assign two logical directories. The following command creates a new user (for an existing Transfer Family server) with logical directories pics and doc.

aws transfer create-user --user-name marymajor --server-id s-11112222333344445 \
   --role arn:aws:iam::1234abcd5678:role/marymajor-role --home-directory-type LOGICAL \
   --home-directory-mappings "[{\"Entry\":\"/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, {\"Entry\":\"/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]" \
   --ssh-public-key-body file://~/.ssh/id_rsa.pub

If marymajor is an existing user and her home directory type is PATH, you can change it to LOGICAL with a similar command as the previous one.

aws transfer update-user --user-name marymajor \
 --server-id s-11112222333344445 --role arn:aws:iam::1234abcd5678:role/marymajor-role \
 --home-directory-type LOGICAL --home-directory-mappings "[{\"Entry\":\"/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, \
 {\"Entry\":\"/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]"

Note the following:

This example provides an alternate configuration for a logical home path.

aws transfer create-user --user-name marymajor --server-id s-11112222333344445 \
  --role arn:aws:iam::1234abcd5678:role/marymajor-role \
  --home-directory-type LOGICAL \
  --home-directory /home/marymajor \
  --home-directory-mappings "[{\"Entry\":\"/home/marymajor/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, {\"Entry\":\"/home/marymajor/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]" \
 --ssh-public-key-body file://~/.ssh/id_rsa.pub

Note the following:

Configure logical directories for Amazon EFS

If your Transfer Family server uses Amazon EFS, the home directory for the user must be created with read and write access before the user can work in their logical home directory. The user cannot create this directory themselves, as they would lack permissions for mkdir on their logical home directory.

If the user's home directory does not exist, and they run an ls command, the system responds as follows:

sftp> ls
remote readdir ("/"): No such file or directory

A user with administrative access to the parent directory needs to create the user's logical home directory.

Custom AWS Lambda response

You can use logical directories with a Lambda function that connects to your custom identity provider. To do so, in your Lambda function, you specify the HomeDirectoryType as LOGICAL, and add Entry and Target values for the HomeDirectoryDetails parameter. For example:

HomeDirectoryType: "LOGICAL"
HomeDirectoryDetails: "[{\"Entry\": \"/\", \"Target\": \"/amzn-s3-demo-bucket/theRealFolder"}]"

The following code is an example of a successful response from a custom Lambda authentication call.

aws transfer test-identity-provider --server-id s-1234567890abcdef0 --user-name myuser
{
    "Url": "https://a1b2c3d4e5.execute-api.us-east-2.amazonaws.com/prod/servers/s-1234567890abcdef0/users/myuser/config", 
    "Message": "", 
    "Response": "{\"Role\": \"arn:aws:iam::123456789012:role/bob-usa-role\",\"HomeDirectoryType\": \"LOGICAL\",\"HomeDirectoryDetails\": \"[{\\\"Entry\\\":\\\"/myhome\\\",\\\"Target\\\":\\\"/amzn-s3-demo-bucket/theRealFolder\\\"}]\",\"PublicKeys\": \"[ssh-rsa myrsapubkey]\"}", 
    "StatusCode": 200
}