# Using logical directories to simplify your Transfer Family directory structures Use logical directories Logical directories simplify your AWS Transfer Family server directory structure. With logical directories, you can create a virtual directory structure with user-friendly names that users navigate when connecting to your Amazon S3 bucket or Amazon EFS file system. This prevents users from seeing the actual directory paths, bucket names, and file system names. **Note** You should use session policies so that your end users can only perform operations that you allow them to perform. You should use logical directories to create a user-friendly, virtual directory for your end users and abstract away bucket names. Logical directory mappings only allow users to access their designated logical paths and subdirectories, and forbid relative paths that traverse the logical roots. Transfer Family validates every path that might include relative elements and actively blocks these paths from resolving before we pass these paths to Amazon S3; this prevents your users from moving beyond their logical mappings. Even though Transfer Family prevents your end users from accessing directories outside of their logical directory, we recommend you also use unique roles or session policies to enforce least privilege at the storage level. ## Understanding chroot and directory structure A **chroot** operation lets you set a user's root directory to any location in your storage hierarchy. This restricts users to their configured home or root directory, preventing access to higher-level directories. Consider a case where an Amazon S3 user is limited to `amzn-s3-demo-bucket/home/${transfer:UserName}`. Without **chroot**, some clients might allow users to move up to /amzn-s3-demo-bucket/home, requiring a logout and login to return to their proper directory. Performing a **chroot** operation prevents this issue. You can create custom directory structures across multiple buckets and prefixes. This is useful if your workflow requires a specific directory layout that bucket prefixes alone can't provide. You can also link to multiple non-contiguous locations within Amazon S3, similar to creating a symbolic link in a Linux file system where your directory path references a different location in the file system. ## Rules for using logical directories This section describes some rules and other considerations for using logical directories. ### Mapping limits + Only one mapping is allowed when `Entry` is `"/"` (no overlapping paths are allowed). + Logical directories support mappings of up to 2.1 MB for custom IDP and AD users, and 2,000 entries for service-managed users. You can calculate your mappings size as follows: 1. Write out a typical mapping in the format `{"Entry":"/entry-path","Target":"/target-path"}`, where `entry-path` and `target-path` are the actual values that you will use. 1. Count the characters in that string, then add one (1). 1. Multiply that number by the approximate number of mappings that you have for your server. If the number that you estimated in step 3 is less than 2.1 MB, then your mappings are within the acceptable limit. ### Target path requirements + Use `${transfer:UserName}` variable if the bucket or file system path has been parameterized based on the username. + Targets can be configured to point to different Amazon S3 buckets or file systems, as long as the associated IAM role has the necessary permissions to access those storage locations. + All targets must begin with a forward slash (`/`) but can't end with one. For example, `/amzn-s3-demo-bucket/images` is correct, while `amzn-s3-demo-bucket/images `and `/amzn-s3-demo-bucket/images/` are not. ### Storage considerations + Amazon S3 is an object store where folders exist only as a virtual concept. When using Amazon S3 storage, Transfer Family reports prefixes as directories in STAT operations, even if there is no zero-byte object with a trailing slash. A proper zero-byte object with a trailing slash is also reported as a directory in STAT operations. This behavior is described in [ Organizing objects in the Amazon S3 console using folders](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) in the *Amazon Simple Storage Service User Guide*. + For applications that need to distinguish between files and folders, use Amazon Elastic File System (Amazon EFS) as your Transfer Family storage option. + If you're specifying logical directory values for your user, the parameter that you use depends on the type of user: + For service-managed users, provide logical directory values in `HomeDirectoryMappings`. + For custom identity provider users, provide logical directory values in `HomeDirectoryDetails`. ### User directory values + The parameter for specifying logical directory values depends on your user type: + For service-managed users, provide logical directory values in `HomeDirectoryMappings`. + For custom identity provider users, provide logical directory values in `HomeDirectoryDetails`. + When using LOGICAL HomeDirectoryType, you can specify a HomeDirectory value for Service Managed users, Active Directory access, and Custom Identity Provider implementations where the HomeDirectoryDetails are provided in the response. If not specified, HomeDirectory defaults to `/`. For details on how to implement logical directories, see [Implementing logical directories](implement-log-dirs.md). # Implementing logical directories **Important** **Root directory requirements** If you are not using Amazon S3 performance optimization settings your root directory must exist on startup. For Amazon S3, this means creating a zero-byte object that ends with a forward slash (`/`). To avoid this requirement, consider enabling Amazon S3 performance optimization when you create or update your server. When specifying a HomeDirectory with LOGICAL HomeDirectoryType, the value must map to one of your logical directory mappings. The service validates this during both user creation and updates to prevent configurations that would not work. **Logical home directory configuration** When using LOGICAL as your HomeDirectoryType, note the following: The HomeDirectory value must correspond to one of your existing logical directory mappings. The system automatically validates this during user creation and updates. This validation prevents configurations that would cause access issues. ## Enable logical directories Enable chroot To use logical directories for a user, set the `HomeDirectoryType` parameter to `LOGICAL`. Do this when you create a new user or update an existing user. ``` "HomeDirectoryType": "LOGICAL" ``` ## Enable `chroot` for users For **chroot**, create a directory structure that consists of a single `Entry` and `Target` pairing for each user. The Entry **/** represents the root folder, while **Target** specifies the actual location in your bucket or file system. ------ #### [ Example for Amazon S3 ] ``` [{"Entry": "/", "Target": "/amzn-s3-demo-bucket/jane"}] ``` ------ #### [ Example for Amazon EFS ] ``` [{"Entry": "/", "Target": "/fs-faa1a123/jane"}] ``` ------ You can use an absolute path as in the previous example, or you can use a dynamic substitution for the username with `${transfer:UserName}`, as in the following example. ``` [{"Entry": "/", "Target": "/amzn-s3-demo-bucket/${transfer:UserName}"}] ``` In the preceding example, the user is locked to their root directory and cannot traverse up higher in the hierarchy. ## Virtual directory structure For a virtual directory structure, you can create multiple `Entry` `Target` pairings, with targets anywhere in your S3 buckets or EFS file systems, including across multiple buckets or file systems, as long as the user’s IAM role mapping has permissions to access them. In the following virtual structure example, when the user logs into AWS SFTP, they are in the root directory with sub-directories of `/pics`, `/doc`, `/reporting`, and `/anotherpath/subpath/financials`. **Note** Unless you choose to optimize performance for your Amazon S3 directories (when you create or update a server), either the user or an administrator needs to create the directories if they don't already exist. Avoiding this issue is a reason to consider optimizing Amazon S3 performance. For Amazon EFS, you still need the administrator to create the logical mappings or the `/` directory. ``` [ {"Entry": "/pics", "Target": "/amzn-s3-demo-bucket1/pics"}, {"Entry": "/doc", "Target": "/amzn-s3-demo-bucket1/anotherpath/docs"}, {"Entry": "/reporting", "Target": "/amzn-s3-demo-bucket2/Q1"}, {"Entry": "/anotherpath/subpath/financials", "Target": "/amzn-s3-demo-bucket2/financials"}] ``` **Note** You can only upload files to the specific folders that you map. This means that in the previous example, you cannot upload to `/anotherpath` or `anotherpath/subpath` directories; only `anotherpath/subpath/financials`. You also cannot map to those paths directly, as overlapping paths are not allowed. For example, assume that you create the following mappings: ``` { "Entry": "/pics", "Target": "/amzn-s3-demo-bucket/pics" }, { "Entry": "/doc", "Target": "/amzn-s3-demo-bucket/mydocs" }, { "Entry": "/temp", "Target": "/amzn-s3-demo-bucket2/temporary" } ``` You can only upload files to those buckets. When you first connect through `sftp`, you are dropped into the root directory, `/`. If you attempt to upload a file to that directory, the upload fails. The following commands show an example sequence: ``` sftp> pwd Remote working directory: / sftp> put file Uploading file to /file remote open("/file"): No such file or directory ``` To upload to any `directory/sub-directory`, you must explicitly map the path to the `sub-directory`. For more information about configuring logical directories and **chroot** for your users, including an AWS CloudFormation template that you can download and use, see [ Simplify your AWS SFTP Structure with chroot and logical directories](https://aws.amazon.com/blogs/storage/simplify-your-aws-sftp-structure-with-chroot-and-logical-directories/) in the AWS Storage Blog. # Configure logical directories examples In this example, we create a user and assign two logical directories. The following command creates a new user (for an existing Transfer Family server) with logical directories `pics` and `doc`. ``` aws transfer create-user \ --user-name marymajor \ --server-id s-11112222333344445 \ --role arn:aws:iam::1234abcd5678:role/marymajor-role \ --home-directory-type LOGICAL \ --home-directory-mappings "[{\"Entry\":\"/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, {\"Entry\":\"/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]" \ --ssh-public-key-body file://~/.ssh/id_rsa.pub ``` If **marymajor** is an existing user and her home directory type is `PATH`, you can change it to `LOGICAL` with a similar command as the previous one. ``` aws transfer update-user \ --user-name marymajor \ --server-id s-11112222333344445 \ --role arn:aws:iam::1234abcd5678:role/marymajor-role \ --home-directory-type LOGICAL \ --home-directory-mappings "[{\"Entry\":\"/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, {\"Entry\":\"/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]" ``` Note the following: + If the directories `/amzn-s3-demo-bucket1/pics` and `/amzn-s3-demo-bucket2/test/mydocs` don't already exist, the user (or an administrator) needs to create them. **Note** These directories are created automatically by the Transfer Family server if you have configured optimized directories. + When **marymajor** connects to the server, and runs the `ls -l` command, Mary sees the following: ``` drwxr--r-- 1 - - 0 Mar 17 15:42 doc drwxr--r-- 1 - - 0 Mar 17 16:04 pics ``` + **marymajor** cannot create any files or directories at this level. However, within `pics` and `doc`, she can add sub-directories. + Files that Mary adds to `pics` and `doc` are added to Amazon S3 paths `/amzn-s3-demo-bucket1/pics` and `/amzn-s3-demo-bucket2/test/mydocs` respectively. + In this example, we specify two different buckets to illustrate that possibility. However, you can use the same bucket for several or all of the logical directories that you specify for the user. This example provides an alternate configuration for a logical home path. ``` aws transfer create-user \ --user-name marymajor \ --server-id s-11112222333344445 \ --role arn:aws:iam::1234abcd5678:role/marymajor-role \ --home-directory-type LOGICAL \ --home-directory /home/marymajor \ --home-directory-mappings "[{\"Entry\":\"/home/marymajor/pics\", \"Target\":\"/amzn-s3-demo-bucket1/pics\"}, {\"Entry\":\"/home/marymajor/doc\", \"Target\":\"/amzn-s3-demo-bucket2/test/mydocs\"}]" \ --ssh-public-key-body file://~/.ssh/id_rsa.pub ``` Note the following: + The mappings provide for a common path, `/home/marymajor`, which is the first part of the two logical paths. Files then can be added to the `pics` and `doc` folders. + As in the previous example, the home directory, `/home/marymajor`, is read-only. ## Configure logical directories for Amazon EFS If your Transfer Family server uses Amazon EFS, the home directory for the user must be created with read and write access before the user can work in their logical home directory. The user cannot create this directory themselves, as they would lack permissions for `mkdir` on their logical home directory. If the user's home directory does not exist, and they run an `ls` command, the system responds as follows: ``` sftp> ls remote readdir ("/"): No such file or directory ``` A user with administrative access to the parent directory needs to create the user's logical home directory. ## Custom AWS Lambda response You can use logical directories with a Lambda function that connects to your custom identity provider. To do so, in your Lambda function, you specify the `HomeDirectoryType` as **LOGICAL**, and add `Entry` and `Target` values for the `HomeDirectoryDetails` parameter. For example: ``` HomeDirectoryType: "LOGICAL" HomeDirectoryDetails: "[{\"Entry\": \"/\", \"Target\": \"/amzn-s3-demo-bucket/theRealFolder"}]" ``` The following code is an example of a successful response from a custom Lambda authentication call. ``` aws transfer test-identity-provider \ --server-id s-1234567890abcdef0 \ --user-name myuser { "Url": "https://a1b2c3d4e5.execute-api.us-east-2.amazonaws.com/prod/servers/s-1234567890abcdef0/users/myuser/config", "Message": "", "Response": "{\"Role\": \"arn:aws:iam::123456789012:role/bob-usa-role\", \"HomeDirectoryType\": \"LOGICAL\", \"HomeDirectoryDetails\": \"[{\\\"Entry\\\":\\\"/myhome\\\",\\\"Target\\\":\\\"/amzn-s3-demo-bucket/theRealFolder\\\"}]\", \"PublicKeys\": \"[ssh-rsa myrsapubkey]\"}", "StatusCode": 200 } ``` **Note** The `"Url":` line is returned only if you are using an API Gateway method as your custom identity provider.