使用 Amazon S3 Transfer Manager 傳輸檔案和目錄 - AWS SDK for Java 2.x
開始使用上傳檔案下載檔案複製物件上傳目錄下載至目錄請參閱完整範例

本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

使用 Amazon S3 Transfer Manager 傳輸檔案和目錄

Amazon S3 Transfer Manager 是 的開放原始碼、高階檔案傳輸公用程式 AWS SDK for Java 2.x。使用它在 Amazon Simple Storage Service (Amazon S3) 之間傳輸檔案和目錄。

在啟用分段的 AWS CRT 型 S3 用戶端或標準 Java 型 S3 非同步用戶端上建置時，S3 Transfer Manager 可以利用效能改善，例如分段上傳 API 和位元組範圍擷取。 S3

使用 S3 Transfer Manager，您也可以即時監控傳輸進度，並暫停傳輸以供稍後執行。

開始使用

將相依性新增至建置檔案

若要使用具有增強分段效能的 S3 Transfer Manager，請設定具有必要相依性的建置檔案。

Use the AWS CRT-based S3 client
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>bom</artifactId>
            <version>2.27.211</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>s3-transfer-manager</artifactId>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk.crt</groupId>
        <artifactId>aws-crt</artifactId>
        <version>0.29.1432</version>
    </dependency>
</dependencies>

1 最新版本。 2最新版本。

Use the Java-based S3 async client
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>bom</artifactId>
            <version>2.27.211</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>s3-transfer-manager</artifactId>
    </dependency>
</dependencies>

1 最新版本。

建立 S3 Transfer Manager 的執行個體

若要啟用平行傳輸，您必須傳入 AWS CRT 型 S3 用戶端或啟用分段的 Java 型 S3 非同步用戶端。下列範例示範如何使用自訂設定來設定 S3 Transfer Manager。

Use the AWS CRT-based S3 client
        S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder()
                .credentialsProvider(DefaultCredentialsProvider.create())
                .region(Region.US_EAST_1)
                .targetThroughputInGbps(20.0)
                .minimumPartSizeInBytes(8 * MB)
                .build();

        S3TransferManager transferManager = S3TransferManager.builder()
                .s3Client(s3AsyncClient)
                .build();
Use the Java-based S3 async client

如果建置檔案中未包含aws-crt相依性，S3 Transfer Manager 會建置在適用於 Java 2.x 的 SDK 中使用的標準 Java 型 S3 非同步用戶端之上。

S3 用戶端的自訂組態 - 需要啟用分段

        S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
                .multipartEnabled(true)
                .credentialsProvider(DefaultCredentialsProvider.create())
                .region(Region.US_EAST_1)
                .build();

        S3TransferManager transferManager = S3TransferManager.builder()
                .s3Client(s3AsyncClient)
                .build();

沒有 S3 用戶端的組態 - 自動啟用分段支援

S3TransferManager transferManager = S3TransferManager.create();

將檔案上傳至 S3 儲存貯體

下列範例顯示檔案上傳範例，以及選用的 LoggingTransferListener，這會記錄上傳進度。

若要使用 Amazon S3 S3，請將UploadFileRequest物件傳遞至 S3TransferManager的 uploadFile 方法。

從 uploadFile方法傳回的 FileUpload 物件代表上傳程序。請求完成後，CompletedFileUpload 物件會包含上傳的相關資訊。

    public void trackUploadFile(S3TransferManager transferManager, String bucketName,
                             String key, URI filePathURI) {
        UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
                .putObjectRequest(b -> b.bucket(bucketName).key(key))
                .addTransferListener(LoggingTransferListener.create())  // Add listener.
                .source(Paths.get(filePathURI))
                .build();

        FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);

        fileUpload.completionFuture().join();
        /*
            The SDK provides a LoggingTransferListener implementation of the TransferListener interface.
            You can also implement the interface to provide your own logic.

            Configure log4J2 with settings such as the following.
                <Configuration status="WARN">
                    <Appenders>
                        <Console name="AlignedConsoleAppender" target="SYSTEM_OUT">
                            <PatternLayout pattern="%m%n"/>
                        </Console>
                    </Appenders>

                    <Loggers>
                        <logger name="software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener" level="INFO" additivity="false">
                            <AppenderRef ref="AlignedConsoleAppender"/>
                        </logger>
                    </Loggers>
                </Configuration>

            Log4J2 logs the progress. The following is example output for a 21.3 MB file upload.
                Transfer initiated...
                |                    | 0.0%
                |====                | 21.1%
                |============        | 60.5%
                |====================| 100.0%
                Transfer complete!
        */
    }

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedFileUpload;
import software.amazon.awssdk.transfer.s3.model.FileUpload;
import software.amazon.awssdk.transfer.s3.model.UploadFileRequest;
import software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Paths;
import java.util.UUID;

從 S3 儲存貯體下載檔案

下列範例顯示下載範例，以及選用使用 LoggingTransferListener 記錄下載進度。

若要使用 S3 Transfer Manager 從 S3 儲存貯體下載物件，請建置 DownloadFileRequest 物件並將其傳遞至 downloadFile 方法。

的 downloadFile方法傳回S3TransferManager的 FileDownload 物件代表檔案傳輸。下載完成後，CompletedFileDownload 會包含下載相關資訊的存取權。

    public void trackDownloadFile(S3TransferManager transferManager, String bucketName,
                             String key, String downloadedFileWithPath) {
        DownloadFileRequest downloadFileRequest = DownloadFileRequest.builder()
                .getObjectRequest(b -> b.bucket(bucketName).key(key))
                .addTransferListener(LoggingTransferListener.create())  // Add listener.
                .destination(Paths.get(downloadedFileWithPath))
                .build();

        FileDownload downloadFile = transferManager.downloadFile(downloadFileRequest);

        CompletedFileDownload downloadResult = downloadFile.completionFuture().join();
        /*
            The SDK provides a LoggingTransferListener implementation of the TransferListener interface.
            You can also implement the interface to provide your own logic.

            Configure log4J2 with settings such as the following.
                <Configuration status="WARN">
                    <Appenders>
                        <Console name="AlignedConsoleAppender" target="SYSTEM_OUT">
                            <PatternLayout pattern="%m%n"/>
                        </Console>
                    </Appenders>

                    <Loggers>
                        <logger name="software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener" level="INFO" additivity="false">
                            <AppenderRef ref="AlignedConsoleAppender"/>
                        </logger>
                    </Loggers>
                </Configuration>

            Log4J2 logs the progress. The following is example output for a 21.3 MB file download.
                Transfer initiated...
                |=======             | 39.4%
                |===============     | 78.8%
                |====================| 100.0%
                Transfer complete!
        */
    }


import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedFileDownload;
import software.amazon.awssdk.transfer.s3.model.DownloadFileRequest;
import software.amazon.awssdk.transfer.s3.model.FileDownload;
import software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener;

import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.UUID;

將 Amazon S3 物件複製到另一個儲存貯體

下列範例示範如何使用 S3 Transfer Manager 複製物件。

若要開始將物件從 S3 儲存貯體複製到另一個儲存貯體，請建立基本 CopyObjectRequest 執行個體。

接著，將基本 包裝在 CopyRequest CopyObjectRequest中，供 S3 Transfer Manager 使用。

S3TransferManager的 copy方法傳回的Copy物件代表複製程序。複製程序完成後，CompletedCopy 物件會包含回應的詳細資訊。

    public String copyObject(S3TransferManager transferManager, String bucketName,
            String key, String destinationBucket, String destinationKey) {
        CopyObjectRequest copyObjectRequest = CopyObjectRequest.builder()
                .sourceBucket(bucketName)
                .sourceKey(key)
                .destinationBucket(destinationBucket)
                .destinationKey(destinationKey)
                .build();

        CopyRequest copyRequest = CopyRequest.builder()
                .copyObjectRequest(copyObjectRequest)
                .build();

        Copy copy = transferManager.copy(copyRequest);

        CompletedCopy completedCopy = copy.completionFuture().join();
        return completedCopy.response().copyObjectResult().eTag();
    }
注意

若要使用 S3 Transfer Manager 執行跨區域複製，請在 AWS CRT 型 S3 用戶端建置器crossRegionAccessEnabled上啟用 ，如下列程式碼片段所示。

S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder()
                .crossRegionAccessEnabled(true)
                .build();

S3TransferManager transferManager = S3TransferManager.builder()
                .s3Client(s3AsyncClient)
                .build();
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.model.CopyObjectRequest;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedCopy;
import software.amazon.awssdk.transfer.s3.model.Copy;
import software.amazon.awssdk.transfer.s3.model.CopyRequest;

import java.util.UUID;

將本機目錄上傳至 S3 儲存貯體

下列範例示範如何將本機目錄上傳至 S3。

首先呼叫S3TransferManager執行個體的 uploadDirectory 方法，傳入 UploadDirectoryRequest。

DirectoryUpload 物件代表上傳程序，該程序會在請求完成時產生 CompletedDirectoryUpload。CompleteDirectoryUpload 物件包含傳輸結果的相關資訊，包括哪些檔案無法傳輸。

    public Integer uploadDirectory(S3TransferManager transferManager,
            URI sourceDirectory, String bucketName) {
        DirectoryUpload directoryUpload = transferManager.uploadDirectory(UploadDirectoryRequest.builder()
                .source(Paths.get(sourceDirectory))
                .bucket(bucketName)
                .build());

        CompletedDirectoryUpload completedDirectoryUpload = directoryUpload.completionFuture().join();
        completedDirectoryUpload.failedTransfers()
                .forEach(fail -> logger.warn("Object [{}] failed to transfer", fail.toString()));
        return completedDirectoryUpload.failedTransfers().size();
    }

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.services.s3.model.ObjectIdentifier;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedDirectoryUpload;
import software.amazon.awssdk.transfer.s3.model.DirectoryUpload;
import software.amazon.awssdk.transfer.s3.model.UploadDirectoryRequest;

import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Paths;
import java.util.UUID;

將 S3 儲存貯體物件下載至本機目錄

您可以將 S3 儲存貯體中的物件下載至本機目錄，如下列範例所示。

若要將 S3 儲存貯體中的物件下載至本機目錄，請先呼叫 Transfer Manager 的 downloadDirectory 方法，傳入 DownloadDirectoryRequest。

DirectoryDownload 物件代表下載程序，該程序會在請求完成時產生 CompletedDirectoryDownload。CompleteDirectoryDownload 物件包含傳輸結果的相關資訊，包括哪些檔案無法傳輸。

    public Integer downloadObjectsToDirectory(S3TransferManager transferManager,
            URI destinationPathURI, String bucketName) {
        DirectoryDownload directoryDownload = transferManager.downloadDirectory(DownloadDirectoryRequest.builder()
                .destination(Paths.get(destinationPathURI))
                .bucket(bucketName)
                .build());
        CompletedDirectoryDownload completedDirectoryDownload = directoryDownload.completionFuture().join();

        completedDirectoryDownload.failedTransfers()
                .forEach(fail -> logger.warn("Object [{}] failed to transfer", fail.toString()));
        return completedDirectoryDownload.failedTransfers().size();
    }

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.model.ObjectIdentifier;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedDirectoryDownload;
import software.amazon.awssdk.transfer.s3.model.DirectoryDownload;
import software.amazon.awssdk.transfer.s3.model.DownloadDirectoryRequest;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.HashSet;
import java.util.Set;
import java.util.UUID;
import java.util.stream.Collectors;

請參閱完整範例

GitHub 包含此頁面上所有範例的完整程式碼。