|
com.amazonaws
aws-java-sdk-bom
>{{1.12.7871}}
pom
import
com.amazonaws
aws-java-sdk-s3
| 1 [最新版本](https://central.sonatype.com/artifact/com.amazonaws/aws-java-sdk-bom)。 2 [最新版本](https://central.sonatype.com/artifact/software.amazon.awssdk/bom)。 3 [最新版本](https://central.sonatype.com/artifact/software.amazon.awssdk.crt/aws-crt)。 ## 客户端构造函数迁移 ### 支持的构造函数（自动迁移） | V1 构造函数 | V2 等效项 | 迁移状态 | | --- | --- | --- | | new TransferManager() | S3TransferManager.create() | 支持 | | TransferManagerBuilder. defaultTransferManager() | S3TransferManager.create() | 支持 | | TransferManagerBuilder. standard().build() | S3TransferManager.builder().build() | 支持 | | new TransferManager(AWSCredentials) | S3TransferManager.builder() .s3Client(S3AsyncClient.builder() .credentialsProvider(...).build()) .build() | 支持 | | new TransferManager( AWSCredentialsProvider) | S3TransferManager.builder() .s3Client(S3AsyncClient.builder() .credentialsProvider(...).build()) .build() | 支持 | ### 不支持的构造函数（需要手动迁移） | V1 构造函数 | V2 等效项 | 迁移说明 | | --- | --- | --- | | new TransferManager(AmazonS3) | 需要手动迁移 | 单独创建 S3AsyncClient | | new TransferManager(AmazonS3, ExecutorService) | 需要手动迁移 | 创建 S3AsyncClient 并配置执行器 | | new TransferManager(AmazonS3, ExecutorService, boolean) | 需要手动迁移 | 不支持 shutDownThreadPools 参数 | ### 手动迁移示例 **V1 代码：** ``` AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient(); TransferManager transferManager = new TransferManager(s3Client); ``` **V2 代码：** ``` // Create an `S3AsyncClient` with similar configuration S3AsyncClient s3AsyncClient = S3AsyncClient.builder() .credentialsProvider(DefaultCredentialsProvider.create()) .build(); // Provide the configured `S3AsyncClient` to the S3 transfer manager builder. S3TransferManager transferManager = S3TransferManager.builder() .s3Client(s3AsyncClient) .build(); ``` ## 客户端方法迁移 目前，迁移工具支持基本 `copy`、`download`、`upload`、`uploadDirectory`、`downloadDirectory`、`resumeDownload` 和 `resumeUpload` 方法。 ### 核心传输方法 | V1 方法 | V2 方法 | 返回类型更改 | 迁移状态 | | --- | --- | --- | --- | | upload(String, String, File) | uploadFile(UploadFileRequest) | Upload → FileUpload | 支持 | | upload(PutObjectRequest) | upload(UploadRequest) | Upload → Upload | 支持 | | download(String, String, File) | downloadFile(DownloadFileRequest) | Download → FileDownload | 支持 | | download(GetObjectRequest, File) | downloadFile(DownloadFileRequest) | Download → FileDownload | 支持 | | copy(String, String, String, String) | copy(CopyRequest) | Copy → Copy | 支持 | | copy(CopyObjectRequest) | copy(CopyRequest) | Copy → Copy | 支持 | | uploadDirectory(String, String, File, boolean) | uploadDirectory( UploadDirectoryRequest) | MultipleFileUpload → DirectoryUpload | 支持 | | downloadDirectory(String, String, File) | downloadDirectory( DownloadDirectoryRequest) | MultipleFileDownload → DirectoryDownload | 支持 | ### 可恢复的传输方法 | V1 方法 | V2 方法 | 迁移状态 | | --- | --- | --- | | resumeUpload(PersistableUpload) | resumeUploadFile(ResumableFileUpload) | 支持 | | resumeDownload(PersistableDownload) | resumeDownloadFile(ResumableFileDownload) | 支持 | ### 生命周期方法 | V1 方法 | V2 方法 | 迁移状态 | | --- | --- | --- | | shutdownNow() | close() | 支持 | | shutdownNow(boolean) | 使用 close() 方法手动调整代码 | 不支持 | ### 不支持的 V1 客户端方法 | V1 方法 | V2 替代方案 | 注意 | | --- | --- | --- | | abortMultipartUploads(String, Date) | 使用低级别 Amazon S3 客户端 | 不支持 | | getAmazonS3Client() | 单独保存引用 | 不支持；v2 中没有 getter | | getConfiguration() | 单独保存引用 | 不支持；v2 中没有 getter | | uploadFileList(...) | 多次调用 uploadFile() | 不支持 | | 使用 TransferStateChangeListener 参数的 copy 方法 | 使用 TransferListener | [参见手动迁移示例](#tm-unsupported-client-methods-copy) | | 使用 S3ProgressListener 参数的 download 方法 | 使用 [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/transfer/s3/progress/TransferListener.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/transfer/s3/progress/TransferListener.html) | [参见手动迁移示例](#tm-unsupported-client-methods-download) | | 使用 4 个或更多参数的 `downloadDirectory` 方法 | | [参见手动迁移示例](#tm-unsupported-client-methods-download-dir) | | 使用 ObjectMetadataProvider 参数的 upload 方法 | 在请求中设置元数据 | [参见手动迁移示例](#tm-unsupported-client-methods-upload) | | 使用 \*Provider 参数的 uploadDirectory 方法 | 在请求中设置标签 | [参见手动迁移示例](#tm-unsupported-client-methods-uploadDirectory) | #### 使用 `TransferStateChangeListener` 参数的 `copy` 方法 + `copy(CopyObjectRequest copyObjectRequest, AmazonS3 srcS3, TransferStateChangeListener stateChangeListener)` + `copy(CopyObjectRequest copyObjectRequest, TransferStateChangeListener stateChangeListener)` ``` // V1 ---------------------------------------------------------------------------------------------- // Initialize source S3 client AmazonS3 s3client = AmazonS3ClientBuilder.standard() .withRegion("us-west-2") .build(); // Initialize Transfer Manager TransferManager tm = TransferManagerBuilder.standard() .withS3Client(srcS3) .build(); CopyObjectRequest copyObjectRequest = new CopyObjectRequest( "amzn-s3-demo-source-bucket", "source-key", "amzn-s3-demo-destination-bucket", "destination-key" ); TransferStateChangeListener stateChangeListener = new TransferStateChangeListener() { @Override public void transferStateChanged(Transfer transfer, TransferState state) { //Implementation of the TransferStateChangeListener } }; Copy copy = tm.copy(copyObjectRequest, srcS3, stateChangeListener); // V2 ---------------------------------------------------------------------------------------------- S3AsyncClient s3AsyncClient = S3AsyncClient.builder() .region(Region.US_WEST_2) .build(); S3TransferManager transferManager = S3TransferManager.builder() .s3Client(s3AsyncClient) .build(); // Create transfer listener (equivalent to TransferStateChangeListener in v1) TransferListener transferListener = new TransferListener() { @Override public void transferInitiated(Context.TransferInitiated context) { //Implementation System.out.println("Transfer initiated"); } @Override public void bytesTransferred(Context.BytesTransferred context) { //Implementation System.out.println("Bytes transferred"); } @Override public void transferComplete(Context.TransferComplete context) { //Implementation System.out.println("Transfer completed!"); } @Override public void transferFailed(Context.TransferFailed context) { //Implementation System.out.println("Transfer failed"); } }; CopyRequest copyRequest = CopyRequest.builder() .copyObjectRequest(req -> req .sourceBucket("amzn-s3-demo-source-bucket") .sourceKey("source-key") .destinationBucket("amzn-s3-demo-destination-bucket") .destinationKey("destination-key") ) .addTransferListener(transferListener) // Configure the transferListener into the request .build(); Copy copy = transferManager.copy(copyRequest); ``` #### 使用 `S3ProgressListener` 参数的 `download` 方法 + `download(GetObjectRequest getObjectRequest, File file, S3ProgressListener progressListener)` + `download(GetObjectRequest getObjectRequest, File file, S3ProgressListener progressListener, long timeoutMillis)` + `download(GetObjectRequest getObjectRequest, File file, S3ProgressListener progressListener, long timeoutMillis, boolean resumeOnRetry)` ``` // V1 ---------------------------------------------------------------------------------------------- S3ProgressListener progressListener = new S3ProgressListener() { @Override public void progressChanged(com.amazonaws.event.ProgressEvent progressEvent) { long bytes = progressEvent.getBytesTransferred(); ProgressEventType eventType = progressEvent.getEventType(); // Use bytes and eventType as needed } @Override public void onPersistableTransfer(PersistableTransfer persistableTransfer) { } }; Download download1 = tm.download(getObjectRequest, file, progressListener); Download download2 = tm.download(getObjectRequest, file, progressListener, timeoutMillis) Download download3 = tm.download(getObjectRequest, file, progressListener, timeoutMillis, true) // V2 ---------------------------------------------------------------------------------------------- TransferListener transferListener = new TransferListener() { @Override public void transferInitiated(Context.InitializedContext context) { // Equivalent to ProgressEventType.TRANSFER_STARTED_EVENT System.out.println("Transfer initiated"); } @Override public void bytesTransferred(Context.BytesTransferred context) { // Equivalent to ProgressEventType.REQUEST_BYTE_TRANSFER_EVENT long bytes = context.bytesTransferred(); System.out.println("Bytes transferred: " + bytes); } @Override public void transferComplete(Context.TransferComplete context) { // Equivalent to ProgressEventType.TRANSFER_COMPLETED_EVENT System.out.println("Transfer completed"); } @Override public void transferFailed(Context.TransferFailed context) { // Equivalent to ProgressEventType.TRANSFER_FAILED_EVENT System.out.println("Transfer failed: " + context.exception().getMessage()); } }; DownloadFileRequest downloadFileRequest = DownloadFileRequest.builder() .getObjectRequest(getObjectRequest) .destination(file.toPath()) .addTransferListener(transferListener) .build(); // For download1 FileDownload download = transferManager.downloadFile(downloadFileRequest); // For download2 CompletedFileDownload completedFileDownload = download.completionFuture() .get(timeoutMillis, TimeUnit.MILLISECONDS); // For download3, the v2 SDK does not have a direct equiavalent to the `resumeOnRetry` method of v1. // If a download is interrupted, you need to start a new download request. ``` #### 使用 4 个或更多参数的 `downloadDirectory` 方法 + `downloadDirectory(String bucketName, String keyPrefix, File destinationDirectory, boolean resumeOnRetry)` + `downloadDirectory(String bucketName, String keyPrefix, File destinationDirectory, boolean resumeOnRetry, KeyFilter filter)` + `downloadDirectory(String bucketName, String keyPrefix, File destinationDirectory, KeyFilter filter)` ``` // V1 ---------------------------------------------------------------------------------------------- KeyFilter filter = new KeyFilter() { @Override public boolean shouldInclude(S3ObjectSummary objectSummary) { //Filter implementation } }; MultipleFileDownload multipleFileDownload = tm.downloadDirectory(bucketName, keyPrefix, destinationDirectory, filter); // V2 ---------------------------------------------------------------------------------------------- // The v2 SDK does not have a direct equiavalent to the `resumeOnRetry` method of v1. // If a download is interrupted, you need to start a new download request. DownloadFilter filter = new DownloadFilter() { @Override public boolean test(S3Object s3Object) { // Filter implementation. } }; DownloadDirectoryRequest downloadDirectoryRequest = DownloadDirectoryRequest.builder() .bucket(bucketName) .filter(filter) .listObjectsV2RequestTransformer(builder -> builder.prefix(keyPrefix)) .destination(destinationDirectory.toPath()) .build(); DirectoryDownload directoryDownload = transferManager.downloadDirectory(downloadDirectoryRequest); ``` #### 使用 `ObjectMetadata` 参数的 `upload` 方法 + `upload(String bucketName, String key, InputStream input, ObjectMetadata objectMetadata)` ``` // V1 ----------------------------------------------------------------------------------------------ObjectMetadata metadata = new ObjectMetadata(); ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentType("text/plain"); // System-defined metadata metadata.setContentLength(22L); // System-defined metadata metadata.addUserMetadata("myKey", "myValue"); // User-defined metadata PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, key, inputStream, metadata); Upload upload = transferManager.upload("amzn-s3-demo-bucket", "my-key", inputStream, metadata); // V2 ---------------------------------------------------------------------------------------------- /* When you use an InputStream to upload in V2, you should specify the content length and use `RequestBody.fromInputStream()`. If you don't provide the content length, the entire stream will be buffered in memory. If you can't determine the content length, we recommend using the CRT-based S3 client. */ Map
software.amazon.awssdk
bom
{{2.31.682}}
pom
import
software.amazon.awssdk
s3-transfer-manager
software.amazon.awssdk.crt
aws-crt
{{0.38.53}}
TransferManagerBuilder tmBuilder =|
TransferManagerBuilder.standard();
S3TransferManager.Builder tmBuilder =| |
S3TransferManager.builder();
tmBuilder.withS3Client(...);|
tmBuilder.setS3Client(...);
tmBuilder.s3Client(...);| |
tmBuilder.withExecutorFactory(...);|
tmBuilder.setExecutorFactory(...);
tmBuilder.executor(...);| |
tmBuilder.withShutDownThreadPools(...);| 不支持。关闭 S3TransferManager 时，不会关闭提供的执行程序 | |
tmBuilder.setShutdownThreadPools(...);
tmBuilder.withMinimumUploadPartSize(...);|
tmBuilder.setMinimumUploadPartSize(...);
S3AsyncClient s3 = S3AsyncClient.crtBuilder().| |
minimumPartSizeInBytes(...).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartUploadThreshold(...);|
tmBuilder.setMultipartUploadThreshold(...);
S3AsyncClient s3 = S3AsyncClient.crtBuilder().| |
thresholdInBytes(...).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartCopyPartSize(...);|
tmBuilder.setMultipartCopyPartSize(...);
S3AsyncClient s3 = S3AsyncClient.crtBuilder().| |
minimumPartSizeInBytes(...).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartCopyThreshold(...);|
tmBuilder.setMultipartCopyThreshold(...);
S3AsyncClient s3 = S3AsyncClient.crtBuilder().| |
thresholdInBytes(...).build();
tmBuilder.s3Client(s3);
tmBuilder.withDisableParallelDownloads(...);| 通过将默认禁用分段的基于 Java 的标准 S3 客户端传递给 Transfer Manager 来禁用并行下载。
tmBuilder.setDisableParallelDownloads(...);
S3AsyncClient s3 =| |
S3AsyncClient.builder().build();
tmBuilder.s3Client(s3);
tmBuilder.withAlwaysCalculateMultipartMd5(...);| 不支持。 | ------ #### [ Use Java-based S3 async client ] **** | 设置 | v1 | v2 - 使用基于 Java 的 S3 异步客户端的 Transfer Manager | | --- | --- | --- | |
tmBuilder.setAlwaysCalculateMultipartMd5(...);
TransferManagerBuilder tmBuilder =|
TransferManagerBuilder.standard();
S3TransferManager.Builder tmBuilder =| |
S3TransferManager.builder();
tmBuilder.withS3Client(...);|
tmBuilder.setS3Client(...);
tmBuilder.s3Client(...);| |
tmBuilder.withExecutorFactory(...);|
tmBuilder.setExecutorFactory(...);
tmBuilder.executor(...);| |
tmBuilder.withShutDownThreadPools(...);| 不支持。关闭 S3TransferManager 时，不会关闭提供的执行程序 | |
tmBuilder.setShutdownThreadPools(...);
tmBuilder.withMinimumUploadPartSize(...);|
tmBuilder.setMinimumUploadPartSize(...);
S3AsyncClient s3 = S3AsyncClient.builder()| |
.multipartConfiguration(cfg ->
cfg.minimumPartSizeInBytes(...)).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartUploadThreshold(...);|
tmBuilder.setMultipartUploadThreshold(...);
S3AsyncClient s3 = S3AsyncClient.builder()| |
.multipartConfiguration(cfg ->
cfg.thresholdInBytes(...)).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartCopyPartSize(...);|
tmBuilder.setMultipartCopyPartSize(...);
S3AsyncClient s3 = S3AsyncClient.builder()| |
.multipartConfiguration(cfg ->
cfg.minimumPartSizeInBytes(...)).build();
tmBuilder.s3Client(s3);
tmBuilder.withMultipartCopyThreshold(...);|
tmBuilder.setMultipartCopyThreshold(...);
S3AsyncClient s3 = S3AsyncClient.builder()| |
.multipartConfiguration(cfg ->
cfg.thresholdInBytes(...)).build();
tmBuilder.s3Client(s3);
tmBuilder.withDisableParallelDownloads(...);| 通过将默认禁用分段的基于 Java 的标准 S3 客户端传递给 Transfer Manager 来禁用并行下载。
tmBuilder.setDisableParallelDownloads(...);
S3AsyncClient s3 =| |
S3AsyncClient.builder().build();
tmBuilder.s3Client(s3);
tmBuilder.withAlwaysCalculateMultipartMd5(...);| 不支持。 | ------ ## 行为更改 ### 异步操作 **V1（阻塞）：** ``` Upload upload = transferManager.upload("amzn-s3-demo-bucket", "key", file); upload.waitForCompletion(); // Blocks until complete ``` **V2（异步）：** ``` FileUpload upload = transferManager.uploadFile(UploadFileRequest.builder() .putObjectRequest(PutObjectRequest.builder() .bucket("amzn-s3-demo-bucket") .key("key") .build()) .source(file) .build()); CompletedFileUpload result = upload.completionFuture().join(); // Blocks until complete // Or handle asynchronously: upload.completionFuture().thenAccept(result -> { System.out.println("Upload completed: " + result.response().eTag()); }); ``` ### 错误处理 **V1：**如果任何子请求失败，则目录传输将完全失败。 **V2：**即使某些子请求失败，目录传输也会成功完成。明确检查错误： ``` DirectoryUpload directoryUpload = transferManager.uploadDirectory(request); CompletedDirectoryUpload result = directoryUpload.completionFuture().join(); // Check for failed transfers if (!result.failedTransfers().isEmpty()) { System.out.println("Some uploads failed:"); result.failedTransfers().forEach(failed -> System.out.println("Failed: " + failed.exception().getMessage())); } ``` ### 通过字节范围提取进行并行下载 在 v2 SDK 中启用自动并行传输功能后，S3 Transfer Manager 使用[字节范围提取](https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance-guidelines.html#optimizing-performance-guidelines-get-range)来并行检索对象的特定部分（分段下载）。使用 v2 下载对象的方式不取决于对象最初的上传方式。所有下载都可以从高吞吐量和并发性中受益。 相比之下，使用 v1 Transfer Manager 时，对象最初的上传方式确实很重要。v1 Transfer Manager 检索对象各个分段的方式与上传分段的方式相同。如果对象最初是作为单个对象上传的，则 v1 Transfer Manager 无法通过使用子请求来加速下载过程。
tmBuilder.setAlwaysCalculateMultipartMd5(...);