# Subscribing to AWS Data Exchange data products on AWS Data Exchange Subscribing to data products At a high level, this is how to subscribe to AWS Data Exchange data products available through AWS Marketplace: 1. Create an AWS account – You must sign up for AWS and create a user before you can subscribe to data products. For more information, see [Setting up AWS Data Exchange](setting-up.md). 1. Browse the public catalog – Products are published to the AWS Marketplace catalog. You can find products and review the associated public or custom offers and product details. If the provider has issued a private offer to your account, the product is available on the **My product offers** page of the AWS Data Exchange console. 1. **Submit a request for a subscription** – You must submit a request to subscribe. The request form requires additional information about your identity and intended use case. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. **Subscriber subscribes to the product** – If you subscribe to a paid product, you are billed on your AWS bill. You get access to the entitled data set. 1. **Uses the included data sets** – You have access to the product data sets according to the terms of the data subscription agreement. You can take any of the following actions depending on the type of data set you have access to: + Export the associated files to your Amazon Simple Storage Service (Amazon S3) or locally through a signed URL. + Call the Amazon API Gateway API. + Query the Amazon Redshift data share. + Access the provider's Amazon S3 data. + Query the provider's AWS Lake Formation data lake (Preview). For more information, see [Jobs in AWS Data Exchange](jobs.md). 1. **Request a data product recommendation** – If you are not able to find a product in the catalog, you can use the **Request data product page** in the AWS Data Exchange console to request personalized recommendations from the AWS Data Exchange Data Discovery Team. For more information, see [Request a recommendation for a data product](subscriber-getting-started.md#request-new-products). **Note** When subscribing to data products from some non-US sellers, you might also receive a tax invoice from the seller. For more information, see [Tax Help - AWS Marketplace Sellers](https://aws.amazon.com/tax-help/marketplace/). ## Related topics + [Product subscriptions in AWS Data Exchange](product-subscriptions.md) + [Getting started as a subscriber in AWS Data Exchange](subscriber-getting-started.md) + [Subscribing to and accessing an AWS Data Exchange product](subscribing-to-product.md) + [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md) + [Sharing AWS Data Exchange license subscriptions in an organization](organizations-sharing.md) + [Accepting Bring Your Own Subscription (BYOS) offers in AWS Data Exchange](subscribe-to-byos-offer.md) + [Accepting private products and offers in AWS Data Exchange](subscribe-to-private-offer.md) + [AWS Data Exchange Heartbeat](heartbeat.md) + [AWS Data Exchange for APIs (Test Product)](subscriber-tutorial-api-product.md) + [Worldwide Event Attendance (Test Product) on AWS Data Exchange](subscriber-tutorial-RS-product.md) + [AWS Data Exchange for AWS Lake Formation (Test Product) (Preview)](subscriber-tutorial-LF-product.md) + [AWS Data Exchange for Amazon S3 (Test Product)](subscriber-tutorial-s3-product.md) + [AWS Data Exchange Provider-Generated Notifications (Test Product)](subscriber-tutorial-provider-notifications.md) + [Data in AWS Data Exchange](data-sets.md) # Product subscriptions in AWS Data Exchange Product subscriptions All AWS Data Exchange products are subscription-based. When you subscribe to a product, you agree to the product's offer terms, including the price, duration, payment schedule, data subscription agreement, and refund policy. When you subscribe to a product, you pay according to the payment schedule chosen by the provider for the duration that you subscribed to. **Important** The data subscription agreement (DSA) sets forth the provider’s terms and conditions for the data product. The use of any data product subscribed to on AWS Data Exchange must also be in compliance with the AWS Customer Agreement or other agreement governing your use of AWS services. **Note** Data products that are part of the [Open Data on AWS](https://aws.amazon.com/opendata/) program are free for anyone to use and do not require a subscription. For more information, see [Using Open Data on AWS data sets with AWS Data Exchange](open-data.md). Each product's public offer terms can contain one or more price and duration combinations. When you subscribe to a product, you can choose the duration of the subscription. You can also choose whether you would like to enable auto-renewal for that subscription, if the provider has enabled it for the product. **Important** If the data provider has indicated that the product contains any categories of sensitive or personal data, for example, mobile IDs, it will be displayed with the product details. For more information about the categories of sensitive data, see [Sensitive categories of information in AWS Data Exchange](sensitive-information.md). If the data provider has indicated that the product contains protected health information (PHI) subject to the Health Insurance Portability and Accountability Act of 1996 (HIPAA), you may not export the product's data sets into your AWS account unless such AWS account is designated as a HIPAA account (as defined in the AWS Business Associate Addendum found in [AWS Artifact](https://docs.aws.amazon.com/artifact/latest/ug/what-is-aws-artifact.html)). After a subscription is processed and active, it appears on your AWS bill according to the payment schedule as part of your AWS Marketplace charges. For more information, see [AWS Marketplace Paying for Products](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-paying-for-products.html). During the duration of your subscription, you can view and access all the product's data sets. You can also export the data sets' assets in jobs. For more information, see [Jobs in AWS Data Exchange](jobs.md). Once a subscription has expired, you can no longer view or export the data sets. **Note** For information about data sets and revisions, including details about what you have access to in your subscription, see [Data sets and revisions](#product-sub-revisions). If a provider decides to unpublish a product, you still have access to the data sets as long as your subscription is active. However, you cannot auto-renew the subscription when it expires. You can view all of your active product subscriptions and auto-renewal status on the **Subscriptions** page of the AWS Data Exchange console. For more information, see [Managing AWS Data Exchange subscriptions](managing-subscriptions.md). **Important** If you enable auto-renew, and the product's offer terms have changed at the time of renewal, then the new product offer terms (including new price and new DSA) apply. This ensures that you keep access to the data regardless of potential changes to offer terms. Visit the **Entitled data sets **page to find and access all of your entitled data sets in a specific AWS Region, based on your active subscriptions. When you subscribe to a data product, we might share your contact information with the provider. For more information, see [Security on AWS Marketplace](https://docs.aws.amazon.com//marketplace/latest/buyerguide/buyer-security.html#what-information-do-you-share-with-the-software-seller-about-the-customers-of-a-product.html) in the *AWS Marketplace Buyer Guide*. When you purchase a data product on AWS Data Exchange that has an upfront commitment, you will receive an invoice from Amazon Web Services (AWS) immediately. You can see charges for each data product by name in the Detail section of the invoice. You will receive separate bills for usage of AWS infrastructure and analytics services such as Amazon Simple Storage Service (Amazon S3) or Amazon Athena. For more information about AWS Billing and Cost Management, see [ Paying for products](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-paying-for-products.html) in the *AWS Marketplace Buyer Guide*. When your subscription to an AWS Data Exchange Files data set ends, you retain access to any files that you already exported. Review your Data Subscription Agreement to verify if your agreement requires that you delete exported data when ending a subscription. ## Data sets and revisions Every product in AWS Data Exchange is made up of one or more data sets, each with one or more revisions. Data sets in AWS Data Exchange are typically different data, and revisions are newer or modified versions of the same data. For more information about data sets and revisions, see [Data in AWS Data Exchange](data-sets.md). Each revision may contain all the data for the data set (updated for the revision), or just the new data since the previous revision. It is even possible that each revision has completely different data. What data to provide in each revision is up to the data provider. When you subscribe to a product, you have access to all data sets in the product. When the data provider creates the offer, they give you access to 0 or more historical revisions, up to all historical revisions. They can also give you access to future revisions that are made available during your subscription period. The terms of the subscription are shown on the product details page in the AWS Data Exchange console. After you subscribe to a product containing files, you can manually export each revision or asset individually, or you can select to automatically export new revisions to your Amazon S3 buckets (up to five buckets maximum) when the provider publishes new revisions. For more information, see [Subscribing to and accessing an AWS Data Exchange product containing file-based data](subscribing-to-data-product.md). For more information about how to export revisions, see [Exporting revisions from AWS Data Exchange](exporting-revisions.md). After you subscribe to a product containing an Amazon API Gateway API, you can view and invoke the data provider's API. For more information, see [Subscribing to and accessing an AWS Data Exchange product containing APIs](subscribing-to-API-product.md). After you subscribe to a product containing Amazon Redshift data sets, you get access to query the data in Amazon Redshift. For more information, see [Subscribing to and accessing an AWS Data Exchange product containing Amazon Redshift data sets](subscribing-to-Redshift-product.md). After you subscribe to an Amazon S3 data access data set, you can view and directly use the provider's Amazon S3 objects. For more information, see [Subscribing to and accessing an AWS Data Exchange product containing Amazon S3 data access](subscribing-to-S3-data-access.md). After you subscribe to a product containing AWS Lake Formation data permission data sets (Preview), you can manage the data in AWS Lake Formation and query it with downstream services like Amazon Athena. ## Data dictionaries and samples Some products have data sets that include data dictionaries and samples. To help you determine whether you want to subscribe to the product, you can view and download the data dictionaries and samples before you subscribe to it. A *data dictionary* is a visual representation of the contents of a data set. It includes details about what columns are included and their meaning. *Samples* are pieces of data that reflect the data that you would receive after you subscribe to the product. Samples can be any file type supported by Amazon S3. For more information about how to discover a product that contains data dictionaries and samples, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). For more information about how to evaluate a product using data dictionaries and samples, see [Evaluate products containing data dictionaries and samples](subscriber-getting-started.md#evaluate-products). # Getting started as a subscriber in AWS Data Exchange Getting started as a subscriber The following topics describe the complete process of becoming a data product subscriber on AWS Data Exchange using the AWS Data Exchange console. The process has the following steps: **Topics** + [ ## Step 1: Set up AWS Data Exchange ](#subscriber-prereqs) + [ ## Step 2: Browse the catalog ](#browse-catalog) + [ ## Step 3: (Optional) Request a recommendation for a data product ](#request-new-products) + [ ## Step 4: (Optional) Evaluate products containing data dictionaries and samples ](#evaluate-products) + [ ## Step 5: Subscribe to and access a product ](#subscribe-to-product) ## Step 1: Set up AWS Data Exchange Before you can use AWS Data Exchange, you must sign up for AWS and create a user. For more information, see [Setting up AWS Data Exchange](setting-up.md). **To set up AWS Data Exchange** 1. Sign up for an AWS account. For more information, see [Sign up for an AWS account](setting-up.md#setting-up-aws-sign-up). 1. Create a user. For more information, see [Create a user](setting-up.md#setting-up-create-iam-user). ## Step 2: Browse the catalog You can find products and review the associated public or custom offers and product details on both AWS Marketplace and AWS Data Exchange. If the provider has issued a private offer to your account, the product is available on the **My product offers page** of the AWS Data Exchange console. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). **To browse the catalog** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. 1. Enter a term or phrase in the **Search** bar and then choose **Search**. 1. (Optional) Under **Browse catalog**, enter in a word or phrase and then choose **Search** to view results matching your query. 1. (Optional) Under **Refine results**, choose from one of the specific **Categories** to browse specific data products. 1. (Optional) Under **Refine results**, use the **Data set type** filter and select from the following options to find products: + **Files (Amazon S3 Objects)** – Products containing file-based data + **Amazon Redshift** – Products containing Amazon Redshift datashares + **API** – Products containing APIs + **Access to Amazon S3** – Products containing Amazon S3 data access + **AWS Lake Formation** – Products containing AWS Lake Formation data permissions (Preview) 1. Select a product from the list of returned results, and review its product details page. ## Step 3: (Optional) Request a recommendation for a data product If you're unable to find a product in the catalog, you can request personalized recommendations from the [AWS Data Exchange Data Discovery Team](https://aws.amazon.com/data-exchange/discover-data/). **To request a data product recommendation** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Request data product**. 1. On the **Request data product** page, for **Details**, enter a **Data description**. 1. (Optional) Expand **Additional details - optional** and complete the fields as directed. 1. Select one or more **Product categories**. 1. Enter an **Example data product URL**. 1. For **Data set type**, choose from **Files (Amazon S3 Objects)**, **Amazon API Gateway API**, **Amazon Redshift datashare**, **AWS Lake Formation data permissions (Preview)** or **Amazon S3 data access**. 1. Enter specific details about the product you want including **Delivery cadence**, **Example data product URL**, **Subscription start date**, **Subscription length**, and **Subscription budget**. 1. If the **Data set type** you chose is **Amazon API Gateway API**, under **Subscription budget**, select **Including metered costs**. 1. For **Data providers**, choose from a list of **Existing providers** or enter the name of **Other providers**. Then indicate whether you have an existing relationship with the providers. 1. Choose **Submit**. You should receive a response from the AWS Data Exchange Data Discovery Team within 2 business days. ## Step 4: (Optional) Evaluate products containing data dictionaries and samples A provider might include a data dictionary and samples of the data set with their product. To help you determine if the product’s data set will meet your needs, you can view and download the data dictionary and samples before you subscribe. For more information, see [Data dictionaries and samples](product-subscriptions.md#dictionaries-and-samples). You can perform the following actions to help with your evaluation of a product’s data sets: + [View a data dictionary](#view-data-dictionary) + [Download a data dictionary](#download-data-dictionary) + [View and download all data dictionaries](#view-download-all-dictionaries) (for products containing multiple data sets) + [Preview a sample](#preview-sample) + [Download a sample](#download-sample) ### Viewing a data dictionary A provider can add one data dictionary per data set that you can view. **To view a data dictionary** 1. On the product detail page, choose the **Data dictionary and samples** tab. 1. View the data dictionary in one of the following ways: + Scroll down to the product **Overview** section to see the data dictionary under **View data dictionaries**. + Choose the **Data dictionaries and samples** tab, expand a data set row, choose the option button next to a data dictionary, and then choose **View all data dictionaries**. 1. (Optional) Enter a keyword or phrase into the **Search** bar to search across all data sets and all tables. 1. (Optional) Modify your search and filters as necessary. ### Downloading a data dictionary A provider can add one data dictionary per data set that you can download. **To download a data dictionary** 1. On the product detail page, choose the **Data dictionary and samples** tab. 1. Expand the data set row by choosing the expand icon (plus icon to the left of the data set name). 1. Choose the option button next to a data dictionary name. 1. Choose **Download**. The data dictionary file is downloaded to your computer. ### Viewing and downloading all data dictionaries If the product has multiple data sets, the provider might add a data dictionary for each data set. To evaluate all the data sets, you might want to view and download all data dictionaries. **To view and download all data dictionaries** 1. On the product detail page, choose the **Data dictionary and samples** tab. 1. Choose **View all data dictionaries**. 1. In the **View data dictionaries** dialog box, choose the **Download (CSV)** to download the .csv file. The .csv file is downloaded to your computer. 1. Choose **Close** to close the dialog box. ### Previewing a sample **To preview a sample** 1. On the product detail page, choose the **Data dictionary and samples** tab. 1. Expand the data set by choosing the expand icon (plus icon to the left of the data set name) 1. Choose the option button next to a sample name. 1. Choose **Preview sample (CSV only)** to preview the sample. 1. (Optional) In the preview dialog box, choose **Download** to download the .csv file. The .csv file is downloaded to your computer. 1. Choose **Close** to close the dialog box. ### Downloading a sample **To download a sample** 1. On the product detail page, choose the **Data dictionary and samples** tab. 1. Expand the data set by choosing the expand icon (plus icon to the left of the data set name) 1. Choose the option button next to a sample name. 1. Choose **Download**. The sample is downloaded to your computer. ## Step 5: Subscribe to and access a product After you discover a product in the AWS Data Exchange catalog and determine that it meets your needs, you can subscribe to the product and then access the product. If you subscribe to a paid product, you are billed on your AWS bill. You get access to the entitled data set. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). For more information about how to subscribe to products containing different types of data sets, see the following: + [Subscribing to and accessing an AWS Data Exchange product containing file-based data](subscribing-to-data-product.md) + [Subscribing to and accessing an AWS Data Exchange product containing APIs](subscribing-to-API-product.md) + [Subscribing to and accessing an AWS Data Exchange product containing Amazon Redshift data sets](subscribing-to-Redshift-product.md) + [Subscribing to and accessing an AWS Data Exchange product containing Amazon S3 data access](subscribing-to-S3-data-access.md) + [Subscribing to and accessing an AWS Data Exchange product containing AWS Lake Formation data sets (Preview)](subscribing-to-LakeFormation-product.md) # Subscribing to and accessing an AWS Data Exchange product Subscribing to a product The following topics describe the process of subscribing to and accessing a product on AWS Data Exchange using the AWS Data Exchange console. **Topics** + [ # Subscribing to and accessing an AWS Data Exchange product containing file-based data ](subscribing-to-data-product.md) + [ # Subscribing to and accessing an AWS Data Exchange product containing APIs ](subscribing-to-API-product.md) + [ # Subscribing to and accessing an AWS Data Exchange product containing Amazon Redshift data sets ](subscribing-to-Redshift-product.md) + [ # Subscribing to and accessing an AWS Data Exchange product containing Amazon S3 data access ](subscribing-to-S3-data-access.md) + [ # Subscribing to and accessing an AWS Data Exchange product containing AWS Lake Formation data sets (Preview) ](subscribing-to-LakeFormation-product.md) + [ # Viewing and downloading a data dictionary in AWS Data Exchange ](view-download-dictionaries.md) **Note** By subscribing to a product, you agree that your use of the product is subject to the provider's offer terms including pricing information and data subscription agreement (DSA). You also agree and acknowledge that AWS may share information about the transaction (including your payment terms and product usage metrics) with the respective seller, reseller, or underlying provider, as applicable, in accordance with the [AWS Privacy Notice](https://aws.amazon.com/privacy/). AWS will issue invoices and collect payments from you on behalf of the provider through your AWS account. Your use of AWS services remains subject to the AWS Customer Agreement or other agreement with AWS governing your use of such services. # Subscribing to and accessing an AWS Data Exchange product containing file-based data Containing file-based data The following topics describe the complete process of subscribing to and accessing a product containing file-based data stored as files on AWS Data Exchange. To complete the process, use the AWS Data Exchange console. For information about how to evaluate a product before subscribing, see [Evaluate products containing data dictionaries and samples](subscriber-getting-started.md#evaluate-products). The process has the following steps: **Topics** + [ ## Step 1: Subscribing to a product containing the file-based data ](#subscribe-to-data-product) + [ ## Step 2: Accessing a product containing file-based data ](#use-product) To practice subscribing to and accessing a product containing file-based data, see the [AWS Data Exchange Heartbeat](heartbeat.md). ## Step 1: Subscribing to a product containing the file-based data If you subscribe to a paid product, you are billed on your AWS bill. You get access to all entitled data sets. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). **To subscribe to a product containing the file-based data** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. 1. Under **Refine results**, use the **Data set type** filter and select **Files (Amazon S3 Objects)** to find products containing file-based data. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Select a data product containing **Files (Amazon S3 Objects)**, and view its product detail page. The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and durations, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under **Similar products**. If the provider has issued a custom offer to your account (for example, a [private offer](subscribe-to-private-offer.md) or [Bring Your Own Subscription (BYOS) offer](subscribe-to-byos-offer.md)), you see those details, too. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the DSA. **Note** Some products require subscription verification. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. 1. Under **Data sets included with your subscription**, view the listed **Data sets**. After the subscription finishes processing, you can choose a data set to access your entitled data or choose **View subscription** to view your subscription. 1. (*Optional*) For **Set up exports - *optional***, select the check boxes for the data sets that contain the revisions that you want to export. Selecting a data set will prepare its most recently published revision to be exported. 1. Choose a **Simple** destination option to select an Amazon S3 bucket location or choose **Advanced** to configure an Amazon S3 key naming pattern. This choice determines where your revisions will be exported. For more information about using key patterns, see [Key patterns when exporting asset revisions from AWS Data Exchange](revision-export-keypatterns.md). 1. For **Auto-export future revisions**, choose whether to turn on or turn off automatic revision export: + **On** – All future revisions will always be exported. + **Off** – Only one export of the most recent revision will be exported. 1. Choose the **Encryption** options, and review the **Amazon S3 pricing**. **Note** If you choose to export using AWS Key Management Service (AWS KMS) encryption, make sure your account has the correct AWS Identity and Access Management (IAM) permissions to create and revoke grants on the AWS KMS key you choose. Without these permissions, automatic export will fail. 1. Choose **Export** to export the data to Amazon S3, or choose **Skip** if you prefer to wait and export or download later. For more information about how to export data after subscribing, see [(Optional) Exporting data](#export-data-after-subscribing). **Note** It can take a few minutes for your subscription to become active after you choose **Subscribe**. If you choose **Export** before the subscription is active, you are prompted to wait until it is complete. After your subscription is active, your export will begin. Navigating away from this page prior to your subscription becoming active will not prevent the subscription from processing. It will prevent your data export from occurring. ## Step 2: Accessing a product containing file-based data After you successfully subscribe to a product, you have access to the product data sets according to the terms of the data subscription agreement (DSA). The following topic describes how to access a product containing file-based data. ### Viewing data sets, revisions, and assets **To view the data sets, revisions, and assets** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, choose **Subscriptions**, and then choose your product. 1. View the data sets that are part of the product under **Entitled data sets**. 1. Choose a data set. 1. View the **Data set overview**, **Auto-export job destinations** (Amazon S3 products only), the **Revisions**, and the **Description** of the data set. For more information, see [Data in AWS Data Exchange](data-sets.md). 1. Choose a revision. Revisions are listed from latest to oldest. 1. View the **Revision overview**, **Assets**, and the **Jobs** that have been performed. For information about exporting file-based assets, see [Exporting AWS Data Exchange assets to an S3 bucket as a subscriber (console)](export-asset-s3-console-sub.md). ### (Optional) Exporting data After your subscription is active, you can set up your Amazon S3 bucket to receive assets that you export. You can export the associated assets to Amazon S3 or you can use jobs with a signed URL. If you want to export or download your data at a later time, including getting new revisions, see [Exporting AWS Data Exchange assets to an S3 bucket as a subscriber (console)](export-asset-s3-console-sub.md). **Important** We recommend that you consider Amazon S3 security features when exporting data to Amazon S3. For more information about general guidelines and best practices, see [Security best practices for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/security-best-practices.html) in the *Amazon Simple Storage Service User Guide*. For more information about how to export data, see [Exporting assets from AWS Data Exchange](exporting-assets.md) and [Exporting revisions from AWS Data Exchange](exporting-revisions.md). # Subscribing to and accessing an AWS Data Exchange product containing APIs Containing APIs The following topics describe the complete process of subscribing to and accessing a product containing APIs on AWS Data Exchange by using the AWS Data Exchange console. For information about how to evaluate a product before subscribing, see [Evaluate products containing data dictionaries and samples](subscriber-getting-started.md#evaluate-products). The process has the following steps: **Topics** + [ ## Step 1: Subscribing to a product containing APIs ](#subscribe-to-API-product) + [ ## Step 2: Accessing an API product ](#use-API-product) To practice subscribing to and accessing a product containing APIs, see the [AWS Data Exchange for APIs (Test Product)](subscriber-tutorial-api-product.md). ## Step 1: Subscribing to a product containing APIs If you subscribe to a paid product, you're billed on your AWS bill. You get access to all entitled data sets. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). A provider might include metered costs to their product containing APIs. If a provider decreases metered costs, the price decrease goes into effect immediately. If the provider increases metered costs, and you're an existing subscriber, the price increase goes into effect on the first day of the month, 90 days after the price increase was submitted OR upon renewal (whichever is sooner). An email message is sent to existing subscribers when the price change is submitted. **Example** For example, assume that a provider submits a metered cost price increase on May 10. Existing subscribers receive an email message about the price change. The price increase goes into effect on September 1. **To subscribe to a product containing APIs** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Under **Refine results**, use the **Data set type** filter and select **API** to find products containing APIs. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Select a product containing APIs, and view its product detail page. The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and durations, metered costs (if included), the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under **Similar products**. If the provider has issued a custom offer to your account (for example, a [private offer](subscribe-to-private-offer.md) or [Bring Your Own Subscription (BYOS) offer](subscribe-to-byos-offer.md)), you see those details, too. 1. Under **Public offer**, view the **API metered costs** (if included). 1. (Optional) In the **Metered cost calculator**, choose **Select metered cost** and then enter the number of units to display an example of the cost. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the DSA. **Note** Some products require subscription verification. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** If you subscribe to a paid product, you're prompted to confirm your decision to subscribe. 1. Under **Data sets included with your subscription**, view the listed **Data sets**. After the subscription finishes processing, you can choose a data set to access your entitled data or choose **View subscription** to view your subscription. ## Step 2: Accessing an API product The following topics provide details about how to access a product that includes API data sets: **Topics** + [ ### Viewing an API ](#view-the-api) + [ ### Downloading the API specification ](#download-api-spec) + [ ### Making an API call (console) ](#make-api-call-console) + [ ### Making an API call (AWS CLI) ](#make-api-call-cli) ### Viewing an API **To view an API** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose a data set. 1. Under the **Revisions** tab, choose a revision. 1. Under **API assets**, choose the API. 1. View the **Asset overview**. 1. Follow the guidance in the **Integration notes** to call the API. ### Downloading the API specification **To download the API specification** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose a data set. 1. Under the **Revisions** tab, choose a revision. 1. Under **API assets**, choose the API. 1. On the **OpenAPI 3.0 specification**, choose **Download API specification**. The specification is downloaded onto your local computer. You can then export the asset to a third-party tool for SDK generation. ### Making an API call (console) You can call a single endpoint in the AWS Data Exchange console. **To make an API call from the console** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose a data set. 1. Under the **Revisions** tab, choose a revision. 1. Under **API assets**, choose the API. 1. For **Integration notes**: 1. Choose **Copy** to use the **Base URL**. 1. Choose **Copy** to use the **Code structure**. 1. Follow the information provided in the specification documentation to call the API. ### Making an API call (AWS CLI) **To make an API call (AWS CLI)** + Use the `send-api-asset` command to call the API. ``` $ AWS dataexchange send-api-asset \ --asset-id $ASSET_ID \ --data-set-id $DATA_SET_ID \ --revision-id $REVISION_ID \ --body "..." \ { "headers": { ... }, "body": "..." } ``` # Subscribing to and accessing an AWS Data Exchange product containing Amazon Redshift data sets Containing Amazon Redshift data sets **Overview for recipients** An Amazon Redshift data set is a data set that contains AWS Data Exchange datashares for Amazon Redshift. Datashares give you read-only access to the tables, views, schemas, and user-defined functions that a data provider adds to the datashare. As a data subscriber, you can find and subscribe to products containing Amazon Redshift data sets. After your subscription starts, you get access to query the data in Amazon Redshift without extracting, transforming, and loading data. You lose access to a product's datashares after your subscription expires. Consider the following: + It might take a few minutes to access the datashares after your subscription starts. The following sections describe the complete process of becoming an Amazon Redshift datashare product subscriber on AWS Data Exchange by using the AWS Data Exchange console. For information about how to evaluate a product before subscribing, see [Evaluate products containing data dictionaries and samples](subscriber-getting-started.md#evaluate-products). The process has the following steps: **Topics** + [ ## Step 1: Subscribing to products containing Amazon Redshift data sets ](#subscribe-Redshift-product) + [ ## Step 2: Accessing the AWS Data Exchange datashares for Amazon Redshift ](#use-Redshift-product) To practice subscribing to and accessing a product containing Amazon Redshift data sets, see the [Worldwide Event Attendance (Test Product) on AWS Data Exchange](subscriber-tutorial-RS-product.md). ## Step 1: Subscribing to products containing Amazon Redshift data sets If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). **To subscribe to a product containing Amazon Redshift data sets** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Under **Refine results**, use the **Data set type** filter and select **Amazon Redshift** to find products containing Amazon Redshift datashares. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Select a product and view its product detail page. The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under **Similar products**. If the provider has issued a custom offer to your account (for example, a [private offer](subscribe-to-private-offer.md) or [Bring Your Own Subscription (BYOS) offer)](subscribe-to-byos-offer.md), you see those details, too. **Important** Be sure to review the date, time, and duration of the cluster’s maintenance window. During the maintenance window, you do not have access to the datashare. 1. In the top right corner, choose **Continue to subscribe**. 1. Review the **Product offer**, the **Subscription terms**, the **Data sets** that are included in the offer, and the **Support information**. 1. Choose whether to enable **Offer auto-renewal** for the subscription **Note** Some products require subscription verification. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Choose **Subscribe**. **Note** If you subscribe to a paid product, you're prompted to confirm your decision to subscribe. 1. Under **Data sets included with your subscription**, view the listed **Data sets**. After the subscription finishes processing, you can choose a data set to access your entitled data or choose **View subscription** to view your subscription. ## Step 2: Accessing the AWS Data Exchange datashares for Amazon Redshift You have access to the product's data sets according to the terms of the data subscription agreement (DSA). As a subscriber, your subscription to a product that includes AWS Data Exchange datashares for Amazon Redshift gives you read-only access to the tables, views, schemas, and functions within the datashare. With a subscription, you can do the following: + Query data without having to extract, transform, or load data. + Access the latest provider data as soon as the provider updates it. For more information, see [Working with AWS Data Exchange datashares](https://docs.aws.amazon.com/redshift/latest/dg/adx-datashare.html) in the *Amazon Redshift Database Developer Guide*. **Note** You lose access to a product's datashares after your subscription expires. For more information about how to subscribe to an Amazon Redshift data set, see [Worldwide Event Attendance (Test Product) on AWS Data Exchange](subscriber-tutorial-RS-product.md). # Subscribing to and accessing an AWS Data Exchange product containing Amazon S3 data access Containing Amazon S3 data access AWS Data Exchange for Amazon S3 allows data subscribers to access third-party data files directly from data providers' Amazon S3 buckets. As a data subscriber, after you are entitled to an AWS Data Exchange for Amazon S3 data set, you can start your data analysis with AWS services such as Amazon Athena, SageMaker AI Feature Store, or Amazon EMR directly using the provider's data in their Amazon S3 buckets. Consider the following: + Providers have the option to enable **Requester Pays**, an Amazon S3 feature, on the Amazon S3 bucket hosting the data offered. If enabled, subscribers pay to read, use, transfer, export, or copy data into their Amazon S3 buckets. For more information, see [Using Requester Pays buckets for storage transfers and usage](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html) in the *Amazon Simple Storage Service User Guide*. + When you subscribe to an AWS Data Exchange for Amazon S3 data product, AWS Data Exchange automatically provisions an Amazon S3 access point and updates its resource policies to grant you read-only access. Amazon S3 access points is a feature of Amazon S3 that simplifies data sharing to an Amazon S3 bucket. For more information, see [Managing data access with Amazon S3 access points](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points.html) in the *Amazon Simple Storage Service User Guide*. + Before you use the Amazon S3 access point Amazon Resource Name (ARN) or alias to access the shared data, you must update your IAM permissions. You can verify that the current role and its associated policy allows `GetObject` and `ListBucket` calls to the provider’s Amazon S3 bucket and the Amazon S3 access point provided by AWS Data Exchange. The following sections describe the complete process of becoming an AWS Data Exchange for Amazon S3 subscriber by using the AWS Data Exchange console. The process has the following steps: **Topics** + [ ## Step 1: Subscribing to products containing Amazon S3 data access ](#subscribe-s3-data-access-product) + [ ## Step 2: Accessing a product containing Amazon S3 data access ](#use-S3-data-access-product) ## Step 1: Subscribing to products containing Amazon S3 data access If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). **To subscribe to a product containing access to Amazon S3** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Under **Refine results**, use the **Data set type** filter and select **Access to Amazon S3** to find products containing access to Amazon S3 data. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Select a product and view its product detail page. The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under **Similar products**. If the provider has issued a custom offer to your account (for example, a [private offer](subscribe-to-private-offer.md) or [Bring Your Own Subscription (BYOS) offer)](subscribe-to-byos-offer.md), you see those details, too. 1. In the top right corner, choose **Continue to subscribe**. 1. Review the **Product offer**, the **Subscription terms**, the **Data sets** that are included in the offer, and the **Support information**. 1. Choose whether enable **Offer auto-renewal** for the subscription **Note** Some products require subscription verification. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Choose **Subscribe**. **Note** If you subscribe to a paid product, you're prompted to confirm your decision to subscribe. 1. Under **Data sets included with your subscription**, view the listed **Data sets**. After the subscription finishes processing, you can choose a data set to access your entitled data or choose **View subscription** to view your subscription. ## Step 2: Accessing a product containing Amazon S3 data access You can run queries to analyze the data in-place without setting up your own Amazon S3 buckets, copying data files into Amazon S3 buckets, or paying associated storage fees. You access the same Amazon S3 objects that the data provider maintains allowing you to use the most current data available. With a subscription, you can do the following: + Analyze data without setting up individual Amazon S3 buckets, copying files, or paying storage fees. + Access the latest provider data as soon as the provider updates it. **To view the data sets, revisions, and assets** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. On the **Entitled data page**, expand a product and choose a data set. 1. View the **Data set overview**. **Note** The data provided is stored in the provider's Amazon S3 bucket. When accessing this data, you’ll be responsible for the cost of the request and the data downloaded from the provider’s Amazon S3 bucket, unless the provider specifies otherwise. 1. Before getting started, your role must have IAM permissions to use your entitled Amazon S3 data access. On the **Data set overview** page, on the **Amazon S3 data access** tab, select **Verify IAM permissions** to determine if your role has the correct permissions to access your data. 1. If you have the necessary IAM permissions, choose **Next** on the **IAM Policy** prompt displayed. If you don't have the needed permissions, follow the prompt to embed the JSON policy in the user or role. 1. Review your **Shared locations** to view the Amazon S3 bucket or prefixes and objects shared by the provider. Review the data access information for Amazon S3 Access Point information to determine if the provider enabled **Requester Pays**. 1. Choose **Browse shared Amazon S3 locations** to view and explore the provider's Amazon S3 bucket, prefixes, and objects shared. 1. Use the **Access Point** alias anywhere you use Amazon S3 bucket names to access your entitled data programmatically. For more information, see [Using access points with compatible Amazon S3 operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-usage-examples.html) in the *Amazon Simple Storage Service User Guide*. 1. (Optional) When you gain an entitlement to an Amazon S3 data access data set that contains data encrypted with a provider’s AWS KMS key, you can view the KMS key ARN in your subscriber console. AWS Data Exchange creates an AWS KMS grant on the key for you, so you can access the encrypted data. You must obtain `kms:Decrypt` IAM permission on the KMS key to read encrypted data from the Amazon S3 Access Point from which you’ve gained entitlement. You can choose between the following IAM policy statements: 1. IAM policy allowing users to decrypt or encrypt data with any KMS key. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": [ "*" ] } ] } ``` ------ 1. IAM policy allowing users to specify the exact KMS key ARNs visible in the subscriber console. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": [ "arn:aws:kms:us-east-1:111122223333:key/KeyId" ] } ] } ``` ------ **Note** AWS KMS grants can take up to 5 minutes for the operation to achieve eventual consistency. You might not have access to the Amazon S3 data access data set until this is complete. For more information, see [Grant in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/grants.html) in the *AWS Key Management Service Developer Guide*. For more information about how to subscribe to an Amazon S3 data set, see [Subscribing to and accessing an AWS Data Exchange product containing Amazon S3 data access](#subscribing-to-S3-data-access). # Subscribing to and accessing an AWS Data Exchange product containing AWS Lake Formation data sets (Preview) Containing AWS Lake Formation data sets (Preview) An AWS Lake Formation data set is a data set that contains AWS Lake Formation data permission assets. As a data subscriber, you can find and subscribe to products containing AWS Lake Formation data sets. Once you're entitled to an AWS Data Exchange for AWS Lake Formation data set, you can query, transform, and share access to the data within your AWS account using AWS Lake Formation, or across your AWS organization using AWS License Manager. ## Step 1: Subscribing to products containing AWS Lake Formation data sets If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see [Subscribing to AWS Data Exchange data products on AWS Data Exchange](subscribe-to-data-sets.md). **To subscribe to a product containing AWS Lake Formation data sets** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left side navigation pane, under **Discover data products**, choose **Browse catalog**. For more information, see [Browse the catalog](subscriber-getting-started.md#browse-catalog). 1. Under **Refine results**, use the **Data set type** filter and select **AWS Lake Formation** to find products containing AWS Lake Formation data sets. 1. Select a product and view its product detail page. The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they're available. You can also continue browsing other product detail pages by choosing a product under **Similar products**. If the provider has issued a custom offer to your account (for example, a [private offer](subscribe-to-private-offer.md) or [Bring Your Own Subscription (BYOS) offer)](subscribe-to-byos-offer.md), you see those details, too. 1. In the top right corner, choose **Continue to subscribe**. 1. Review the **Product offer**, the **Subscription terms**, the **Data sets** that are included in the offer, and the **Support information**. 1. Choose whether to enable **Offer auto-renewal** for the subscription. **Note** Some products require subscription verification. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Choose **Subscribe**. If you subscribe to a paid product, you're prompted to confirm your decision to subscribe. 1. Under **Data sets included with your subscription**, view the listed **Data sets**. After the subscription finishes processing, you can choose a data set to access your entitled data or choose **View subscription** to view your subscription. ## Step 2: Accessing the AWS Data Exchange datashares for AWS Lake Formation After you subscribe to a product containing AWS Lake Formation data sets, you can use Lake Formation compatible query engines, like Amazon Athena, to query your data. **After subscription completion, you must do the following:** 1. Accept the AWS Resource Access Manager (AWS RAM) share within 12 hours after you subscribe to the product. You can accept the AWS RAM share from your subscription page or the entitled data page for your AWS Lake Formation data permission data set on the AWS Data Exchange console. You only need to accept an AWS RAM share once per provider. For more information about accepting a resource share invitation from AWS RAM, see [Accepting a resource share invitation from AWS RAM](https://docs.aws.amazon.com/lake-formation/latest/dg/accepting-ram-invite.html). 1. Navigate to AWS Lake Formation and create resource links from the new shared resources. 1. Navigate to Athena or another AWS Lake Formation compatible query engine to query your data. # Viewing and downloading a data dictionary in AWS Data Exchange Viewing and downloading a data dictionary Providers can attach data dictionaries to all AWS Data Exchange products. The following procedures describe how to view and download a data dictionary. For more information about data dictionaries and samples, see [Data dictionaries and samples](product-subscriptions.md#dictionaries-and-samples). **To view and download a data dictionary** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. On the **Entitled data page**, expand a product and choose a data set. 1. On the data set details page, choose the **Data dictionary** tab. + View the data dictionary on the **Data dictionary** tab. + Download the data dictionary by choosing **Download** and then saving the file to your computer. # Subscription verification for subscribers in AWS Data Exchange Subscription verification for subscribers When you subscribe to AWS Data Exchange public products, you are required to submit additional information about your identity and your intended use case. The provider reviews this information before approving subscriptions. **Topics** + [Completing a subscription request](complete-sub-request.md) + [Reviewing your pending subscription requests](review-pending-sub-requests.md) + [Email notifications](email-notifications-sub.md) # Completing a subscription request in AWS Data Exchange Completing a subscription request AWS Data Exchange public products require subscription verification. After choosing **Continue to subscribe**, you must complete an additional form on the **Complete subscription request** page. **To complete a subscription request** 1. On the **Complete subscription request** page, review and choose the product offer (if more than one offer is available). 1. Review the **Subscription terms**, included **Data sets**, **Support information**, and **Refund policy**. 1. Choose if you want to renew the offer automatically when it expires. 1. On the **Subscription request form**, your AWS account ID will be added automatically. Complete the form by completing the following fields: + **Company name** + **Name** + **Email address** + **Company location** + **Intended use case** In addition to your proposed use case, you may include additional comments that could help the provider evaluate your request. 1. Choose **Send subscription request to provider**. After you submit your request, the provider has up to 45 days to approve or decline your request. # Reviewing your pending AWS Data Exchange subscription requests Reviewing your pending subscription requests Review your pending subscriptions for AWS Data Exchange products that require subscription verification. **To review your pending AWS Data Exchange subscription requests** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. Choose **Subscriptions**. 1. Choose **Subscription requests**. 1. Review the status of your pending subscription requests. Each subscription request is uniquely identified by its ID. The ID is visible to both the provider and the subscriber. You can use the subscription request ID to identify the request in your communications with the provider. After a provider approves your request, the subscription appears on the **Subscriptions** page. **Note** You can cancel a pending subscription request at any time as long as it hasn't expired or already been processed. # Email notifications to verify subscriptions in AWS Data Exchange Email notifications You receive an email notification to your AWS account email address when your request is approved, declined, or when it expires. Although most subscription request status changes result in an email notification, the delivery of these emails is on a best-effort basis. **Note** You will not receive email notifications for subscription request status changes that you have initiated yourself (for example, cancelling a subscription). # Sharing AWS Data Exchange license subscriptions in an organization Sharing license subscriptions in an organization When you subscribe to AWS Data Exchange products, an agreement is created that grants you license to use those products. If your AWS account is a member of an organization, you can share that license for AWS Data Exchange products with the other accounts in that organization. **Note** For more information about AWS Organizations, see the [AWS Organizations User Guide](https://docs.aws.amazon.com/organizations/latest/userguide/). The following topics outline the process of sharing the licenses across accounts. **Topics** + [ ## Prerequisites for license sharing ](#license-sharing-prereqs) + [ ## Step 1: View your licenses ](#view-share-licenses) + [ ## Step 2: Share your licenses ](#share-licenses) ## Prerequisites for license sharing Before you can share licenses for data products, you must first set up license sharing for your organization. Complete the following tasks to set up license sharing for your organization: + Give AWS Marketplace permission to manage licenses on your behalf so that it can create the associated license grants when you purchase or share your licenses. For more information, see [Service-linked roles for AWS Marketplace](https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-using-service-linked-roles.html) in the *AWS Marketplace Buyer Guide*. + Set up AWS License Manager for first use. For more information, see [ Getting started with AWS License Manager](https://docs.aws.amazon.com/license-manager/latest/userguide/getting-started.html) in the *AWS License Manager User Guide*. ## Step 1: View your licenses The following topics outline the process of viewing your licenses. **Topics** + [ ### Viewing all licenses ](#view-all-licenses) + [ ### Viewing a single license ](#view-single-licenses) ### Viewing all licenses You can use the AWS License Manager console to view all of the licenses for AWS Data Exchange products that you purchased. **To view all licenses for your subscribed products** 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/marketplace/). 1. Open the [AWS License Manager console](https://console.aws.amazon.com/license-manager). 1. In the left navigation pane, choose **Granted licenses**. 1. View all the licenses for your subscribed products. ### Viewing a single license You can use the AWS License Manager console to view a single license for an AWS Data Exchange data grant. **To view a license for a single subscription** 1. Sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. Under **My subscriptions**, choose ****Subscriptions****. 1. Choose a subscription. 1. On the next page, choose **View license** or **Distribute with License Manager**. What you see varies, depending on the data grant's distribution permissions. 1. View the details on the **License detail** page. ## Step 2: Share your licenses You can manage and share your licenses with other accounts in your organization by using AWS License Manager. For more details about using License Manager with AWS managed licenses, see [ Granted licenses](https://docs.aws.amazon.com/license-manager/latest/userguide/granted-licenses.html) and [Seller issued licenses](https://docs.aws.amazon.com/license-manager/latest/userguide/granted-licenses.html) in the *AWS License Manager User Guide*. # Accepting Bring Your Own Subscription (BYOS) offers in AWS Data Exchange BYOS offers As a subscriber, you might want to migrate your existing data subscriptions to AWS Data Exchange. Bring your own subscription (BYOS) functionality allows you to migrate and fulfill existing subscriptions with participating data providers at no additional cost. With BYOS offers, any billing relationship between providers and subscribers continues. BYOS offers are not subject to fulfillment fees. As a subscriber, you receive an AWS Marketplace invoice for the subscription with no charge for a fulfilment fee. Because the subscription lifecycle starts outside of AWS Data Exchange, the workflow for migrating the existing subscriptions to AWS Data Exchange using BYOS requires collaboration between the provider and subscriber. **Important** With BYOS offers, you're migrating a subscription that predates the availability of this product on AWS. AWS might verify your BYOS offer with the existing subscription agreement. If AWS cannot verify your BYOS offer, the offer and entitlements can be revoked without notice. Before creating or accepting a BYOS offer on AWS Data Exchange, the provider and subscriber should perform the following steps together: **Prerequisites** 1. The provider and the subscriber contact each other about implementing a BYOS AWS Data Exchange solution. 1. The subscriber provides the AWS account ID that they want to use to subscribe to data products on AWS Data Exchange. The subscriber accepts the BYOS offer as follows. **To accept a BYOS offer** 1. Sign in to the AWS Data Exchange console. 1. In the left navigation pane, from **Discover data products**, choose **My product offers**. 1. Select the offer to which you would like to subscribe. You can use the filter at the top of the page to choose between **All products**, **Private products**, and **Public products**. 1. Choose **Continue to subscribe**. 1. Review the terms of the offer, the data subscription agreement, and the included data sets. 1. If you accept the terms of the offer, review and accept the acknowledgement, and then choose **Subscribe**. # Accepting private products and offers in AWS Data Exchange Private products and offers ADX Data providers can provide a product to you in AWS Data Exchange that isn't available to the general public, or they can offer their product at terms that are different from the publicly available offer terms. A private offer can be different from the public offer in any dimension, including price, duration, payment schedule, data subscription agreement, or refund policy. **Note** Unlike Bring Your Own Subscription (BYOS) offers, private offers are not required to be based on an existing subscription that predates the product's availability on AWS Data Exchange. The provider must create a custom offer for your AWS account ID to target the offer to you. If a private offer hasn't been extended to you, you can request one by contacting a provider using the contact information on the details page of the public offer. As a subscriber, you can accept a private offer as follows. **To accept a private offer** 1. Sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, from **Discover data products**, choose **My product offers**. 1. Find the product offer you are looking for in the list. You can filter at the top of the page to choose between **All products**, **Private products**, or **Public products**. 1. Select the offer to which you want to subscribe. 1. Under **Custom offers**, view the **API metered costs** (if included). 1. (Optional) In the **Metered cost calculator**, choose **Select metered cost** and enter the number of units to display an example of the cost. 1. Choose **Continue to subscribe**. 1. Review the terms of the offer, the payment schedule, the data subscription agreement, and the included data sets. **Note** To accept a private offer with a multiple payment schedule, you must be on invoice billing terms. You can [create a support ticket](https://console.aws.amazon.com/support/home#/case/create%3FissueType=customer-service) if you want to switch to invoice billing terms. Private offers with a multiple payment schedule are not eligible for automatic renewal. 1. If you accept the terms of the offer, review and accept the acknowledgement, and then choose **Subscribe**. **Note** Your account is automatically invoiced according to the dates specified in the payment schedule. # Managing AWS Data Exchange subscriptions Managing subscriptions The following topics describe how to manage your subscriptions in AWS Data Exchange. **Topics** + [Viewing your subscriptions](view-subscriptions.md) + [Turning subscription auto-renewal on or off](turn-on-off-auto-renewal.md) + [Unsubscribing from a product](unsubscribe-product.md) # Viewing your AWS Data Exchange subscriptions Viewing your subscriptions View your subscriptions through the AWS Data Exchange console. **To view your subscriptions** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, choose **Subscriptions**. 1. View the list of your subscriptions. # Turning subscription auto-renewal on or off in AWS Data Exchange Turning subscription auto-renewal on or off Manage your subscription auto-renewals through the AWS Data Exchange console. **To turn subscription auto-renewal on or off** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, choose **Subscriptions**, and then choose your product. 1. On the product detail page, under **Renewal terms**, turn on **Auto-renewal** or turn off **Auto-renewal**. A success message appears, confirming your updated renewal settings. # Unsubscribing from an AWS Data Exchange product Unsubscribing from a product Use the AWS Data Exchange console to unsubscribe from a data product. **Note** If you require immediate removal of a subscription, contact AWS Data Exchange Customer Support by using the [AWS Support Center Console](https://console.aws.amazon.com/support/home#/case/create%3FissueType=customer-service). **Warning** After your subscription expires, you immediately lose access to all data sets and APIs associated with the product. Make sure you export or back up any critical data before your subscription ends. **To unsubscribe from a product** 1. Sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. In the left navigation pane, under **My subscriptions**, choose **Subscriptions**. 1. Select the subscription from which you want to unsubscribe. 1. Under **Renewal terms**, turn off the **Auto-renewal enabled** option. 1. Do not export any more data, and let the subscription run its course. **Note** For paid products, consult the provider's refund policy. Contact the provider for any exceptions. # Products for learning about interacting with AWS Data Exchange Products for learning about interacting with AWS Data Exchange AWS Data Exchange offers the following products that help subscribers understand how to subscribe to and interact with an AWS Data Exchange data product. **Topics** + [AWS Data Exchange Heartbeat](heartbeat.md) + [AWS Data Exchange for APIs](subscriber-tutorial-api-product.md) + [Worldwide Event Attendance](subscriber-tutorial-RS-product.md) + [AWS Data Exchange for AWS Lake Formation (Preview)](subscriber-tutorial-LF-product.md) + [AWS Data Exchange for Amazon S3](subscriber-tutorial-s3-product.md) + [AWS Data Exchange Provider-Generated Notifications](subscriber-tutorial-provider-notifications.md) # AWS Data Exchange Heartbeat AWS Data Exchange Heartbeat AWS Data Exchange Heartbeat (Test product) is a free product that subscribers can use to understand how to interact with an AWS Data Exchange product subscription. You can use it for testing purposes and to get familiar with the AWS Data Exchange API and concepts. AWS Data Exchange Heartbeat contains a single data set named **Heartbeat**. Approximately every 15 minutes, a new revision is published to this data set. ## Example content of a revision Each new revision contains two assets: + Epoch asset + Manifest asset ## Epoch asset Each AWS Data Exchange Heartbeat revision contains a JSON file Amazon Simple Storage Service (Amazon S3) object that contains a single array. The array's name is `TimestampsSinceLastRevision`, and its value is a list of each UNIX Epoch second that has elapsed since the last revision. The name of the asset is in the form `Epoch{start}-{end}.json` where `{start}` and `{end}` represent the Epoch seconds corresponding to the period of time covered by the revision. ## Manifest asset Each AWS Data Exchange Heartbeat revision contains a JSON file S3 object that contains metadata about the revision and the schema of the Epoch asset JSON file. The name of the asset is in the form `Manifest{start}-{end}.json` where `{start}` and `{end}` represent the Epoch seconds corresponding to the period of time covered by the revision. The following example shows the content of a manifest file. ``` { "manifestSchemaVersion":"1.0", "schema":"{ \"type\":\"object\", \"properties\":{ \"TimestampsSinceLastRevision\":{ \"type\":\"array\", \"description\":\"List of epoch timestamps in seconds.\", \"items\":{ \"type\":\"number\", \"description\":\"Epoch timestamp in seconds.\" } } } }", "startTimestamp":1554898111, "endTimestamp":1554905311, "numberOfTimestamps":7201 } ``` The following topic describes how to subscribe to AWS Data Exchange Heartbeat on AWS Data Exchange. **Topics** + [ ## Example content of a revision ](#revision-example) + [ ## Epoch asset ](#epoch-asset) + [ ## Manifest asset ](#manifest-asset) + [ # Subscribing to AWS Data Exchange Heartbeat on AWS Data Exchange ](how-to-subscribe.md) # Subscribing to AWS Data Exchange Heartbeat on AWS Data Exchange Subscribing to AWS Data Exchange Heartbeat The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to AWS Data Exchange Heartbeat. **To find and subscribe to AWS Data Exchange Heartbeat** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **AWS Data Exchange Heartbeat** and press **Enter**. 1. Choose the **AWS Data Exchange Heartbeat** product to view its details page. 1. (Optional) To view the data dictionary, scroll down to the product **Overview** section to see the data dictionary under **Data dictionaries**. 1. (Optional) To download the data dictionary, choose the **Data dictionary and samples** tab, choose the option button next to **Data dictionary**, and then choose **Download**. 1. (Optional) To download the sample, choose the option button next to the sample name (**Heartbeat manifest sample.json**), and then choose **Download**. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the data subscription agreement. **Note** AWS Data Exchange Heartbeat doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** AWS Data Exchange Heartbeat is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. 1. On the **Set up your first export** page, select the check boxes for the data sets containing the revisions you would like to export. Selecting a data set will prepare its most recently published revision to be exported. 1. Choose an Amazon S3 bucket location or configure an Amazon S3 key naming pattern. This will determine where your revisions will be exported. For more information about using key patterns, see [Key patterns when exporting asset revisions from AWS Data Exchange](revision-export-keypatterns.md). 1. Choose **Export** to export the data to Amazon S3, or choose **Skip** if you'd rather wait and export or download later. **Note** It can take a few minutes for your subscription to become active after you choose **Subscribe**. If you choose **Export** before the subscription is active, you are prompted to wait until it is complete. After your subscription is active, your export will begin. Navigating away from this page prior to your subscription becoming active will not prevent the subscription from processing. It will prevent your data export from occurring. # AWS Data Exchange for APIs (Test Product) AWS Data Exchange for APIs AWS Data Exchange for APIs (Test Product) is a free product that is made available to subscribers to understand how to interact with an AWS Data Exchange product containing API data sets. You can use this product for testing purposes and to learn how to make API calls to providers in order to retrieve API-based data. AWS Data Exchange for APIs (Test Product) contains an API data set named **AWS Data Exchange for APIs (Test Product)** that is in the US East (N. Virginia) Region. **Topics** + [Subscribing to AWS Data Exchange for APIs (Test Product)](subscribe-to-API-test-product.md) + [Viewing the API](tutorial-view-api.md) + [Downloading the API specification](tutorial-download-api-spec.md) + [Making an API call](tutorial-make-api-call-console.md) # Subscribing to AWS Data Exchange for APIs (Test Product) on AWS Data Exchange Subscribing to AWS Data Exchange for APIs (Test Product) The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to AWS Data Exchange for APIs (Test Product). **To find and subscribe to AWS Data Exchange for APIs (Test Product)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **AWS Data Exchange for APIs** and press **Enter**. 1. Choose the **AWS Data Exchange for APIs (Test Product)** and view its details page. 1. (Optional) To download the sample, choose the **Data dictionary and samples** tab, choose the option button next to the sample name (**ADX for APIs sample.json**), and then choose **Download**. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose the **Product offer**. **Note** AWS Data Exchange for APIs (Test Product) is a free product. 1. Review the **Subscription terms**, **Data sets**, and **Support information**. 1. Choose whether to enable **Offer auto-renewal** for the subscription. **Note** AWS Data Exchange for APIs (Test Product) doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Choose **Subscribe**. It can take a few minutes for your subscription to become active after you choose **Subscribe**. Navigating away from this page before your subscription becomes active will not prevent the subscription from processing. # Viewing the AWS Data Exchange API Viewing the API You can view the API with AWS Data Exchange for APIs (Test Product) using the following steps. **To view the API** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose the product titled ****AWS Data Exchange** for APIs (Test Product)** and then choose the **AWS Data Exchange for APIs** data set. 1. Under the **Revisions** tab, choose a revision. 1. Under **API assets**, choose the API. 1. View the **Asset overview**. 1. Follow the guidance in the **Integration notes** to call the API. # Downloading the AWS Data Exchange API specification Downloading the API specification You can download the API specification with AWS Data Exchange for APIs (Test Product) using the following steps. **To download the API specification** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose the product titled ****AWS Data Exchange** for APIs (Test Product)** and then choose the **AWS Data Exchange for APIs** data set. 1. Under the **Revisions** tab, choose a revision. 1. Under **API assets**, choose the API. 1. On the **OpenAPI 3.0 specification**, choose **Download API specification**. The specification is downloaded onto your local computer. You can then export the asset to a third-party tool for SDK generation. # Making an AWS Data Exchange API call Making an API call You can call a single endpoint in the AWS Data Exchange console. **To make an API call from the console** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **My subscriptions**, choose **Entitled data**. 1. Choose the product titled ****AWS Data Exchange** for APIs (Test Product)** and then choose the **AWS Data Exchange for APIs** data set. 1. Under the **Revisions** tab, choose the revision. 1. Under **API assets**, choose the API. You will see the sample **Code structure** and **OpenApi 3.0 specification** to structure your API request, which you can use in the AWS Command Line Interface to call the API. 1. Under **Integration notes**, choose **Copy** to copy the **Code structure** and then paste it into the AWS CLI. 1. Replace the sample values with the parameter key-value pairs you need using the information in the specification documentation. Following is a sample API request for **AWS Data Exchange for APIs (Test Product)**. ``` aws dataexchange send-api-asset \ --data-set-id 8d494cba5e4720e5f6072e280daf70a8 \ --revision-id b655d5be3da04fcbdca21a5a2932d789 \ --asset-id 8550cfab16b444a794402f2c3f11eae1 \ --method POST \ --path "someresource" \ --query-string-parameters 'param1=value1,param2=value2' \ --request-headers 'header=header_value' \ --body "{\"body_param\":\"body_param_value\"}" ``` # Worldwide Event Attendance (Test Product) on AWS Data Exchange Worldwide Event Attendance Worldwide Event Attendance (Test Product) is a free product that helps subscribers understand how to subscribe to and interact with an AWS Data Exchange product containing Amazon Redshift data sets. You can use this product for testing purposes and to learn how to query, analyze, and build applications within minutes. Worldwide Event Attendance (Test Product) contains an Amazon Redshift data set named **Worldwide Event Data (Test Data)** that is in the US East (N. Virginia) AWS Region. You use the AWS Data Exchange console to find and subscribe to Worldwide Event Attendance (Test Product). Then, you can use either the Amazon Redshift console or SQL commands to query the datashare. **Topics** + [Subscribing to Worldwide Event Attendance (Test Product)](subscribe-to-test-product.md) + [ # Querying Worldwide Event Attendance (Test Product) data with an Amazon Redshift cluster (console) ](query-RS-data-console.md) + [ # Querying Worldwide Event Attendance (Test Product) data on Amazon Redshift (SQL) ](query-RS-data-SQL.md) # Subscribing to Worldwide Event Attendance (Test Product) on AWS Data Exchange Subscribing to Worldwide Event Attendance (Test Product) The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to Worldwide Event Attendance (Test Product). **To find and subscribe to Worldwide Event Attendance (Test Product)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **Worldwide Event Attendance (Test Product)** and press **Enter**. 1. Choose the **Worldwide Event Attendance (Test Product)** to view its details page. 1. (Optional) To view the data dictionary, scroll down to the product **Overview** section to see the data dictionary under **Data dictionaries**. 1. (Optional) To download the data dictionary, choose the **Data dictionary and samples** tab, choose the option button next to **Data dictionary**, and then choose **Download**. 1. (Optional) To preview the sample, choose the option button next to the sample name (**Worldwide Event Attendance Sample.csv**), and then choose **Preview sample (CSV only)**. 1. (Optional) To download the sample, choose the option button next to the sample name (**Worldwide Event Attendance Sample.csv**), and then choose **Download**. If you are previewing the sample, you can also choose **Download** in the sample preview dialog box. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the data subscription agreement. **Note** Worldwide Event Attendance (Test Product) doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** Worldwide Event Attendance (Test Product) is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. It can take a few minutes for your subscription to become active after you choose **Subscribe**. Navigating away from this page prior to your subscription becoming active will not prevent the subscription from processing. # Querying Worldwide Event Attendance (Test Product) data with an Amazon Redshift cluster (console) The following procedure shows how to set up and query the datashare using the Amazon Redshift console. **To query Worldwide Event Attendance (Test Product) data on Amazon Redshift (console)** 1. Open and sign in to the Amazon Redshift console. 1. Choose **Clusters**, and choose your existing RA3 cluster. 1. Choose the **Datashares** tab. 1. Select the datashare you want to create the database from. 1. Under **Subscriptions to AWS Data Exchange datashares**, choose **Create database from datashare**. 1. In **Create database from datashare**, enter the **Database name** for your new database, and then choose **Create**. 1. Choose the **Marketplace** icon on the navigation pane, and open the **Query editor**. 1. Under **Resources**, select a database and a schema. 1. Run the following SQL query. `select * from database.schema.table` # Querying Worldwide Event Attendance (Test Product) data on Amazon Redshift (SQL) The following procedure shows how to set up and query the datashare using the SQL commands. **To query Worldwide Event Attendance (Test Product) data on Amazon Redshift (SQL)** 1. To find the datashare, run the following command. `SHOW DATASHARES [ LIKE 'namepattern' ]` This command lists all datashares, including the one from Worldwide Event Attendance (Test Product), in addition to the provider's `account_id` and `namespace`. For more information, see [Show Datashares](https://docs.aws.amazon.com/redshift/latest/dg/r_SHOW_DATASHARES.html) in the *Amazon Redshift Database Developer Guide*. 1. Run the following command to create a database from the datashare. `CREATE DATABASE database_name` `FROM DATASHARE datashare_name OF ACCOUNT account_id NAMESPACE namespace_guid` For more information, see [Create Database](https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_DATABASE.html) in the *Amazon Redshift Database Developer Guide*. 1. Run the following SQL query. `select * from database.schema.table` # AWS Data Exchange for AWS Lake Formation (Test Product) (Preview) AWS Data Exchange for AWS Lake Formation (Preview) AWS Data Exchange for AWS Lake Formation (Test Product) is a free product that helps subscribers understand how to subscribe to and interact with an AWS Data Exchange product containing AWS Lake Formation data sets. You can use this product for testing purposes and learn how to query, analyze, and share data internally within minutes. **Topics** + [ # Subscribing to AWS Data Exchange for AWS Lake Formation (Test Product) on AWS Data Exchange (Preview) ](subscribe-to-LF-test-product.md) + [ # Setting up and querying AWS Data Exchange for Lake Formation (Test Product) (Preview) ](query-LF-data-console.md) # Subscribing to AWS Data Exchange for AWS Lake Formation (Test Product) on AWS Data Exchange (Preview) The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to AWS Data Exchange for Lake Formation (Test Product). (Preview) **To subscribe to AWS Data Exchange for Lake Formation (Preview)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **AWS Data Exchange for Lake Formation (Test product)** and choose **Enter**. 1. Choose **AWS Data Exchange for Lake Formation (Test Product)** to view its details page. 1. (Optional) To view the data dictionary, scroll down to the product **Overview** section to see the data dictionary under **Data dictionaries**. 1. (Optional) To download the data dictionary, choose the **Data dictionary and samples** tab, choose the option button next to **Data dictionary**, and then choose **Download**. 1. (Optional) To preview the sample, choose the option button next to the sample name (**AWS Data Exchange for Lake Formation (Test Product)**), and then choose **Preview sample (CSV only)**. 1. (Optional) To download the sample, choose the option button next to the sample name (**AWS Data Exchange for Lake Formation (Test Product)**), and then choose **Download**. 1. If you are previewing the sample, you can also choose **Download** in the sample preview dialog box. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination and review the offer details, including the data subscription agreement. **Note** **AWS Data Exchange for Lake Formation (Test Product)** doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** **AWS Data Exchange for Lake Formation (Test product)** is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. It can take a few minutes for your subscription to become active after you choose **Subscribe**. Navigating away from this page prior to your subscription becoming active will not prevent the subscription from processing. # Setting up and querying AWS Data Exchange for Lake Formation (Test Product) (Preview) The following procedure shows how to set up and query a Lake Formation data permission set (Preview) using the AWS Management Console. **To enable querying on the AWS Data Exchange for Lake Formation (Test Product) data set (Preview)** 1. Open and sign in to the AWS Data Exchange console. 1. From the left navigation pane under **My subscriptions**, choose **Entitled data**. 1. From the list of **Products**, choose **AWS Data Exchange for Lake Formation (Test Product) (Preview)**. 1. Choose **Accept** to accept the AWS RAM share. **Note** You must accept the AWS RAM share within 12 hours of subscribing to the data product. If your AWS RAM share invitation expires, select **Request invitation** and allow several business days for a new share to be sent. You only need to accept the AWS RAM share once for each provider that you license Lake Formation data sets from. 1. Open the [Lake Formation console](https://console.aws.amazon.com/lakeformation/). 1. Sign in as a principal who has the Lake Formation `CREATE_TABLE` or `CREATE_DATABASE` permission, as well as the `glue:CreateTable` or `glue:CreateDatabase` AWS Identity and Access Management (IAM) permission. 1. In the navigation pane, choose **Tables**, and then choose **Create table**. 1. On the **Create table** page, choose **Resource Link**, and then provide the following information: + **Resource link name ** – Enter a name that adheres to the same rules as a table name. The name can be the same as the name of the target shared table. + **Database** – The database in the local Data Catalog must contain the resource link. + **Shared table** – Select one of the tables shared through AWS Data Exchange for Lake Formation (Test product). All of the table names shared through that product begin with `adxlf_test`, or enter a local (owned) or shared table name. The list contains all of the tables shared with your account. The database and owner account ID are listed with each table. If you don’t see a table that you know was shared with your account, check the following: + If you aren’t a data lake administrator, confirm with your administrator that you were granted Lake Formation permissions on the table. + If you’re a data lake administrator and your account is not the same AWS organization as the granting account, confirm that you’ve accepted the AWS Resource Access Manager (AWS RAM) resource share invitation for the table. For more information, see [Accepting a resource share invitation from AWS RAM](https://docs.aws.amazon.com/lake-formation/latest/dg/accepting-ram-invite.html). + **Shared table's database** – If you selected a shared table from the list, this field is populated with the shared table's database in the external account. If you didn't select a shared table, enter a local database for a resource link to a local table, or the shared table's database in the external account. + **Shared table owner** – If you selected a shared table from the list, this field is populated with the shared table's owner account ID. If you didn't select a shared table, enter your AWS account ID for a resource link to a local table, or the ID of the AWS account that shared the table. **To query the AWS Data Exchange for Lake Formation (Test Product) data set (Preview) with Amazon Athena (Console)** 1. Sign in to the [Amazon Athena console](https://console.aws.amazon.com/athena) with a role that has permissions for Amazon Athena. 1. In the Amazon Athena query editor, choose the resource link that you created previously. 1. Choose the additional menu options icon next to `source_data` and choose **Preview table**. 1. Choose **Run query**. **To allow querying on the AWS Data Exchange for Lake Formation (Test Product) data set (Preview) (AWS CLI)** 1. To retrieve a list of all invitations available to your AWS account, enter the following command. The AWS CLI `query` parameter lets you restrict the output to only those invitations shared from AWS Data Exchange. `$ AWS ram get-resource-share-invitations` `--region us-east-1` `--query 'resourceShareInvitations[?` `senderAccountId==147854383891]'` 1. Find the invitations for the AWS Data Exchange for Lake Formation data set. Then, note the `resourceShareInvitationArn` in the output to use in the following command to accept the invitation. `$ AWS ram accept-resource-share-invitation --region us-east-1 --resource-share-invitation-arn [resourceShareInvitationArn]` If successful, the response shows that the status has changed from **PENDING** to **ACCEPTED**. 1. Create a resource link to one of the tables shared through the AWS Data Exchange for Lake Formation data set with the following command: `aws glue create-table --database-name [local_database_to_store_resource_link] --table-input '{"Name":"resource_link_name","TargetTable":{"CatalogId":"[account_owning_original_table]","DatabaseName":"[shared_db_in_provider_account]","Name":"[shared_table_in_provider_account]"}}'.` **Note** To create resource links, use the Lake Formation `CREATE_TABLE` or `CREATE_DATABASE` permission, as well as the `glue:CreateTable` or `glue:CreateDatabase` IAM permission. # AWS Data Exchange for Amazon S3 (Test Product) AWS Data Exchange for Amazon S3 AWS Data Exchange for Amazon S3 (Test Product) is a product that helps subscribers understand how to subscribe to and interact with an AWS Data Exchange product. In this tutorial, the product contains Amazon Simple Storage Service (Amazon S3) data access data sets.You can use this product for testing purposes and to learn how to query and analyze data directly from a data provider's Amazon S3 bucket. You can run queries to analyze the data in-place without setting up your own Amazon S3 buckets, copying data files into Amazon S3 buckets, or paying associated storage fees. **Topics** + [Subscribing to AWS Data Exchange for Amazon S3 (Test Product)](subscribe-to-s3-test-product.md) + [ # Setting up and querying AWS Data Exchange for Amazon S3 (Test Product) ](query-s3-data-access-cli.md) # Subscribing to AWS Data Exchange for Amazon S3 (Test Product) Subscribing to AWS Data Exchange for Amazon S3 (Test Product) The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to AWS Data Exchange for Amazon S3 (Test Product). **To find and subscribe to AWS Data Exchange for Amazon S3 (Test Product)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **AWS Data Exchange for Amazon S3 (Test Product)** and choose **Enter**. 1. Choose **AWS Data Exchange for Amazon S3 (Test Product)** to view its details page. 1. (Optional) To view the data dictionary, scroll down to the product **Overview** section to see the data dictionary under **Data dictionaries**. 1. (Optional) To download the data dictionary, choose the **Data dictionary and samples** tab, choose the option button next to **Data dictionary**, and then choose **Download**. 1. (Optional) To preview the sample, choose the option button next to the sample name of **Blockchain Transactions (Test Data)**, and then choose **Preview sample (CSV only)**. 1. (Optional) To download the sample, choose the option button next to the sample name of **Blockchain Transactions (Test Data)**, and then choose **Download**. If you're previewing the sample, you can also choose **Download** in the sample preview dialog box. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination. Choose whether to enable auto-renewal for the subscription, and review the offer details, including the data subscription agreement. **Note** **AWS Data Exchange for Amazon S3 (Test Product)** doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** **AWS Data Exchange for Amazon S3 (Test Product)** is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. It can take a few minutes for your subscription to become active after you choose **Subscribe**. Navigating away from this page before your subscription becomes active will not prevent the subscription from processing. # Setting up and querying AWS Data Exchange for Amazon S3 (Test Product) The following procedure shows how to set up and query an Amazon S3 data access data set using the AWS Command Line Interface (AWS CLI). Before querying, you must obtain the appropriate AWS Identity and Access Management (IAM) permissions to attach policies to your user. To access data in a provider's bucket directly through the Amazon S3 delivery method, embed the following JSON policy to the user or role. **To set up AWS Data Exchange for Amazon S3 (Test Product)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane under **My subscriptions**, choose **Entitled data** in the AWS Region that hosts the data set. For the purposes of this tutorial, the Region is **us-east-1**. 1. From the list of **Products**, choose **AWS Data Exchange for Amazon S3 (Test Product)** and then choose the **Blockchain Transactions (Test Data)** data set. 1. Choose **Verify IAM permissions**. **Note** If you don't have the correct permissions, you'll receive a notification detailing how to create and attach the IAM policy to your user or role. In the following example, replace each *user input placeholder* with your own information. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": [ "arn:aws:s3:us-east-1:111122223333:accesspoint/my-access-point", "arn:aws:s3:::aws-data-exchange-s3-data-access-btc-demo-us-east-1" ] }, { "Effect": "Allow", "Action": "s3:GetObject", "Resource": [ "arn:aws:s3:us-east-1:111122223333:accesspoint/my-access-point/object/*", "arn:aws:s3:::aws-data-exchange-s3-data-access-btc-demo-us-east-1/*" ] } ] } ``` ------ **To allow querying on the AWS Data Exchange for Amazon S3 (Test Product) data access data set using the AWS CLI** 1. Open **AWS CloudShell** in **us-east-1**. 1. Choose the copy button next to the access point alias to copy and paste the code inside. After the command is added inAWS CloudShell with the correct access point alias, you can see the list of Amazon S3 objects included in this product. **Important** When a provider has enabled Requester Pays, the subscriber pays for the data transfer and the request. The provider pays for the data storage. For more information, see [Using Requester Pays buckets for storage transfers and usage](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html) in the *Amazon Simple Storage Service User Guide*. 1. (Optional) You can also copy an object to your local system using the following command. `aws s3api get-object --bucket --key 'v1.0/btc/transactions/date=2022-11-27/part-00000-03a88dba-27dd-4f59-a890-70a3d2c7ad26-c000.snappy.parquet' AWS_btc.snappy.parquet --request-payer requester ` # AWS Data Exchange Provider-Generated Notifications (Test Product) AWS Data Exchange Provider-Generated Notifications AWS Data Exchange Provider-Generated Notifications (Test Product) is a free product that helps subscribers understand how to subscribe to and interact with an AWS Data Exchange product using provider-generated notifications. Providers use this feature to notify you of important events related to their data sets. You'll receive these events in a consistent, structured format using Amazon EventBridge, that you can use to build automated workflows. Provider-generated notifications also supports the delivery of human-readable notification to emails and chat programs using [AWS User Notifications](https://docs.aws.amazon.com/notifications/latest/userguide/what-is-service.html). **Topics** + [Subscribing to AWS Data Exchange for Provider-Generated Notifications (Test Product)](subscribe-to-provider-generated-notifications-test-product.md) + [ # Configuring AWS Data Exchange provider-generated notifications using Amazon EventBridge ](configuring-provider-generated-notifications-using-amazon-eventbridge.md) # Subscribing to AWS Data Exchange for Provider-Generated Notifications (Test Product) Subscribing to AWS Data Exchange for Provider-Generated Notifications (Test Product) The following procedure shows how to subscribe to AWS Data Exchange Provider-Generated Notifications (Test Product). **To find and subscribe to AWS Data Exchange for Provider-Generated Notifications (Test Product)** 1. Open and sign in to the [AWS Data Exchange console](https://console.aws.amazon.com/dataexchange). 1. From the left navigation pane, under **Discover data products**, choose **Browse catalog**. 1. From the search bar, enter **AWS Data Exchange Provider-Generated Notifications (Test Product)** and choose **Enter**. 1. Choose **AWS Data Exchange for Provider-Generated Notifications (Test Product)** to view its details page. 1. In the top right corner, choose **Continue to subscribe**. 1. Choose your preferred price and duration combination. Choose whether to enable auto-renewal for the subscription, and review the offer details, including the data subscription agreement. **Note** **AWS Data Exchange for Provider-Generated Notifications (Test Product)** doesn't require subscription verification, but some products do. For more information, see [Subscription verification for subscribers in AWS Data Exchange](subscription-verification-sub.md). 1. Review the pricing information, choose the pricing offer, and then choose **Subscribe**. **Note** **AWS Data Exchange for Provider-Generated Notifications (Test Product)** is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe. It can take a few minutes for your subscription to become active after you choose **Subscribe**. Navigating away from this page before your subscription becomes active will not prevent the subscription from processing. # Configuring AWS Data Exchange provider-generated notifications using Amazon EventBridge AWS Data Exchange delivers provider-generated notifications using Amazon EventBridge. Your role must be able to create Amazon EventBridge rules, a target, and must be able to subscribe to a data product. AWS Data Exchange events are published to your default Amazon EventBridge event bus in the same AWS Region as where your data set is located. Use the following steps to create an Amazon EventBridge rule for provider-generated notifications: **To create an Amazon EventBridge rule for provider-generated notifications** 1. Create a target for the Amazon EventBridge rule. For a simple Lambda function in Python do the following: 1. Navigate to the AWS Lambda console. 1. Choose **Create function** and select **Author from scratch**. 1. Provide a function name and select **Python 3.10** as the **runtime**. Choose **Create function**. 1. Enter the following code for **lambda\$1function.py**: ``` import json def lambda_handler(event, context): print(" ".join(["Event of type", event["detail-type"], "received!"])) print(" ".join(["Details", json.dumps(event["detail"])])) return {"statusCode": 200, "body": json.dumps("Hello from Lambda!") } ``` 1. Navigate to the EventBridge console. 1. Navigate to the **Rules** and select the default event bus. 1. Choose **Create rule** and provide the **Name** and optional **Description**. Make sure the **Rule** type is **Rule with an event pattern**. 1. Choose **Next**. 1. Make sure the **Event source** is **AWS events** or **EventBridge partner events**. Under **Creation method**, select **Custom pattern** (JSON editor). Under **Event pattern**, enter the following JSON: ``` { "source": ["aws.dataexchange"], "detail-type": ["Data Set Update Delayed", "Data Updated in Data Set", "Deprecation Planned for Data Set", "Schema Change Planned for Data Set"] } ``` 1. Choose **Next**. 1. For **Target 1**, select **AWS service** and choose **Lambda function**. 1. For the **function**, select the function created in Step 1. Complete the creation of the rule. This Lambda function will be triggered any time a provider-generated notification is delivered. From the **Monitor** tab in the Lambda console, you can view recent invocations of the function.