# Refreshing data in Amazon Quick Sight Refreshing data When refreshing data, Amazon Quick Sight handles datasets differently depending on the connection properties and the storage location of the data. If Quick Sight connects to the data store by using a direct query, the data automatically refreshes when you open an associated dataset, analysis, or dashboard. Filter controls are refreshed automatically every 24 hours. To refresh SPICE datasets, Quick Sight must independently authenticate using stored credentials to connect to the data. Quick Sight can't refresh manually uploaded data—even from S3 buckets, even though it's stored in SPICE—because Quick Sight doesn't store its connection and location metadata. If you want to automatically refresh data that's stored in an S3 bucket, create a dataset by using the **S3** data source card. For files that you manually uploaded to SPICE, you refresh these manually by importing the file again. If you want to reuse the name of the original dataset for the new file, first rename or delete the original dataset. Then give the preferred name to the new dataset. Also, check that the field names are the same name and data type. Open your analysis, and replace the original dataset with the new dataset. For more information, see [Replacing datasets](replacing-data-sets.md). You can refresh your [SPICE](spice.md) datasets at any time. Refreshing imports the data into SPICE again, so the data includes any changes since the last import. For Amazon Quick Sight Standard Edition, you can do a full refresh of your SPICE data at any time. For Amazon Quick Sight Enterprise Edition, you can do a full refresh or an incremental refresh (SQL-based data sources only) at any time. **Note** If your dataset uses CustomSQL, refreshing incrementally might not benefit you. If the SQL query is complex, your database may not be able to optimize the filter with the look-back window. This can cause the query that pulls in the data to take longer than a full refresh. We recommend that you try reducing query execution time by refactoring the custom SQL. Note that results might vary depending on the type of optimization you make. You can refresh SPICE data by using any of the following approaches: + You can use the options on **Datasets** page. + You can refresh a dataset while editing a dataset. + You can schedule refreshes in the dataset settings. + You can use the [CreateIngestion](https://docs.aws.amazon.com/quicksight/latest/APIReference/API_CreateIngestion.html) API operation to refresh the data. When you create or edit a SPICE dataset, you can enable email notifications about data loading status. This option notifies the owners of the dataset if the data fails to load or refresh. To turn on notifications, select the **Email owners when a refresh fails** option that appears on the **Finish data set creation** screen. This option isn't available for datasets that you create by using **Upload a File** on the datasets page. In the following topics, you can find an explanation of different approaches to refreshing and working with SPICE data. **Topics** + [ # Importing data into SPICE ](spice.md) + [ # Refreshing SPICE data ](refreshing-imported-data.md) + [ # Using SPICE data in an analysis ](spice-in-an-analysis.md) + [ # View SPICE ingestion history ](view-history-of-spice-ingestion.md) + [ # Troubleshooting skipped row errors ](troubleshooting-skipped-rows.md) + [ # SPICE ingestion error codes ](errors-spice-ingestion.md) + [ # Updating files in a dataset ](updating-file-dataset.md) # Importing data into SPICE When you import data into a dataset rather than using a direct SQL query, it becomes *SPICE data* because of how it's stored. *SPICE (Super-fast, Parallel, In-memory Calculation Engine)* is the robust in-memory engine that Amazon Quick Sight uses. It's engineered to rapidly perform advanced calculations and serve data. In Enterprise edition, data stored in SPICE is encrypted at rest. When you create or edit a dataset, you choose to use either SPICE or a direct query, unless the dataset contains uploaded files. Importing (also called *ingesting*) your data into SPICE can save time and money: + Your analytical queries process faster. + You don't need to wait for a direct query to process. + Data stored in SPICE can be reused multiple times without incurring additional costs. If you use a data source that charges per query, you're charged for querying the data when you first create the dataset and later when you refresh the dataset. SPICE capacity is allocated separately for each AWS Region. Default SPICE capacity is automatically allocated to your home AWS Region. For each AWS account, SPICE capacity is shared by all the people using Quick Sight in a single AWS Region. The other AWS Regions have no SPICE capacity unless you choose to purchase some. Quick Sight administrators can view how much [SPICE](#spice) capacity you have in each AWS Region and how much of it is currently in use. A Quick Sight administrator can purchase more SPICE capacity or release unused SPICE capacity as needed. For more information, see [Configure SPICE memory capacity](managing-spice-capacity.md). **Topics** + [ ## Estimating the size of SPICE datasets ](#spice-capacity-formula) ## Estimating the size of SPICE datasets The size of a dataset in SPICE relative to your Quick account's SPICE capacity is called *logical size*. A dataset's logical size isn't the same as the size of the dataset's source file or table. The computation of a dataset's logical size occurs after all the data type transformations and calculated columns are defined during data preparation. These fields are materialized in SPICE in a way that enhances query performance. Any changes you make in an analysis have no effect on the logical size of the data in SPICE. Only changes that are saved in the dataset apply to SPICE capacity. The logical size of a SPICE dataset depends on the data types of the dataset fields and the number of rows in the dataset. The three types of SPICE data are decimals, dates, and strings. You can transform a field's data type during the data preparation phase to fit your data visualization needs. For example, the file you want to import might contain all strings (text). But for these to be used in a meaningful way in an analysis, you prepare the data by changing the data types to their proper form. Fields containing prices can be changed from strings to decimals, and fields containing dates can be changed from strings to dates. You can also create calculated fields and exclude fields that you don't need from the source table. When you are finished preparing your dataset and all transformations are complete, you can estimate the logical size of the final schema. **Note** Geospatial data types use metadata to interpret the physical data type. Latitude and longitude are numeric. All other geospatial categories are strings. In the formula below, decimals and dates are calculated as 8 bytes per cell with 4 extra bytes for auxillary. Strings are calculated based on the text's length in UTF-8 encoding plus 24 bytes for auxillary. String data types require more space because of the extra indexing required by SPICE to provide high query performance. ``` Logical dataset size in bytes = (Number of Numeric cells * (12 bytes per cell)) + (Number of Date cells * (12 bytes per cell)) + SUM ((24 bytes + UTF-8 encoded length) per Text cell) ``` The formula above should only be used to estimate the size of a single dataset in SPICE. The SPICE capacity usage is the total size of all datasets in an account in a specific region. Quick Sight does not recommend that you use this formula to estimate the total SPICE capacity that your Quick Sight account is using. # Refreshing SPICE data ## Refreshing a dataset Use the following procedure to refresh a [SPICE](spice.md) dataset based on an Amazon S3 or database data source in the **Data** tab. If there is a schema change in a database, Quick Sight will not be able to auto-detect it, resulting in an ingestion failure. Edit and save the dataset to update the schema and avoid ingestion failures. **To refresh SPICE data from the Data tab** 1. Select **Data** from the left navigation menu. In the **Datasets** tab, choose the dataset to open it. 1. On the dataset details page that opens, choose the **Refresh** tab and then choose **Refresh now**. 1. Keep the refresh type as **Full refresh**. 1. If you are refreshing an Amazon S3 dataset, choose one of the following options for **S3 Manifest**: + To use the same manifest file you last provided to Amazon Quick Sight, choose **Existing Manifest**. If you have changed the manifest file at the file location or URL that you last provided, the data returned reflects those changes. + To specify a new manifest file by uploading it from your local network, choose **Upload Manifest**, and then choose **Upload manifest file**. For **Open**, choose a file, and then choose **Open**. + To specify a new manifest file by providing a URL, enter the URL of the manifest in **Input manifest URL**. You can find the manifest file URL in the Amazon S3 console by opening the context (right-click) menu for the manifest file, choosing **Properties**, and looking at the **Link** box. 1. Choose **Refresh**. 1. If you are refreshing an Amazon S3 dataset, choose **OK**, then **OK** again. If you are refreshing a database dataset, choose **OK**. ## Incrementally refreshing a dataset | | | --- | |  Applies to: Enterprise Edition  | For SQL-based data sources, such as Amazon Redshift, Amazon Athena, PostgreSQL, or Snowflake, you can refresh your data incrementally within a look-back window of time. An *incremental refresh* queries only data defined by the dataset within a specified look-back window. It transfers all insertions, deletions, and modifications to the dataset, within that window's timeframe, from its source to the dataset. The data currently in SPICE that's within that window is deleted and replaced with the updates. With incremental refreshes, less data is queried and transferred for each refresh. For example, let's say you have a dataset with 180,000 records that contains data from January 1 to June 30. On July 1, you run an incremental refresh on the data with a look-back window of seven days. Quick Sight queries the database asking for all data since June 24 (7 days ago), which is 7,000 records. Quick Sight then deletes the data currently in SPICE from June 24 and after, and appends the newly queried data. The next day (July 2), Quick Sight does the same thing, but queries from June 25 (7,000 records again), and then deletes from the existing dataset from the same date. Rather than having to ingest 180,000 records every day, it only has to ingest 7,000 records. Use the following procedure to incrementally refresh a [SPICE](spice.md) dataset based on a SQL data source from the **Datasets** tab. **To incrementally refresh a SQL-based SPICE dataset** 1. Choose **Data** from the left navigation menu. On the **Datasets** tab, choose the dataset to open it. 1. On the dataset details page that opens, choose the **Refresh** tab and then choose **Refresh now**. 1. For **Refresh type**, choose **Incremental refresh**. 1. If this is your first incremental refresh on the dataset, choose **Configure**. 1. On the **Configure incremental refresh** page, do the following: 1. For **Date column**, choose a date column that you want to base the look-back window on. 1. For **Window size**, enter a number for **size**, and then choose an amount of time that you want to look back for changes. You can choose to refresh changes to the data that occurred within a specified number of hours, days, or weeks from now. For example, you can choose to refresh changes to the data that occurred within two weeks of the current date. 1. Choose **Submit**. ## Refreshing a dataset during data preparation Use the following procedure to refresh a [SPICE](spice.md) dataset based on an Amazon S3 or database data source during data preparation. **To refresh SPICE data during data preparation** 1. Choose **Data** from the left navigation menu. On the **Datasets** tab, choose the dataset, and then choose **Edit Data Set**. 1. On the dataset screen, choose **Refresh now**. 1. Keep the refresh type set to **Full refresh**. 1. (Optional) If you are refreshing an Amazon S3 dataset, choose one of the following options for **S3 Manifest**: + To use the same manifest file that you last provided to Amazon Quick Sight, choose **Existing Manifest**. If you have changed the manifest file at the file location or URL that you last provided, the data returned reflects those changes. + To specify a new manifest file by uploading it from your local network, choose **Upload Manifest**, and then choose **Upload manifest file**. For **Open**, choose a file, and then choose **Open**. + To specify a new manifest file by providing a URL, enter the URL of the manifest in **Input manifest URL**. You can find the manifest file URL in the Amazon S3 console by opening the context (right-click) menu for the manifest file, choosing **Properties**, and looking at the **Link** box. 1. Choose **Refresh**. 1. If you are refreshing an Amazon S3 dataset, choose **OK**, then **OK** again. If you are refreshing a database dataset, choose **OK**. ## Refreshing a dataset on a schedule Use the following procedure to schedule refreshing the data. If your dataset is based on a direct query and not stored in [SPICE](spice.md), you can refresh your data by opening the dataset. You can also refresh your data by refreshing the page in an analysis or dashboard. **To refresh [SPICE](spice.md) data on a schedule** 1. Choose **Data** from the left navigation menu. On the **Datasets** tab, choose the dataset to open it. 1. On the dataset details page that opens, choose the **Refresh** tab and then choose **Add new schedule**. 1. On the **Create a refresh schedule** screen, choose settings for your schedule: 1. For **Time zone**, choose the time zone that applies to the data refresh. 1. For **Starting time**, choose a date and time for the refresh to start. Use HH:MM and 24-hour format, for example 13:30. 1. For **Frequency**, choose one of the following: + For Standard or Enterprise editions, you can choose **Daily**, **Weekly**, or **Monthly**. + **Daily**: Repeats every day. + **Weekly**: Repeats on the same day of each week. + **Monthly**: Repeats on the same day number of each month. To refresh data on the 29th, 30th or 31st day of the month, choose **Last day of month** from the list. + For Enterprise edition only, you can choose **Hourly**. This setting refreshes your dataset every hour, beginning at the time that you choose. So, if you select 1:05 as the starting time, the data refreshes at five minutes after the hour, every hour. If you decide to use an hourly refresh, you can't also use additional refresh schedules. To create an hourly schedule, remove any other existing schedules for that dataset. Also, remove any existing hourly schedule before you create a daily, weekly, or monthly schedule. 1. Choose **Save**. Scheduled dataset ingestions take place within 10 minutes of the scheduled date and time. Using the Quick console, you can create five schedules for each dataset. When you have created five, the **Create** button is turned off. ## Incrementally refreshing a dataset on a schedule | | | --- | |  Applies to: Enterprise Edition  | For SQL-based data sources, such as Amazon Redshift, Athena, PostgreSQL, or Snowflake, you can schedule incremental refreshes. Use the following procedure to incrementally refresh a [SPICE](spice.md) dataset based on a SQL data source in the **Datasets** tab. **To set an incremental refresh schedule for a SQL-based SPICE dataset** 1. Choose **Data** from the left navigation menu. On the **Datasets** tab, choose the dataset to open it. 1. On the dataset details page that opens, choose the **Refresh** tab and then choose **Add new schedule**. 1. On the **Create a schedule** page, for **Refresh type**, choose **Incremental refresh**. 1. If this is your first incremental refresh for this dataset, choose **Configure**, and then do the following: 1. For **Date column**, choose a date column that you want to base the look-back window on. 1. For **Window size**, enter a number for **size**, and then choose an amount of time that you want to look back for changes. You can choose to refresh changes to the data that occurred within a specified number of hours, days, or weeks from now. For example, you can choose to refresh changes to the data that occurred within two weeks of the current date. 1. Choose **Submit**. 1. For **Time zone**, choose the time zone that applies to the data refresh. 1. For **Repeats**, choose one of the following: + You can choose **Every 15 minutes**, **Every 30 minutes**, **Hourly**, **Daily**, **Weekly**, or **Monthly**. + **Every 15 minutes**: Repeats every 15 minutes, beginning at the time you choose. So, if you select 1:05 as the starting time, the data refreshes at 1:20, then again at 1:35, and so on. + **Every 30 minutes**: Repeats every 30 minutes, beginning at the time you choose. So, if you select 1:05 as the starting time, the data refreshes at 1:35, then again at 2:05, and so on. + **Hourly**: Repeats every hour, beginning at the time you choose. So, if you select 1:05 as the starting time, the data refreshes at five minutes after the hour, every hour. + **Daily**: Repeats every day. + **Weekly**: Repeats on the same day of each week. + **Monthly**: Repeats on the same day number of each month. To refresh data on the 29th, 30th or 31st day of the month, choose **Last day of month** from the list. + If you decide to use refresh every 15 or 30 minutes, or hourly, you can't also use additional refresh schedules. To create a refresh schedule every 15 minutes, 30 minutes, or hourly, remove any other existing schedules for that dataset. Also, remove any existing minute or hourly schedule before you create a daily, weekly, or monthly schedule. 1. For **Starting**, choose a date for the refresh to start. 1. For **At**, specify the time that the refresh should start. Use HH:MM and 24-hour format, for example 13:30. Scheduled dataset ingestions take place within 10 minutes of the scheduled date and time. In some cases, something might go wrong with the incremental refresh dataset that makes you want to roll back your dataset. Or you might no longer want to refresh the dataset incrementally. If so, you can delete the scheduled refresh. To do so, choose the dataset on the **Datasets** page, choose **Schedule a refresh**, and then choose the x icon to the right of the scheduled refresh. Deleting an incremental refresh configuration starts a full refresh. As part of this full refresh, all the configurations prepared for incremental refreshes are removed. # Using SPICE data in an analysis When you use stored data to create an analysis, a data import indicator appears next to the dataset list at the top of the **Fields list** pane. When you first open the analysis and the dataset is importing, a spinner icon appears. After the SPICE import is complete, the indicator displays the percentage of rows that were successfully imported. A message also appears at the top of the visualization pane to provide counts of the rows imported and skipped. If any rows were skipped, you can choose **View summary** in this message bar to see details about why those rows failed to import. To edit the dataset and resolve the issues that led to skipped rows, choose **Edit data set**. For more information about common causes for skipped rows, see [Troubleshooting skipped row errors](troubleshooting-skipped-rows.md). If an import fails altogether, the data import indicator appears as an exclamation point icon, and an **Import failed** message is displayed. # View SPICE ingestion history You can view the ingestion history for SPICE datasets to find out, for example, when the latest ingestion started and what its status is. The SPICE ingestion history page includes the following information: + Date and time that the ingestion started (UTC) + Status of the ingestion + Amount of time that the ingestion took + The number of aggregated rows in the dataset. + The number of rows ingested during a refresh. + Rows skipped and rows ingested (imported) successfully + The job type for the refresh: scheduled, full refresh, and so on Use the following procedure to view a dataset's SPICE ingestion history. **To view a dataset's SPICE ingestion history** 1. From the homepage, choose **Data** at left. 1. On the **Datasets** tab, choose the dataset that you want to examine. 1. On the dataset details page that opens, choose the **Refresh** tab. SPICE ingestion history is shown at bottom. 1. (Optional) Choose a time frame to filter the entries from the last hour to the last 90 days. 1. (Optional) Choose a specific job status to filter the entries, for example **Running** or **Completed**. Otherwise, you can view all entries by choosing **All**. # Troubleshooting skipped row errors When you import data, Amazon Quick Sight previews a portion of your data. If it can't interpret a row for any reason, Quick Sight skips the row. In some cases, the import will fail. When this happens, Quick Sight returns an error message that explains the failure. Fortunately, there's a limited number of things that can go wrong. Some issues can be avoided by being aware of examples like the following: + Make sure that there is no inconsistency between the field data type and the field data, for example occasional string data in a field with a numeric data type. Here are a few examples that can be difficult to detect when scanning the contents of a table: + `''` – Using an empty string to indicate a missing value + `'NULL'` – Using the word "null" to indicate a missing value + `$1000` – Including a dollar sign in a currency value turns it into a string + `'O'Brien'` – Using punctuation to mark a string that itself contains the same punctuation. However, this type of error isn't always this easy to find, especially if you have a lot of data, or if your data is typed in by hand. For example, some customer service or sales applications involve entering information verbally provided by customers. The person who originally typed in the data might have put it in the wrong field. They might add, or forget to add, a character or digit. For example, they might enter a date of "0/10/12020" or enter someone's gender in a field meant for age. + Make sure that your imported file is correctly processed with or without a header. If there is a header row, make sure that you choose the **Contains header** upload option. + Make sure that the data doesn't exceed one or more of the [Data source quotas](data-source-limits.md). + Make sure that the data is compatible with the [Supported data types and values](supported-data-types-and-values.md). + Make sure that your calculated fields contain data that works with the calculation, rather than being incompatible with or excluded by the function in the calculated field. For example, if you have a calculated field in your dataset that uses [parseDate](parseDate-function.md), Quick Sight skips rows where that field doesn't contain a valid date. Quick Sight provides a detailed list of the errors that occur when the SPICE engine attempts to ingest data. When a saved dataset reports skipped rows, you can view the errors so you can take action to fix the issues. **To view errors for rows that were skipped during SPICE ingestion (data import)** 1. Choose **Data** at left. In the **Datasets** tab, choose the problematic dataset to open it. 1. On the dataset details page that opens, choose the **Refresh** tab. SPICE ingestion history is shown at bottom. 1. For the ingestion with the error, choose **View error summary**. This link is located under the **Status** column. 1. Examine the **File import log** that opens. It displays the following sections: + **Summary** – Provides a percentage score of how many rows were skipped out of the total number of rows in the import. For example, if there are 864 rows skipped out of a total of 1,728, the score is 50.00%. + **Skipped Rows** – Provides the row count, field name, and error message for each set of similar skipped rows. + **Troubleshooting** – Provides a link to download a file that contains error information. 1. Under **Troubleshooting**, choose **Download error rows file**. The error file has a row for each error. The file is named `error-report_123_fe8.csv`, where `123_fe8` is replaced with a unique identifying string. The file contains the following columns: + **ERROR\$1TYPE** – The type or error code for the error that occurred when importing this row. You can look up this error in the [SPICE ingestion error codes](errors-spice-ingestion.md) section that follows this procedure. + **COLUMN\$1NAME** – The name of the column in your data that caused the error. + All the columns from your imported row – The remaining columns duplicate the entire row of data. If a row has more than one error, it can appear multiple times in this file. 1. Choose **Edit data set** to make changes to your dataset. You can filter the data, omit fields, change data types, adjust existing calculated fields, and add calculated fields that validate the data. 1. After you've made changes indicated by the error codes, import the data again. If more SPICE ingestion errors appear in the log, step through this procedure again to fix all remaining errors. **Tip** If you can't solve the data issues in a reasonable amount of time by using the dataset editor, consult the administrators or developers who own the data. In the long run, it's more cost-effective to cleanse the data closer to its source, rather than adding exception processing while you're preparing the data for analysis. By fixing it at the source, you avoid a situation where multiple people fix the errors in different ways, resulting in different reporting results later on. **To practice troubleshooting skipped rows** 1. Download [samples/csv-files-for-troubleshooting-skipped-rows.zip](samples/csv-files-for-troubleshooting-skipped-rows.zip). 1. Extract the files into a folder that you can use to upload the sample .csv file into Quick Sight. The zip file contains the following two text files: + `sample dataset - data ingestion error.csv` – A sample .csv file that contains issues that cause skipped rows. You can try to import the file yourself to see how the error process works. + `sample data ingestion error file` – A sample error file generated during SPICE ingestion while importing the sample .csv file into Quick Sight. 1. Import the data by following these steps: 1. Choose **Data**, **Datasets** tab, **New**, **Dataset**. 1. Choose **Upload a file**. 1. Find and choose the file named `sample dataset - data ingestion error.csv`. 1. Choose **Upload a file**, **Edit settings and prepare data**. 1. Choose **Save** to exit. 1. Choose your dataset to view its information, then choose **View error summary**. Examine the errors and the data to help you resolve the issues. # SPICE ingestion error codes SPICE ingestion error codes The following list of errors codes and descriptions can help you understand and troubleshoot issues with data ingestion into SPICE. ## Error codes for skipped rows Row import errors The following list of errors codes and descriptions can help you understand and troubleshoot issues with skipped rows. ****ARITHMETIC\$1EXCEPTION**** – An arithmetic exception occurred while processing a value. ****ENCODING\$1EXCEPTION**** – An unknown exception occurred while converting and encoding data to SPICE. ****OPENSEARCH\$1CURSOR\$1NOT\$1ENABLED**** – The OpenSearch domain doesn't have SQL cursors enabled (`"opendistro.sql.cursor.enabled" : "true"`). For more information, see [Authorizing connections to Amazon OpenSearch Service](opensearch.md). ****INCORRECT\$1FIELD\$1COUNT**** – One or more rows have too many fields. Make sure that the number of fields in each row matches the number of fields defined in the schema. ****INCORRECT\$1SAGEMAKER\$1OUTPUT\$1FIELD\$1COUNT**** – The SageMaker AI output has an unexpected number of fields. ****INDEX\$1OUT\$1OF\$1BOUNDS**** – The system requested an index that isn't valid for the array or list being processed. ****MALFORMED\$1DATE**** – A value in a field can't be transformed to a valid date. For example, if you try to convert a field that contains a value like `"sale date"` or `"month-1"`, the action generates a malformed date error. To fix this error, remove nondate values from your data source. Check that you aren't importing a file with a column header mixed into the data. If your string contains a date or time that doesn't convert, see [Using unsupported or custom dates](using-unsupported-dates.md). ****MISSING\$1SAGEMAKER\$1OUTPUT\$1FIELD**** – A field in the SageMaker AI output is unexpectedly empty. ****NUMBER\$1BITWIDTH\$1TOO\$1LARGE**** – A numeric value exceeds the length supported in SPICE. For example, your numeric value has more than 19 digits, which is the length of a `bigint` data type. For a long numeric sequence that isn't a mathematical value, use a `string` data type. ****NUMBER\$1PARSE\$1FAILURE**** – A value in a numeric field is not a number. For example, a field with a data type of `int` contains a string or a float. ****SAGEMAKER\$1OUTPUT\$1COLUMN\$1TYPE\$1MISMATCH**** – The data type defined in the SageMaker AI schema doesn't match the data type received from SageMaker AI. ****STRING\$1TRUNCATION**** – A string is being truncated by SPICE. Strings are truncated where the length of the string exceeds the SPICE quota. For more information about SPICE, see [Importing data into SPICE](spice.md). For more information about quotas, see [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html). ****UNDEFINED**** – An unknown error occurred while ingesting data. ****UNSUPPORTED\$1DATE\$1VALUE**** – A date field contains a date that is in a supported format but is not in the supported range of dates, for example "12/31/1399" or "01/01/10000". For more information, see [Using unsupported or custom dates](using-unsupported-dates.md). ## Error codes during data import Data import errors For imports and data refresh jobs that fail, Quick Sight provides an error code indicating what caused the failure. The following list of errors codes and descriptions can help you understand and troubleshoot issues with data ingestion into SPICE. ****ACCOUNT\$1CAPACITY\$1LIMIT\$1EXCEEDED**** – This data exceeds your current SPICE capacity. Purchase more SPICE capacity or clean up existing SPICE data and then retry this ingestion. ****CONNECTION\$1FAILURE**** – Amazon Quick Sight can't connect to your data source. Check the data source connection settings and try again. ****CUSTOMER\$1ERROR**** – There was a problem parsing the data. If this persists, contact Amazon Quick Sight technical support. ****DATA\$1SET\$1DELETED**** – The data source or dataset was deleted or became unavailable during ingestion. ****DATA\$1SET\$1SIZE\$1LIMIT\$1EXCEEDED**** – This dataset exceeds the maximum allowable SPICE dataset size. Use filters to reduce the dataset size and try again. For information on SPICE quotas, see [Data source quotas](data-source-limits.md). ****DATA\$1SOURCE\$1AUTH\$1FAILED**** – Data source authentication failed. Check your credentials and use the **Edit data source** option to replace expired credentials. ****DATA\$1SOURCE\$1CONNECTION\$1FAILED**** – Data source connection failed. Check the URL and try again. If this error persists, contact your data source administrator for assistance. ****DATA\$1SOURCE\$1NOT\$1FOUND**** – No data source found. Check your Amazon Quick Sight data sources. ****DATA\$1TOLERANCE\$1EXCEPTION**** – There are too many invalid rows. Amazon Quick Sight has reached the quota of rows that it can skip and still continue ingesting. Check your data and try again. ****FAILURE\$1TO\$1ASSUME\$1ROLE**** – Amazon Quick Sight couldn't assume the correct AWS Identity and Access Management (IAM) role. Verify the policies for `Amazon Quick Sight-service-role` in the IAM console. ****FAILURE\$1TO\$1PROCESS\$1JSON\$1FILE**** – Amazon Quick Sight couldn't parse a manifest file as valid JSON. ****IAM\$1ROLE\$1NOT\$1AVAILABLE**** – Amazon Quick Sight doesn't have permission to access the data source. To manage Amazon Quick Sight permissions on AWS resources, go to the **Security and Permissions** page under the **Manage Amazon Quick Sight** option as an administrator. ****INGESTION\$1CANCELED**** – The ingestion was canceled by the user. ****INGESTION\$1SUPERSEDED**** – This ingestion has been superseded by another workflow. This happens when a new ingestion is created while another one is still in progress. Avoid manually editing the dataset multiple times in a short period, because each manual edit creates a new ingestion which will supersede and end the previous ingestion. ****INTERNAL\$1SERVICE\$1ERROR**** – An internal service error occurred. ****INVALID\$1DATA\$1SOURCE\$1CONFIG**** – Invalid values appeared in connection settings. Check your connection details and try again. ****INVALID\$1DATAPREP\$1SYNTAX**** – Your calculated field expression contains invalid syntax. Correct the syntax and try again. ****INVALID\$1DATE\$1FORMAT**** – An invalid date format appeared. ****IOT\$1DATA\$1SET\$1FILE\$1EMPTY**** – No AWS IoT Analytics data was found. Check your account and try again. ****IOT\$1FILE\$1NOT\$1FOUND**** – An indicated AWS IoT Analytics file wasn't found. Check your account and try again. ****OAUTH\$1TOKEN\$1FAILURE**** – Credentials to the data source have expired. Renew your credentials and retry this ingestion. ****PASSWORD\$1AUTHENTICATION\$1FAILURE**** – Incorrect credentials appeared for a data source. Update your data source credentials and retry this ingestion. ****PERMISSION\$1DENIED**** – Access to the requested resources was denied by the data source. Request permissions from your database administrator or ensure proper permission has been granted to Amazon Quick Sight before retrying. ****QUERY\$1TIMEOUT**** – A query to the data source timed out waiting for a response. Check your data source logs and try again. ****ROW\$1SIZE\$1LIMIT\$1EXCEEDED**** – The row size quota exceeded the maximum. ****S3\$1FILE\$1INACCESSIBLE**** – Couldn't connect to an S3 bucket. Make sure that you grant Amazon Quick Sight and users necessary permissions before you connect to the S3 bucket. ****S3\$1MANIFEST\$1ERROR**** – Couldn't connect to S3 data. Make sure that your S3 manifest file is valid. Also verify access to the S3 data. Both Amazon Quick Sight and the Amazon Quick Sight user need permissions to connect to the S3 data. ****S3\$1UPLOADED\$1FILE\$1DELETED**** – The file or files for the ingestion were deleted (between ingestions). Check your S3 bucket and try again. ****SOURCE\$1API\$1LIMIT\$1EXCEEDED\$1FAILURE**** – This ingestion exceeds the API quota for this data source. Contact your data source administrator for assistance. ****SOURCE\$1RESOURCE\$1LIMIT\$1EXCEEDED**** – A SQL query exceeds the resource quota of the data source. Examples of resources involved can include the concurrent query quota, the connection quota, and physical server resources. Contact your data source administrator for assistance. ****SPICE\$1TABLE\$1NOT\$1FOUND**** – An Amazon Quick Sight data source or dataset was deleted or became unavailable during ingestion. Check your dataset in Amazon Quick Sight and try again. For more information, see [Troubleshooting skipped row errors](troubleshooting-skipped-rows.md). ****SQL\$1EXCEPTION**** – A general SQL error occurred. This error can be caused by query timeouts, resource constraints, unexpected data definition language (DDL) changes before or during a query, and other database errors. Check your database settings and your query, and try again. ****SQL\$1INVALID\$1PARAMETER\$1VALUE**** – An invalid SQL parameter appeared. Check your SQL and try again. ****SQL\$1NUMERIC\$1OVERFLOW**** – Amazon Quick Sight encountered an out-of-range numeric exception. Check related values and calculated columns for overflows, and try again. ****SQL\$1SCHEMA\$1MISMATCH\$1ERROR**** – The data source schema doesn't match the Amazon Quick Sight dataset. Update your Amazon Quick Sight dataset definition. ****SQL\$1TABLE\$1NOT\$1FOUND**** – Amazon Quick Sight can't find the table in the data source. Verify the table specified in the dataset or custom SQL and try again. ****SSL\$1CERTIFICATE\$1VALIDATION\$1FAILURE**** – Amazon Quick Sight can't validate the Secure Sockets Layer (SSL) certificate on your database server. Check the SSL status on that server with your database administrator and try again. ****UNRESOLVABLE\$1HOST**** – Amazon Quick Sight can't resolve the host name of the data source. Verify the host name of the data source and try again. ****UNROUTABLE\$1HOST**** – Amazon Quick Sight can't reach your data source because it's inside a private network. Ensure that your private VPC connection is configured correctly in Enterprise Edition, or allow Amazon Quick Sight IP address ranges to allow connectivity for Standard Edition. # Updating files in a dataset To get the latest version of files, you can update files in your dataset. You can update these types of files: + Comma-delimited (CSV) and tab-delimited (TSV) text files + Extended and common log format files (ELF and CLF) + Flat or semistructured data files (JSON) + Microsoft Excel (XLSX) files Before updating a file, make sure that the new file has the same fields in the same order as the original file currently in the dataset. If there are field (column) discrepancies between the two files, an error occurs and you need to fix the discrepancies before attempting to update again. You can do this by editing the new file to match the original. Note that if you want to add new fields, you can append them after the original fields in the file. For example, in a Microsoft Excel spreadsheet, you can append new fields to the right of the original fields. **To update a file in a dataset** 1. In Quick Sight, choose **Data** at left. 1. In the **Datasets** tab, choose the dataset that you want to update, and then choose **Edit dataset**. 1. On the data preparation page that opens, choose the drop-down list for the file that you want to update, and then choose **Update file**. 1. On the **Update file** page that opens, choose **Upload file**, and then navigate to a file. Quick Sight scans the file. 1. If the file is a Microsoft Excel file, choose the sheet that you want on the **Choose your sheet** page that opens, and then choose **Select**. 1. Choose **Confirm file update** on the following page. A preview of some of the sheet columns is shown for your reference. A message that the file updated successfully appears at top right and the table preview updates to show the new file data.