After careful consideration, we decided to end support for Amazon FinSpace, effective October 7, 2026. Amazon FinSpace will no longer accept new customers beginning October 7, 2025. As an existing customer with an Amazon FinSpace environment created before October 7, 2025, you can continue to use the service as normal. After October 7, 2026, you will no longer be able to use Amazon FinSpace. For more information, see [Amazon FinSpace end of support](https://docs.aws.amazon.com/finspace/latest/userguide/amazon-finspace-end-of-support.html). # Prepare and analyze data in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. You can perform data preparation and analysis on datasets in Amazon FinSpace. You cannot analyze datasets directly, instead you can create data views from datasets that enable you to perform analysis. ** **Topics** ** + [ # Working with Amazon FinSpace notebooks ](working-with-amazon-finSpace-notebooks.md) + [ # Working with Spark clusters in Amazon FinSpace ](working-with-Spark-clusters.md) + [ # Importing library in Amazon FinSpace ](import-library.md) + [ # Accessing Amazon S3 Bucket from FinSpace notebook ](access-s3-buckets.md) + [ # Amazon FinSpace Spark time series library ](finspace-time-series-library.md) # Working with Amazon FinSpace notebooks **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. Amazon FinSpace notebook provides an Integrated Development Environment (IDE) that lets you access data from the FinSpace Data Catalog to perform data preparation and analysis. FinSpace simplifies the use of Apache Spark providing access to fully managed Spark Clusters using easy to launch cluster templates. For more information, see [Apache Spark](https://spark.apache.org/). **Note** In order to use notebooks and Spark clusters, you must be a superuser or a member of a group with necessary permissions - **Access Notebooks, Manage Clusters**. The Spark clusters are terminated daily at midnight US Eastern time. FinSpace notebooks are programmed using Python. Python and Spark integration is achieved using the PySpark library. For more information, see [PySpark](https://spark.apache.org/docs/latest/api/python/). ** **Topics** ** + [ # Opening the notebook environment ](opening-the-notebook-environment.md) + [ # Working in the notebook environment ](working-in-the-notebook-environment.md) + [ # Access datasets from a notebook ](access-datasets-notebook.md) + [ # Example notebooks ](example-notebook.md) # Opening the notebook environment **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. **Note** In order to open a notebook environment, you must be a superuser or a member of a group with necessary permissions - **Access Notebooks**. Expect a one-time setup delay of 15-20 minutes for the notebook environment after creating a new user. **You can open a notebook environment in the following ways** + Using the data view cards on homepage. + From the dataset details page under **Data Overview** tab. + From the dataset details page under **All Data Views** tab. ## Access notebook from homepage **To access notebook environment from the recently created data views** 1. Sign in to the FinSpace web application. For more information, see [Signing in to the Amazon FinSpace web application](signing-into-amazon-finspace.md). 1. From the homepage, under the **Status of Data Views** section, find the recently created data view. 1. On the data view card, choose **Analyze**. The notebook opens in a new tab on your browser. ## Access notebook from Data Overview tab **To access notebook environment from the overview tab** 1. Sign in to the FinSpace web application. For more information, see [Signing in to the Amazon FinSpace web application](signing-into-amazon-finspace.md). 1. From the homepage, search for a dataset. 1. Choose the dataset name to view the dataset details page. 1. From the **Data Overview** tab, under **Analyze Data** section, choose **Analyze in Notebook** in the data view card. The notebook opens in a new tab on your browser. ![\[A screenshot that shows the data overview tab.\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/data-view-card.png) **** ## Access notebooks from All Data Views tab **To access notebook environment from the list of all data views for a dataset** 1. Sign in to the FinSpace web application. For more information, see [Signing in to the Amazon FinSpace web application](signing-into-amazon-finspace.md). 1. From the homepage, search for a dataset. 1. Choose the dataset name to view the dataset details page. 1. Choose **All Data Views** tab. 1. From the **Data Views** table, choose **Analyze in Notebook** for any of the data views. The notebook opens in a new tab on your browser. ![\[A screenshot that shows the All Data Views tab.\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/all-data-views-tab.png) # Working in the notebook environment **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. Choosing **Go to Notebook** or **Analyze in Notebook** will open Jupyter Lab in a new tab in your web browser. You will land in the launcher page of SageMaker studio. **To start a notebook with FinSpace kernel** 1. In the upper-left corner of SageMaker Studio, choose **Amazon SageMaker Studio** to open Studio Launcher. 1. On the **Launcher** page, choose **Notebooks and compute resources**. 1. For **Select a SageMaker image**, choose the FinSpace PySpark image. 1. Choose **Notebook** to create a notebook in the FinSpace PySpark image. ## FinSpace kernel The FinSpace PySpark Kernel comes with all the libraries required to access and work with data stored in FinSpace, including the Spark Cluster management API and time series analytics library. The FinSpace Cluster Management API is used to instantiate and connect the notebook instance to a dedicated Spark Cluster. FinSpace Spark clusters use Kerberos authentication for additional security. FinSpace provides with complete resource isolation when working with Spark Clusters. When a FinSpace PySpark Kernel is instantiated for the first time in a new notebook session, you can expect a startup time of about 3 to 5 minutes to allow bootstrapping of all dependencies on the image supporting the notebook. # Access datasets from a notebook **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. You can conveniently and securely access all datasets to prepare and analyze data from your Amazon FinSpace notebook. The following sections show how to access data from a FinSpace notebook. **Note** In order to use notebooks and Spark clusters, you must be a superuser or a member of a group with necessary permissions - **Access Notebooks, Manage Clusters**. ## Access data using a pre-populated notebook **To access data using a pre-populated notebook** 1. Sign in to the FinSpace web application. For more information, see [Signing in to the Amazon FinSpace web application](signing-into-amazon-finspace.md). 1. Open a notebook by using one of the three methods listed in [Opening the notebook environment](opening-the-notebook-environment.md). In the notebook, the dataset ID and data view ID are pre-populated. 1. Run all cells to print the schema and content of the data view. ## Access data using a newly created notebook **To access data using a newly created notebook** 1. Run the following code from your notebook to instantiate a cluster and connect the FinSpace PySpark image to the cluster. ``` %local from aws.finspace.cluster import FinSpaceClusterManager finspace_clusters = FinSpaceClusterManager() finspace_clusters.auto_connect() ``` The output should be similar to the following output ``` Cluster is starting. It will be operational in approximately 5 to 8 minutes Started cluster with cluster ID: 8x6zd9cq and state: STARTING ...... cleared existing credential location Persisted krb5.conf secret to /etc/krb5.conf re-establishing connection... Persisted keytab secret to /home/sagemaker-user/livy.keytab Authenticated to Spark cluster Persisted Sparkmagic config to /home/sagemaker-user/.Sparkmagic/config.json Started Spark cluster with clusterId: 8x6zd9cq finished reloading all magics & configurations Persisted FinSpace cluster connection info to /home/sagemaker-user/.Sparkmagic/FinSpace_connection_info.json SageMaker Studio Environment is now connected to your FinSpace Cluster: 8x6zd9cq at GMT: 2021-01-15 02:13:50. ``` **Note** Without the `%local` at the beginning of the cell, your code will be executed on the Spark cluster. 1. To access the data view, you will need the dataset ID and data view ID. To get these IDs 1. In the FinSpace web application, open the dataset details page of the dataset that you want to analyze. 1. Under the **All Data Views** tab, find the data view that you want to analyze. 1. Choose **Details**. 1. Copy the **Data View ID** and **Dataset ID** to use in the notebook. 1. Initialize dataset ID and data view ID in the notebook. ``` dataset_id = "rgg1hj1" data_view_id = "VrvKEKnA1El2nr821BaLTQ" ``` 1. Instantiate FinSpace Analytics Manager to access the data and read into a Spark DataFrame. ``` from aws.finspace.analytics import FinSpaceAnalyticsManager finspace_analytics = FinSpaceAnalyticsManager(Spark = Spark) df = finspace_analytics.read_data_view(dataset_id = dataset_id, data_view_id = data_view_id) ``` # Example notebooks **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. You can access example notebooks and Python scripts illustrating how to use Amazon FinSpace to prepare and analyze data using Spark Clusters and the time series analytics library. You can clone the gitrepo in your Jupyter Lab for easy access to the example notebooks. # Working with Spark clusters in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. Amazon FinSpace simplifies how to work with Spark clusters by offering easy to use cluster configuration templates that allow you to launch, connect, resize, and terminate without worrying to manage the underlying infrastructure. Every user in FinSpace with **Access Notebooks** and **Manage Clusters** permission can instantiate one cluster. **Note** In order to use notebooks and Spark clusters, you must be a superuser or a member of a group with necessary permissions - **Access Notebooks, Manage Clusters**. You can choose one of the following cluster configuration templates: + Small + Medium + Large + XLarge + 2XLarge **Note** You are charged by the minute for using the Spark clusters. Terminate your Spark cluster when you are done using it. ## Import FinSpace cluster management library Use the following code to import the cluster management library in a notebook. ``` %local from aws.finspace.cluster import FinSpaceClusterManager ``` ## Start a Spark cluster ****Use the following code to start and connect your notebook to a Spark cluster. ``` %local from aws.finspace.cluster import FinSpaceClusterManager finspace_clusters = FinSpaceClusterManager() finspace_clusters.auto_connect() ``` For a newly created cluster, the output should be similar to the following. ``` Cluster is starting. It will be operational in approximately 5 to 8 minutes Started cluster with cluster ID: 8x6zd9cq and state: STARTING ...... cleared existing credential location Persisted krb5.conf secret to /etc/krb5.conf re-establishing connection... Persisted keytab secret to /home/sagemaker-user/livy.keytab Authenticated to Spark cluster Persisted Sparkmagic config to /home/sagemaker-user/.Sparkmagic/config.json Started Spark cluster with clusterId: 8x6zd9cq finished reloading all magics & configurations Persisted FinSpace cluster connection info to /home/sagemaker-user/.Sparkmagic/FinSpace_connection_info.json SageMaker Studio Environment is now connected to your FinSpace Cluster: 8x6zd9cq at GMT: 2021-01-15 02:13:50. ``` You can expect a startup time of about 5 to 8 minutes when instantiating a cluster for the first time. Once a cluster is created, any newly created notebook will detect and connect to the running cluster when an `auto_connect()` call is issued and this operation is instantaneous. ## List details for Spark clusters **Use the following code to list the Spark cluster name and details** ``` %local finspace_clusters.list() ``` The output should be similar to the following output. ``` {'clusters': [{'clusterId': '8x6zd9cq', 'clusterStatus': {'state': 'RUNNING', 'reason': 'Started successfully', 'details': ''}, 'name': 'hab-cluster-3e51', 'currentTemplate': 'FinSpace-Small', 'requestedTemplate': 'FinSpace-Small', 'clusterTerminationTime': 1610676314, 'createdTimestamp': 1610676374420, 'modifiedTimestamp': 1610676823805}, {'clusterId': '3ysaqx3g', 'clusterStatus': {'state': 'TERMINATED', 'reason': 'Initiated by user', 'details': ''}, 'name': 'hab-cluster-c4f9', 'currentTemplate': 'FinSpace-Small', 'requestedTemplate': 'FinSpace-Small', 'clusterTerminationTime': 1610478542, 'createdTimestamp': 1610478602457, 'modifiedTimestamp': 1610514182552}]} ``` In the output above, you can see the `clusterId` **8x6zd9cq** is a small cluster with state equals to **RUNNING**, and the `clusterId` **3ysaqx3g** is a small cluster with state equals to **TERMINATED**. ## Resize Spark cluster Scale your Spark cluster up or down based on your compute needs and the volume of data you need to analyze. **To resize clusters** 1. Type the following code to update your cluster to a **Large** size. ``` %local finspace_clusters.update('8x6zd9cq','Large') ``` The output will look like below ``` {'clusterId': '8x6zd9cq', 'clusterStatus': {'state': 'UPDATING', 'reason': 'Initiated by user'}} ``` 1. Note that the `update()` operation runs asynchronous so that you can continue to work on the cluster as the update operation completes. 1. Check the status of the update operation using the `list()` function. ``` {'clusters': [{'clusterId': '8x6zd9cq', 'clusterStatus': {'state': 'UPDATING', 'reason': 'Initiated by user', 'details': ''}, 'name': 'hab-cluster-3e51', 'currentTemplate': 'Small', 'requestedTemplate': 'Large', 'clusterTerminationTime': 1610676314, 'createdTimestamp': 1610676374420, 'modifiedTimestamp': 1610682765327}} ``` 1. In the output above, the `clusterId` **8x6zd9cq** is being updated from **Small** to **Large**. ## Terminate Spark cluster Terminate your Spark cluster once your work is done, so that you don't incur additional charges. **To terminate your Spark cluster** 1. Type the following code to terminate a cluster. ``` %local finspace_clusters.terminate('8x6zd9cq') ``` 1. You can check the state of the cluster using the `list()` function. # Importing library in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. You can install notebook-scoped libraries on a running Amazon FinSpace cluster directly via a FinSpace notebook. This capability is useful in scenarios in which you do not have access to a PyPI repository but need to analyze and visualize a dataset. Notebook-scoped libraries provide you the following benefits: + **Runtime installation** – You can import Python libraries from PyPI repositories and install them on your remote cluster on the fly when you need them. These libraries are instantly available to your Spark runtime environment. There is no need to restart the notebook session or recreate your cluster. + **Dependency isolation** – The libraries you install using FinSpace notebooks are isolated to your notebook session and don't interfere with bootstrapped cluster libraries or libraries installed from other notebook sessions. These notebook-scoped libraries take precedence over bootstrapped libraries. Multiple notebook users can import their preferred version of the library and use it without dependency clashes on the same cluster. + **Portable library environment** – The library package installation happens from your notebook file. This allows you to recreate the library environment when you switch the notebook to a different cluster by re-executing the notebook code. At the end of the notebook session, the libraries you install through FinSpace notebooks are automatically removed from the hosting cluster. The following example code shows how to install pandas and matplotlib from the PiPY repository. ``` sc.install_pypi_package("pandas==0.25.1") #Install pandas version 0.25.1 sc.install_pypi_package("matplotlib", "https://pypi.org/simple") #Install matplotlib from given PyPI repository ``` You can uninstall packages using the `uninstall_package` PySpark API. ``` sc.uninstall_package('pandas') ``` # Accessing Amazon S3 Bucket from FinSpace notebook **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. An Amazon FinSpace environment can be configured to access your Amazon S3 buckets from FinSpace notebook. **Note** In order to setup access to an S3 bucket, you must be authorized to access the FinSpace page in AWS Management Console and make changes to bucket-level permissions in Amazon S3. **To find your infrastructure account number** 1. Sign in to the AWS Management Console and open the Amazon FinSpace console at [https://console.aws.amazon.com/finspace](https://console.aws.amazon.com/finspace/landing). 1. On the FinSpace console, from the list of environments, choose the environment that you want to setup to access an S3 bucket. If there are no environments available, create one by following the steps listed in [Create an Amazon FinSpace environment](create-an-amazon-finspace-environment.md). 1. On the environment page, copy and save the FinSpace infrastructure account name. **To setup access for FinSpace infrastructure account in S3 bucket policy** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. Choose the bucket that you want to access from your FinSpace environment. 1. Set a bucket policy for the bucket with following json code. For example, if your bucket name is `example-bucket` and your FinSpace infrastructure account number is `123456789101` below would be the example policy. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Id": "CrossAccountAccess", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::123456789101:role/FinSpaceServiceRole" ] }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::example-bucket/*" }, { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::123456789101:role/FinSpaceServiceRole" ] }, "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::example-bucket" } ] } ``` ------ Using the above policy, you should be able to access `example-bucket` from the Jupyter notebook of a FinSpace environment, which is associated with the FinSpace infrastructure account number `123456789101`. # Amazon FinSpace Spark time series library **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. Amazon FinSpace PySpark Kernel delivers a time series analytics library to prepare and analyze historical financial time series data using FinSpace managed Spark clusters. You can use the time series library to analyze high-density data like US options historical Options Price Reporting Authority (OPRA) with billions of daily events or sparse time series data such as quotes for fixed income instruments. The time series library is available to use in the FinSpace notebook environment. The time-series library is logically organized in four stages for a conceptual framework. Every stage provides a set of functions and you can plug your own functions. 1. **Collect** – The objective of this stage is to collect the series of events that arrive at an irregular frequency into uniform intervals called bars. You can perform collection with your functions or use the FinSpace functions to calculate bars such as time bars. 1. **Summarize** – The objective of this stage is to take collected data in bars from previous stage and summarize it using the events captures within a bar. 1. **Fill and Filter** – The data produced in the previous stage could have missing bars where no data was collected or contain data that is not desired to be used in the next stage. The objective of this stage is to prepare a dataset of features with evenly spaced intervals and filter out any data outside desired time window. 1. **Analytics** – At this stage, a prepared dataset of features is ready for application of technical and statistical indicators. You can bring your own indicator functions or choose one of the FinSpace functions for this stage. ![\[A diagram that shows the time series framework.\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/ts-framework.png) See the following sections to learn more about supported functions in the time series library. ** **Topics** ** + [ # Collect time bars operations in Amazon FinSpace ](time-series-collect.title.md) + [ # Summarize bars operations in Amazon FinSpace ](time-series-summarize-bars.title.md) + [ # Fill and filter operations in Amazon FinSpace ](time-series-fill-filter.title.md) + [ # Analyze operations in Amazon FinSpace ](time-series-analyze.title.md) + [ # Using the Amazon FinSpace library ](finspace-using-the-library.title.md) # Collect time bars operations in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. The objective of functions at this stage is to collect the series of events that arrive at an irregular frequency into uniform intervals called bars. You can perform collection with your functions or use the Amazon FinSpace functions to calculate bars. Collect functions are available in the `aws.finspace.timeseries.spark.windows` module and include the following list of functions. ## Compute analytics on features ``` aws.finspace.timeseries.spark.windows.compute_analytics_on_features(data, new_column, func, partition_col_list=None, add_intermediate=False) ``` Appends to data Dataframe, a new column whose value is computed by executing pandas user defined function (UDF) on a window of rows as specified by the function window dependency member. **Parameters** + `data` (DataFrame) – input dataframe + `new_column` (str) – name of new column to add + `input_spec` – input specification + `func` (Callable[…​, Column]) – function to calculate over data + `grouping_col_list` – a single or list of columns to group window on + `add_intermediate` (Optional[bool]) – include intermediate data used in the calculation **Return type** `DataFrame` **Returns** ## Compute features on time bars ``` aws.finspace.timeseries.spark.windows.compute_features_on_time_bars(data, new_column, func, force_ordering=False,*ordering_cols) ``` Reduces data by applying function preserving all other columns. **Parameters** + `data` (DataFrame) – input DataFrame + `new_column` (str) – new column name + `func` (Callable[…​, Column]) – function to calculate over data + `force_ordering` (Optional[bool]) – return data in sort in timecolumn order + `ordering_cols` (str) – list of cols to orderBy on **Return type** `DataFrame` **Returns** `DataFrame` ## Create time bars ``` aws.finspace.timeseries.spark.windows.create_time_bars(data, timebar_column, grouping_col_list, input_spec, timebar_spec, force_ordering=False) ``` Appends a column to the data frame in data with a rolling window of data. An optional `force_ordering` flag ensures that the rolling data is order by the `timebar_column`. **Parameters** + `data` (Union[Column, DataFrame]) – input dataframe + `timebar_column` (str) – new timebar column name + `grouping_col_list` (Union[str, List[str]]) – list of columns to group results on + `input_spec` (BarInputSpec) – the input spec used to generate the time bars + `timebar_spec` (Union[TimeBarSpec, Column]) – the timebar spec used to generate the time bars + `force_ordering` (Optional[bool]) – optional force ordering in windows **Return type** `DataFrame` **Returns** `DataFrame` ## Spark spec module ### Bar input spec ``` class aws.finspace.timeseries.spark.spec.BarInputSpec(bar_structure_name, *bar_value_columns) ``` Bases: object This class is responsible for modeling the input specification of bar operations. ### Calc input spec ``` class aws.finspace.timeseries.spark.spec.CalcInputSpec(timestamp_column, holiday_calendar=, **kwargs_func_to_column)` ``` Bases: object This class is responsible for modeling the input specification of calculation operations. ### Time bar spec ``` class aws.finspace.timeseries.spark.spec.TimeBarSpec(timestamp_column, window_duration, slide_duration=None, start_time=None) ``` Bases: object This class models the input time window specification, and associated calendar. ### to\$1window() Create an equivalent spark window from TimeBarSpec. # Summarize bars operations in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. The objective of this stage is to take collected data in bars from previous stage and summarize it using the events captures within a bar. Collect functions are available in the `aws.finspace.timeseries.spark.summarizer` module and include the following list of functions. ## Bar count ``` aws.finspace.timeseries.spark.summarizer.bar_cnt(input_series) ``` **Returns** the number of items in interval. **Parameters** + `input_series` (Series) – a series window produced through groupby **Return type** `Series` **Returns** `pandas.Series` ## Close ``` aws.finspace.timeseries.spark.summarizer.close(price) ``` Returns the last row, called close as its the closing price of that interval. **Parameters** + `price` (Series) – a series Window produced through group by **Return type** `Series` **Returns** `pandas.Series` ## First last high low ``` aws.finspace.timeseries.spark.summarizer.first_last_high_low(sort_col: list, price: list) -> list ``` **Return type** `list` ## First last high low presorted ``` aws.finspace.timeseries.spark.summarizer.*first_last_high_low_presorted*(price: list) -> list ``` **Return type** `list` ## High ``` aws.finspace.timeseries.spark.summarizer.high(price) ``` Returns the highest price in that interval. **Parameters** + `price` (Series) – a DataFrame Window produced through groupby **Return type** `Series` **Returns** `pandas.Series` ## Low ``` aws.finspace.timeseries.spark.summarizer.low(price) ``` Returns the lowest price in that interval. **Parameters** + `price` (Series) – a series Window produced through groupby **Return type** `Series` **Returns** `pandas.Series` ## Low high ``` aws.finspace.timeseries.spark.summarizer.lowhigh(value) -> list ``` **Return type** `list` ## Open high low close (OHLC) The first, high, low, and last value over an interval. ``` aws.finspace.timeseries.spark.summarizer.ohlc_func(sort_col: list, price: list) -> list ``` **Return type** `list` ## Open high low close pre-sorted (OHLC) The first, high, low, and last value over an interval. ``` aws.finspace.timeseries.spark.summarizer.ohlc_func_pre_sorted(price: list) -> list ``` **Return type** `list` ## Open high low close scala (OHLC) The first, high, low, and last value over an interval. ``` aws.finspace.timeseries.spark.summarizer.ohlc_scala(timeseries, values) ``` ## Open ``` aws.finspace.timeseries.spark.summarizer.open(price) ``` Returns the first row, the opening price over that interval. **Parameters** + `price` (Series) – a series window produced through groupby **Return type** `Series` **Returns** `pandas.Series` ## Standard deviation ``` aws.finspace.timeseries.spark.summarizer.std(price) ``` Returns the standard deviation over that interval. **Parameters** + `price` (Series) – a series Window produced through groupby **Return type** `Series` **Returns** `pandas.Series` ## Time Delta ``` aws.finspace.timeseries.spark.summarizer.*time_delta*(time_series: list, ref_date: datetime.date) -> list ``` **Return type** `list` ## Total volume ``` aws.finspace.timeseries.spark.summarizer.total_volume(volume) ``` The total volume over that interval. **Parameters** + `volume` (Series) – input volume **Return type** `DataFrame` **Returns** ## Volume and close ``` aws.finspace.timeseries.spark.summarizer.volume_and_close(price: list, vol: list) -> list ``` **Return type** `list` ## Volume weighted average price (VWAP) ``` aws.finspace.timeseries.spark.summarizer.vwap(price, volume) ``` The volume weighted average price over that interval. **Parameters** + `price` (Series) – input price series + `volume` (Series) – input volume **Return type** `DataFrame` **Returns** # Fill and filter operations in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. The data produced after summarizing bars could have missing bars where no data was collected or contain data that is not desired to be used in the next stage. The objective of this stage is to prepare a dataset with evenly spaced intervals and filter out any data outside desired time window. Fill and Filter are available in the `aws.finspace.timeseries.spark.prepare` module. ## Fill and filter functions ``` aws.finspace.timeseries.spark.prepare.time_bar_fill_and_filter(data, timebar_column_name, business_calendar, time-bar_spec, start_date, end_date, fill_value=None, start_time=None, end_time=None) ``` The data produced after summarizing bars could have missing bars where no data was collected or contain data that is not desired to be used in the next stage. The objective of this stage is to prepare a dataset with evenly spaced intervals and filter out any data outside desired time window. Fill and Filter are available in the `aws.finspace.timeseries.spark.prepare` module. The Fill and filter function will fill with nulls in all rows that need to exist, and filter all rows that are outside the business calendar date/time range in a given calendar. **Parameters** + `data` (DataFrame) – input dataframe + `timebar_column_name` (str) – name of the timebar column to fill against + `business_calendar` (AbstractCalendar) – business calendar + `timebar_spec` (TimeBarSpec) – time bar input spec associated with the bars that were created. it provides the bar frequency + `start_date` (date) – start of date + `end_date` (date) – end date + `fill_value` (Optional[float]) – value to fill + `start_time` (Optional[time]) – start time of the day + `end_time` (Optional[time]) – end time of the day **Return type** `DataFrame` **Returns** `DataFrame` ## Calendars module Use the calendar module for defining a calendar schedule to be used in fill and filter. ### Abstract calendar ``` class aws.finspace.finance.calendars.AbstractCalendar Bases: object ``` Defines abstract class for calendars. ``` DISRUPTIONS = 'DISRUPTIONS' EARLY_CLOSINGS = 'EARLY_CLOSING' EARLY_CLOSING_TIME = datetime.time(13, 30) END_OF_TRADING = 'END_OF_TRADING' HOLIDAYS = 'HOLIDAYS' START_OF_TRADING = 'START_OF_TRADING' TZINFO ='TZINFO' ``` ### Create schedule ``` create_schedule_from_to(from_date, to_date, time_bar_spec_window_duration, from_time=None, to_time=None, tzinfo=) Abstract method, provide override ``` Creates a list of dates associated with a particular type of calendar. **Parameters** + `from_date`(date) – from date + `to_date` (date) – to date + `time_bar_spec_window_duration` (str) –  + `from_time` (Optional[time]) – from time + `to_time` (Optional[time]) – to time **Return type** `array` **Returns** raw\$1calendar\$1data() Return type `Dict[str,Any]` Returns raw calendar data ### NYSE calendar ``` class aws.finspace.finance.calendars.NYSECalendar20192020 Bases: aws.finspace.finance.calendars.USEndOfDayCalendarActAct_NoWeekends ``` Returns a holiday calendar with no weekends, and according to the NYSE exchange trading holidays and half-days for [2019](https://www.tradinghours.com/exchanges/nyse/market-holidays/2019) and [2020](https://www.tradinghours.com/exchanges/nyse/market-holidays/2020). ``` create_schedule_from_to(from_date, to_date, time_bar_spec_window_duration, from_time=None, to_time=None) ``` **Parameters** + `from_date` (date) – from date + `to_date` (date) – to date + `time_bar_spec_window_duration` (str) + `from_time` (Optional[time]) – from time + `to_time` (Optional[time]) – to time + `tzinfo` – time to localize to **Return type** `array` **Returns** raw\$1calendar\$1data() Return type `Dict[str,Any]` Returns raw calendar data ### End of day calendar actual ``` class aws.finspace.finance.calendars.USEndOfDayCalendarActAct_NoWeekends Bases: aws.finspace.finance.calendars.AbstractCalendar ``` Return 30/360 calendar, without weekends, without exchange hours. ``` create_schedule_from_to(from_date, to_date, time_bar_spec_window_duration, from_time=None, to_time=None) ``` **Parameters** + `from_date` (date) – from date + `to_date` (date) – to date + `time_bar_spec_window_duration` (str) + `from_time` (Optional[time]) – from time + `to_time` (Optional[time]) – to time + `tzinfo` – time to localize to **Return type** `array` **Returns** raw\$1calendar\$1data() Return type `Dict[str,Any]` Returns raw calendar data # Analyze operations in Amazon FinSpace **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. At this stage, a prepared dataset of features is ready for application of technical and statistical indicators. You can bring your own indicator functions or choose one of the FinSpace functions for this stage. ## Acceleration bands (ABANDS) ``` aws.finspace.timeseries.spark.analytics.abands(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Acceleration Bands (ABANDS) created by Price Headley plots upper and lower envelope bands around a simple moving average. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Accumulation/Distribution (AD) ``` aws.finspace.timeseries.spark.analytics.acc_dist_indicator(time_col_name,price_col_name, high_col_name, low_col_name, volume_col_name) ``` The Accumulation/Distribution (AD) study attempts to quantify the amount of volume flowing into or out of an instrument by identifying the position of the close of the period in relation to that period's high/low range. The volume for the period is then allocated accordingly to a running continuous total. In this indicator, if the divisor, high-low is 0, and hence the current money flow volume is nan, then it means that price, which must fall between high and low is also going to equal high. In that case the numerator is 0 as well which means that the contribution should really be 0 in this case. Hence below we filter the NANs out in the equation. [https://www.investopedia.com/terms/a/accumulationdistribution.asp](https://www.investopedia.com/terms/a/accumulationdistribution.asp) **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price + `volume_col_name` (str) – asset volume for the bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Average directional movement index rating (ADXR) ``` aws.finspace.timeseries.spark.analytics.adrx_indicator(adx_period, period, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Average Directional Movement Index Rating (ADXR) is an element of the Directional Movement System, developed by J. Welles Wilder. ADXR quantifies the change in momentum of the Average Directional Index (ADX). This indicator is the result of adding two values of the Average Directional Index (the current ADX value and the ADX value n-periods ago), after which dividing this sum by two, or: `ADXR = (ADX + ADX n-periods ago) / 2` **Parameters** + `adx_period` (int) – look back window for Average Directional Index + `period` (int) – look back window for Average Directional Index + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Average directional movement index (ADX) ``` aws.finspace.timeseries.spark.analytics.adx_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Average Directional Movement Index (ADX) is designed to quantify trend strength by measuring the amount of price movement in a single direction. The ADX is part of the Directional Movement system published by J. Welles Wilder, and is the average resulting from the Directional Movement indicators. **Parameters** + `tenor` – window size + `time_col_name` – name of time column + `price_col_name` – input array of closing prices over bar + `high_col_name` – input array of high prices over bar + `low_col_name` – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Moving average convergence divergence (MACD) ``` aws.finspace.timeseries.spark.analytics.apo_indicator(short_tenor, long_tenor, time_col_name, input_array_col_name) ``` The Moving Average Convergence Divergence (MACD) was developed by Gerald Appel, and is based on the differences between two moving averages of different lengths, a Fast and a Slow moving average. A second line, called the Signal line is plotted as a moving average of the MACD. A third line, called the MACD Histogram is optionally plotted as a histogram of the difference between the MACD and the Signal Line. Learn [more](https://www.investopedia.com/terms/m/macd.asp). **Parameters** + `short_tenor` (int) – short window size + `long_tenor` (int) – long window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Aroon down indicator ``` aws.finspace.timeseries.spark.analytics.aroon_down_indicator(tenor, time_col_name, price_col_name) ``` `Aroon down indicator = ((Number of periods - Number of periods since lowest low) / Number of periods) * 100` **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Aroon oscillator ``` aws.finspace.timeseries.spark.analytics.aroon_oscillator(tenor, time_col_name, price_col_name) ``` The Aroon Oscillator is a trend-following indicator that uses aspects of the Aroon Indicator (Aroon Up and Aroon Down)to gauge the strength of a current trend and the likelihood that it will continue. Readings above zero indicate that an uptrend is present, while readings below zero indicate that a downtrend is present. Traders watch for zero line crossovers to signal potential trend changes. They also watch for big moves, above 50 or below -50 to signal strong price moves. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Aroon up indicator ``` aws.finspace.timeseries.spark.analytics.aroon_up_indicator(tenor, time_col_name, price_col_name) ``` `Aroon up indicator = ((Number of periods - Number of periods since highest high) / Number of periods) * 100` **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Average true range (ATR) ``` aws.finspace.timeseries.spark.analytics.average_true_range(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Average True Range (ATR) study measures the size of the period's range, and takes into account any gap from the close of the previous period. Learn [more](https://www.investopedia.com/terms/a/atr.asp). **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Bollinger band (BBANDS) ``` aws.finspace.timeseries.spark.analytics.bollinger_bands(tenor, no_std, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Bollinger Band (BBANDS) study created by John Bollinger plots upper and lower envelope bands around the price of the instrument. The width of the bands is based on the standard deviation of the closing prices from a moving average of price. Learn [more](https://www.investopedia.com/terms/b/bollingerbands.asp). **Parameters** + `tenor` (int) – window to perform the calculation over + **no\$1std** (int) – number of standard deviations + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** float ## Chaikin money flow ``` aws.finspace.timeseries.spark.analytics.chaiken_money_flow_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name, vol ume_col_name) ``` Developed by Marc Chaikin, Chaikin Money Flow measures the amount of Money Flow Volume over a specific period. Money Flow Volume forms the basis for the Accumulation Distribution Line. Instead of a cumulative total, Chaikin Money Flow sums Money Flow Volume for a specific look-back period, typically 20 or 21 days. The resulting indicator fluctuates above/below the zero line just like an oscillator. Chartists weigh the balance of buying or selling pressure with the absolute level of Chaikin Money Flow. Additionally, chartists can look for crosses above or below the zero line to identify changes on money flow. **Parameters** + `tenor` (int) – look back window + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar + `volume_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Chaikens volatility indicator ``` aws.finspace.timeseries.spark.analytics.chaikens_volatility_indicator(tenor, time_col_name, high_col_name, low_col_name) ``` Marc Chaikin's Volatility indicator compares the spread between a security's high and low prices, quantifying volatility as a widening of the range between the high and the low price. **Parameters** + `tenor` (int) – look back + `time_col_name` (str) – name of time column + `price_col_name` – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Chande momentum indicator ``` aws.finspace.timeseries.spark.analytics.cmo_indicator(tenor, time_col_name, price_col_name) ``` The Chande Momentum Oscillator (CMO) developed by Tushar Chande attempts to capture the momentum of the instrument. The indicator oscillates between -100 and 100 with overbought level of 50 and oversold level of -50. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Commodity channel index (CCI) ``` aws.finspace.timeseries.spark.analytics.commodity_channel_index(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Commodity Channel Index (CCI) compares the current mean price with the average mean price over a typical window of 20 periods. Learn [more](https://www.investopedia.com/terms/c/commoditychannelindex.asp). **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Coppock curve ``` aws.finspace.timeseries.spark.analytics.coppock_curve_indicator(roc1_period, roc2_period, wma_period, time_col_name, price_col_name) ``` The Coppock Curve is a long-term price momentum indicator used primarily to recognize major downturns and upturns in a stock market index. It is calculated as a 10-month weighted moving average of the sum of the 14-month rate of change and the 11-month rate of change for the index. It is also known as the Coppock Guide. **Parameters** + **roc1\$1period** (int) – rate of change 1 look back period + **roc2\$1period** (int) – rate of change 2 look back period + **wma\$1period** (int) – weighted moving average look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of high prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Debug UDF call ``` aws.finspace.timeseries.spark.analytics.debug_udf_call(tenor, time_col_name, *kwargs) ``` **Return type** `Callable[. . . , Column]` ## Directional movement indicators (DMI) ``` aws.finspace.timeseries.spark.analytics.dmi_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Directional Movement Indicators (DMI) are components of the Directional Movement system published by J. Welles Wilder, and are computed with the Average Directional Movement Index (ADX). Two indicators are plotted, a Positive DI ( \$1DI ) and a Negative DI ( -DI ). **Parameters** + `tenor` (int) – window size, typically 2 periods + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Donchian channels ``` aws.finspace.timeseries.spark.analytics.donchian_channel_indicator(tenor, time_col_name, high_col_name, low_col_name) ``` Donchian Channels are three lines generated by moving average calculations that comprise an indicator formed by upper and lower bands around a mid-range or median band. The upper band marks the highest price of a security over N periods while the lower band marks the lowest price of a security over N periods. The area between the upper and lower bands represents the Donchian Channel. Career futures trader Richard Donchian developed the indicator in the mid-twentieth century to help him identify trends. He would later be nicknamed **The Father of Trend Following**. **Parameters** + `tenor` \$1look back + `time_col_name` \$1name of time column + `high_col_name` \$1input array of high prices over bar + `low_col_name` input array of low prices over bar **Return type** \$1Callabl[. . . , Column] **Return** pandas/spark user defined scalar function ## Double exponential moving average (DEMA) ``` aws.finspace.timeseries.spark.analytics.double_exponential_moving_average(tenor, time_col_name, price_col_name) ``` The Double Exponential Moving Average (DEMA) by Patrick Mulloy attempts to offer a smoothed average with less lag than a straight exponential moving average. The calculation is more complex than just a moving average of a moving average as shown in the formula below. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Detrended price oscillator (DPO) ``` aws.finspace.timeseries.spark.analytics.dpo_indicator(tenor, time_col_name, price_col_name) ``` A detrended price oscillator is an oscillator that strips out price trends in an effort to estimate the length of price cycles from peak to peak or trough to trough. Unlike other oscillators, such as the stochastic or moving average convergence divergence (MACD), the DPO is not a momentum indicator. It highlights peaks and troughs in price, which are used to estimate buy and sell points in line with the historical cycle. **Parameters** + `tenor` – look back period + `time_col_name` – name of time column + `high_col_name` – input array of high prices over bar + `low_col_name` – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Return** pandas/spark user defined scalar function ## Ease of movement indicator ``` aws.finspace.timeseries.spark.analytics.ease_of_movement_indicator(tenor, time_col_name, high_col_name, low_col_name, vol- ume_col_name, scale=1000000) ``` Richard Arms’ Ease of Movement indicator is a technical study that attempts to quantify a mix of momentum and volume information into one value. The intent is to use this value to discern whether prices are able to rise, or fall, with little resistance in the directional movement. Theoretically, if prices move easily, they will continue to do so for a period of time that can be traded effectively. **Parameters** + `tenor` – look back window + `time_col_name` – name of time column + `high_col_name` – input array of high prices over bar + `low_col_name` – input array of low prices over bar + `volume_col_name` – input array of total volume over bar + `scale` – scale multiplier **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Elder-ray index ``` aws.finspace.timeseries.spark.analytics.elder_ray_index_indicator(tenor, time_col_name, price_col_name) ``` The Elder-Ray Index is a technical indicator developed by Dr. Alexander Elder that measures the amount of buying and selling pressure in a market. This indicator consists of three separate indicators known as **bull power** and **bear power**, which are derived from a 13-period exponential moving average (EMA). The three indicator help traders determine the trend direction and isolate spots to enter and exit trades. **Parameters** + `tenor` (int) – look back + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Exponential moving average ``` aws.finspace.timeseries.spark.analytics.exponential_moving_average(tenor, time_col_name, input_array_col_name) ``` Compute exponential moving average on entire data set. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Stochastic fast (StochF) ``` aws.finspace.timeseries.spark.analytics.fast_stock_oscillator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Stochastic Fast (StochF) normalizes price as a percentage between 0 and 100. Normally two lines are plotted, the %K line and a 3 day moving average of the %K which is called %D. A fast stochastic is created by not smoothing the %K line with a moving average before it is displayed. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Fisher transform ``` aws.finspace.timeseries.spark.analytics.fisher_transformation_indicator(tenor, time_col_name, high_col_name, low_col_name) ``` The Fisher Transform is a technical indicator created by J.F. Ehlers that converts prices into a Gaussian normal distribution. In this way, the indicator highlights when prices have moved to an extreme, based on recent prices. This may help in spotting turning points in the price of an asset. It also helps show the trend and isolate the price waves within a trend. **Parameters** + `time_col_name` (str) – name of time column + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Force index ``` aws.finspace.timeseries.spark.analytics.force_index_indicator(tenor, time_col_name, price_col_name, volume_col_name) ``` The force index is a technical indicator that measures the amount of power used to move the price of an asset. The term and its formula were developed by psychologist and trader AlexanderElder and published in his 1993 book Trading for a Living. The force index uses price and volume to determine the amount of strength behind a price move. The index is an oscillator, fluctuating between positive and negative territory. It is unbounded meaning the index can go up or down indefinitely. **Parameters** + `tenor` – look back window + `time_col_name` – name of time column + `price_col_name` – input array of closing prices over bar + `volume_col_name` – input array of total volume over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Hull moving average (HMA) ``` aws.finspace.timeseries.spark.analytics.hull_moving_average_indicator(tenor, time_col_name, price_col_name) ``` The Hull Moving Average (HMA) was developed by Alan Hull for the purpose of reducing lag, increasing responsiveness while at the same time eliminating noise. Its calculation is elaborate and makes use of the Weighted Moving Average (WMA). It emphasizes recent prices over older ones, resulting in a fast-acting yet smooth moving average that can be used to identify the prevailing market trend. **Parameters** + `tenor` (int) – look back + `time_col_name` (str) – name of time column + `price_col_name` – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Ichimoku study ``` aws.finspace.timeseries.spark.analytics.ichimoku_indicator(time_col_name, price_col_name, short_period=9, medium_period=26, long_period=52) ``` The Ichimoku study was developed by Goichi Hosoda pre-World War II as a forecasting model for financial markets. The study is a trend following indicator that identifies mid-points of historical highs and lows at different lengths of time and generates trading signals similar to that of moving averages/MACD. A key difference between Ichimoku and moving averages is Ichimoku charts lines are shifted forward in time creating wider support/resistance areas mitigating the risk of false breakouts. Learn [more](https://www.investopedia.com/terms/i/ichimoku-cloud.asp). **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `short_period` (int) – short period window, usually 9 + `medium_period` (int) – long period window, usually 26 + `long_period` (int) – long period window, usually 52 **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Kaufman adaptive moving average (KAMA) ``` aws.finspace.timeseries.spark.analytics.kama_indicator(time_col_name, price_col_name, er_period=10, fast_ema_period=30, slow_ema_period=2) ``` Developed by Perry Kaufman, Kaufman's Adaptive Moving Average (KAMA) is a moving average designed to account for market noise or volatility. KAMA will closely follow prices when the price swings are relatively small and the noise is low. KAMA will adjust when the price swings widen and follow prices from a greater distance. This trend-following indicator can be used to identify the overall trend, time turning points and filter price movements. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices over bar + **er\$1period** (int) – efficiency ratio tenor + **slow\$1ema\$1period** (int) + **fast\$1ema\$1period** (int) **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Keltner channel ``` aws.finspace.timeseries.spark.analytics.keltner_indicator(time_col_name, price_col_name, high_col_name, low_col_name, atr_factor=2, ewm_tenor=20, atr_tenor=20) ``` The Keltner Channel was introduced in 1960 by Chester W. Keltner in his book **How To Make Money in Commodities**, and is also explained by Perry Kaufman's book **The New Commodity Trading Systems and Methods**. Keltner Channels plots three lines, consisting of a exponential moving average (typically of the average price) with upper and lower bands plotted above and below this moving average.The width of the bands is based on a user defined factor applied to the Average True Range, with this result added to and subtracted from the middle moving average line. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price + **atr\$1factor** (float) – ATR multiplier + **ewm\$1tenor** (int) – tenor of expo moving average + **atr\$1tenor** (int) – tenor of ATR **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Klinger oscillator indicator ``` aws.finspace.timeseries.spark.analytics.klinger_oscillator_indicator(short_period, long_period, time_col_name, price_col_name, high_col_name, low_col_name, volume_col_name) ``` The indicator was developed by Stephen Klinger to determine the long-term trend of money flow while remaining sensitive enough to detect short-term fluctuations. The indicator compares the volume flowing through securities with the security price movements and then converts the result into an oscillator. The Klinger oscillator shows the difference between two moving averages which are based on more than price. Traders watch for divergence on the indicator to signal potential price reversals. Like other oscillators, a signal line can be added to provide additional trade signals. **Parameters** + `short_period` (int) – short exponential moving average look back typically 34 + `long_period` (int) – long exponential moving average look back typically 55 + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of high prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar + `volume_col_name` (str) – input array of total volume over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Linear regression ``` aws.finspace.timeseries.spark.analytics.linear_regression(tenor, time_col_name, input1_col_name, input2_col_name) ``` Takes two arrays of asset prices and then produces slope and intercept, where input1\$1col\$1name is the independent variable and input2\$1col\$1name is the dependent variable. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input1_col_name` (str) – name of input array column + `input2_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Linear weighted moving average ``` aws.finspace.timeseries.spark.analytics.linear_weighted_moving_average(tenor, time_col_name, price_col_name) ``` Learn [more](https://www.investopedia.com/terms/l/linearlyweightedmovingaverage.asp). **Parameters** + `tenor` (int) – interval length + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Mass index ``` aws.finspace.timeseries.spark.analytics.mass_index_indicator(ema_period, sum_period, time_col_name, high_col_name, low_col_name) ``` Mass index is a form of technical analysis that examines the range between high and low stock prices over a period of time. Mass index, developed by Donald Dorsey in the early 1990s, suggests that a reversal of the current trend will likely take place when the range widens beyond a certain point and then contracts. **Parameters** + `ema_period` – look back for exponential moving average typically 9 periods + `sum_period` – look back for sum of exponential moving average typically 25 periods + `time_col_name` – name of time column + `high_col_name` – input array of high prices over bar + `low_col_name` – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Max indicator ``` aws.finspace.timeseries.spark.analytics.max_indicator(tenor, time_col_name, price_col_name) ``` Compute max over look back period **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Money flow indicator ``` aws.finspace.timeseries.spark.analytics.mf_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name, vol ume_col_name) ``` The Money Flow Index (MFI) was developed by Gene Quong and Avrum Soudack. It uses both price and volume to measure buying and selling pressure. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices over bar + `low_col_name` (str) – input array of low asset price over bar + `volume_col_name` (str) – input array of total volume over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## MidPoint indicator ``` aws.finspace.timeseries.spark.analytics.midpoint_indicator(tenor, time_col_name, price_col_name) ``` The Midpoint calculation is similar to the Midprice, except the highest and lowest values are returned from the same input field. The default indicator calculates the highest close and lowest close within the look back period and averages the two values. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## MidPrice indicator ``` aws.finspace.timeseries.spark.analytics.midprice_indicator(tenor, time_col_name, high_col_name, low_col_name) ``` The Midprice returns the midpoint value from two different input fields. The default indicator calculates the highest high and lowest low within the look back period and averages the two values to return the Midprice. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Min ``` aws.finspace.timeseries.spark.analytics.*min_indicator*(tenor, time_col_name, price_col_name) ``` Compute minimum value over look back period. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Min and max over period ``` aws.finspace.timeseries.spark.analytics.minmax_indicator(tenor, time_col_name, price_col_name) ``` Min and Max over a tenor period. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Momentum ``` aws.finspace.timeseries.spark.analytics.momentum_indicator(tenor, time_col_name, input_array_col_name) ``` Compute momentum **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Moving average ``` aws.finspace.timeseries.spark.analytics.moving_average(tenor, time_col_name, input_array_col_name) ``` Compute moving average on window of tenor size utilizing function average\$1at\$1point **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Moving average convergence divergence (MACD) ``` aws.finspace.timeseries.spark.analytics.moving_average_converge_diverge(short_tenor, long_tenor, time_col_name, input_array_col_name) ``` The Moving Average Convergence Divergence (MACD) was developed by Gerald Appel, and is based on the differences between two moving averages of different lengths, a Fast and a Slow moving average. A second line, called the Signa line is plotted as a moving average of the MACD. A third line, called the MACD Histogram is optionally plotted as a histogram of the difference between the MACD and the Signal Line. Learn [more](https://www.investopedia.com/terms/m/macd.asp). **Parameters** + `short_tenor` (int) – short window size + `long_tenor` (int) – long window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Moving average convergence divergence historical (MACD) ``` aws.finspace.timeseries.spark.analytics.moving_average_converge_diverge_hist(short_tenor, long_tenor, signal_line_tenor, time_col_name, input_array_col_name) ``` **Parameters** + `short_tenor` (int) – short window size + `long_tenor` (int) – long window size + `signal_line_tenor` (int) – signal line tenor size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Normalized average true range (NATR) ``` aws.finspace.timeseries.spark.analytics.natr_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` Normalized Average True Range (NATR) attempts to normalize the average true range values across instruments by using the formula below. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Directional movement (DM) ``` aws.finspace.timeseries.spark.analytics.neg_dm_indicator(time_col_name, high_col_name, low_col_name) ``` Directional Movement (DM) is defined as the largest part of the current period price range that lies outside the previous period price range. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Negative volume index (NVI) ``` aws.finspace.timeseries.spark.analytics.negative_volume_indicator(period, time_col_name, price_col_name, volume_col_name) ``` The Negative Volume Index (NVI) is a cumulative indicator that uses the change in volume to decide when the smart money is active. Paul Dysart first developed this indicator in the 1930s. Dysart's Negative Volume Index works under the assumption that the smart money is active on days when volume decreases and the not-so-smart money is active on days when volume increases. **Parameters** + `period`(int) – returns period typically 1 + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `volume_col_name` (str) – input array of total volume over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Balance volume (OBV) ``` aws.finspace.timeseries.spark.analytics.on_balance_volume(time_col_name, price_col_name, volume_col_name) ``` On Balance Volume (OBV) maintains a cumulative running total of the amount of volume occurring on up periods compared to down periods. Learn [more](https://www.investopedia.com/terms/o/onbalancevolume.asp). **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – array of asset prices + `volume_col_name` (str) – array of volume associated with prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Pairwise realized correlation ``` aws.finspace.timeseries.spark.analytics.pairwise_realized_correlation(tenor, time_col_name, asset1_col_name, asset2_col_name) ``` Takes two arrays of asset prices and then produces a pairwise correlation, returns nan if the array is less than window. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `asset1_col_name` (str) – name of input array column + `asset2_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Percent price oscillator (PPO) ``` aws.finspace.timeseries.spark.analytics.percentage_price_oscillator(short_period, long_period, signal_line_period, time_col_name, price_col_name) ``` Compute Percentage price oscillator. The Percent Price Oscillator (PPO) is based on the differences between two moving averages of different lengths, a Fast and a Slow moving average. The PPO is the difference of the two averages divided by the slower of the two moving averages, which tends to normalize the values across different instruments. **Parameters** + `short_period` (int) – slow ema period + `long_period` (int) – slow ema period + `signal_line_period` (int) – signal line ema period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Directional movement (DM) ``` aws.finspace.timeseries.spark.analytics.pos_dm_indicator(time_col_name, high_col_name, low_col_name) ``` Directional Movement (DM) is defined as the largest part of the current period's price range that lies outside the previous period's price range. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Price channel ``` aws.finspace.timeseries.spark.analytics.price_channel_indicator(tenor, time_col_name, price_col_name) ``` The Price Channel displays two lines, with the upper line representing the highest price and the lower line representing the lowest price for a given look back interval. The bands can optionally be smoothed with a moving average, or shifted to the right with an offset value. Basic uses of the Price Channel are to identify breakouts form the channel and to determine placement of trailing stops. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Price volume trend (PVT) ``` aws.finspace.timeseries.spark.analytics.pvt_indicator(time_col_name, price_col_name, volume_col_name) ``` Compute price volume indicator. The Price Volume Trend (PVT) study attempts to quantify the amount of volume flowing into or out of an instrument by identifying the close of the period in relation to the previous period's close. The volume for the period is then allocated accordingly to a running continuous total. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `volume_col_name` (str) – input array of volumes at the asset prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Realized correlation matrix ``` aws.finspace.timeseries.spark.analytics.realized_correlation_matrix(tenor, time_col_name, *kwargs) ``` **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Realized volatility ``` aws.finspace.timeseries.spark.analytics.realized_volatility(tenor, time_col_name, asset_price_col_name) ``` Takes in an array of asset prices and computes realized volatility. Learn [more](https://en.wikipedia.org/wiki/Realized_variance). **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + **asset\$1price\$1col\$1name** – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Realized volatiliy spread ``` aws.finspace.timeseries.spark.analytics.realized_volatility_spread(tenor, time_col_name, asset1_col_name, asset2_col_name) ``` Compute realized volatility spread between two assets. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `asset1_col_name` (str) – name of input array column + `asset2_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Rate of change (ROC) ``` aws.finspace.timeseries.spark.analytics.roc_indicator(tenor, time_col_name, input_array_col_name) ``` The Rate of Change (ROC) indicator compares the current price with the previous price from a selected number of periods ago. The current price is divided by the previous price and expressed as a percentage. This indicator is also commonly known as a momentum indicator. Learn [more](https://www.investopedia.com/terms/p/pricerateofchange.asp). **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Relative strength index (RSI) ``` aws.finspace.timeseries.spark.analytics.rsi(tenor, time_col_name, input_array_col_name) ``` The Relative StrengthIndex (RSI) was published by J. Welles Wilder. The current price is normalized as a percentage between 0 and 100. The name of this oscillator is misleading because it does not compare the instrument relative to another instrument or set of instruments, but rather represents the current price relative to other recent pieces within the selected look back window length. Learn [more](https://www.investopedia.com/terms/r/rsi.asp). **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `input_array_col_name` (str) – name of input array column **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Parabolic stop and reverse (SAR) ``` aws.finspace.timeseries.spark.analytics.sar_indicator(time_col_name, price_col_name, high_col_name, low_col_name, period=2, af=0.02, sar_rising=True) ``` The Parabolic Stop and Reverse (SAR) calculates trailing stop points to use with long and short positions. The SAR was published by J. Welles Wilderas part of a complete trend following system. The dotted lines above the price designate trailing stops for short positions; those below the price are sell stops for long positions. **Parameters** + `time_col_name` – name of time column + `price_col_name` – input array of closing prices over bar + `high_col_name` – input array of high prices over bar + `low_col_name` – input array of low prices over bar + `period` – window size **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Slow stock oscillator ``` aws.finspace.timeseries.spark.analytics.slow_stock_oscillator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` The Stochastic (Stoch) normalizes price as a percentage between 0 and 100. Normally two lines are plotted, the %K line and a moving average of the %K which is called %D. A slow stochastic can be created by initially smoothing the %K line with a moving average before it is displayed. The length of this smoothing is set in the Slow K Period. Without the initial smoothing (i.e., setting the Slow K Period to a valueof 1 ) the %K becomes the **Raw %K** value, and is also known as a fast stochastic. Learn [https://www.investopedia.com/terms/s/stochasticoscillator.asp](https://www.investopedia.com/terms/s/stochasticoscillator.asp) [more]. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Standard deviation indicator ``` aws.finspace.timeseries.spark.analytics.stddev_indicator(tenor, time_col_name, price_col_name) ``` Compute standard deviation over a window. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Stochastic RSI (StochRSI) ``` aws.finspace.timeseries.spark.analytics.stoch_rsi_indicator(rsi_tenor, tenor, time_col_name, price_col_name) ``` The Stochastic RSI (StochRSI) is an indicator used in technical analysis that ranges between zero and one (or zero and 100 on some charting platforms) and is created by applying the Stochastic oscillator formula to a set of relative strength index (RSI) values rather than to standard price data. Using RSI values within the Stochastic formula gives traders an idea of whether the current RSI value is overbought or oversold. **Parameters** + **rsi\$1tenor** (int) – window size for rsi + `tenor` (int) – window size for stoch rsi + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Triple exponential moving average (T3) ``` aws.finspace.timeseries.spark.analytics.t3_ema_indicator(tenor, time_col_name, price_col_name) ``` The Triple Exponential Moving Average (T3) by Tim Tillson attempts to offers a moving average with better smoothing then traditional exponential moving average. `EMA1 = EMA(x,Period) EMA2 = EMA(EMA1,Period) GD = EMA1*(1+vFactor)) - (EMA2*vFactor) T3 = GD (GD ( GD(t, Period, vFactor), Period, vFactor), Period, vFactor)` Where vFactor is a volume factor between 0 and 1 which determines how the moving averages responds. A value of 0 returns an EMA. A value of 1 returns DEMA. Tim Tillson advised or preferred a value of 0.7. **Parameters** + `tenor` – window size + `time_col_name` – name of time column + `price_col_name` – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Time series forecast (TSF) ``` aws.finspace.timeseries.spark.analytics.time_series_forecast_indicator(tenor, scale, time_col_name, time_axis_col_name, price_col_name) ``` The Time Series Forecast (TSF) indicator displays the statistical trend of a security's price over a specified time period. The trend is based on linear regression analysis. Rather than plotting a straight linear regression trend line, the Time Series Forecast plots the last point of multiple linear regression trend lines. The difference between the TSF and the moving linear regression is that the TSF adds the slope of the linear regression to the linear regression essentially projecting the position of the linear regression forward one period. **Parameters** + `tenor` (str) – look back + `scale` (str) – time scale, either minutes or seconds or days + `time_col_name` (str) – name of time column + `time_axis_col_name` (str) – name of independent time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## True range ``` aws.finspace.timeseries.spark.analytics.tr_indicator(time_col_name, price_col_name, high_col_name, low_col_name) ``` Calculate True Range. **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Triangular simple moving average ``` aws.finspace.timeseries.spark.analytics.trima_indicator(tenor, time_col_name, price_col_name) ``` Triangular Simple Moving Average. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Triple exponential moving average (TEMA) ``` aws.finspace.timeseries.spark.analytics.triple_exponential_moving_average(tenor,time_col_name, price_col_name) ``` The Triple Exponential Moving Average (TEMA) by Patrick Mulloy offers a moving average with less lag than traditional exponential moving average. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Triple exponential moving average oscillator (TRIX) ``` aws.finspace.timeseries.spark.analytics.trix_indicator(tenor, time_col_name, price_col_name) ``` The Triple Exponential Moving AverageOscillator (TRIX) by Jack Hutson is a momentum indicator that oscillates around zero. It displays the percentage rate of change between two triple smoothed exponential moving averages. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Timeseries linear regression ``` aws.finspace.timeseries.spark.analytics.ts_linear_regression(tenor, scale, time_col_name, time_axis_col_name, value_axis_col_name) ``` Takes two arrays, the first is the time axis, and the second is the value axis and then produces slope and intercept, where time axis is the independent variable and arr2 is the dependent variable. **Parameters** + `tenor` (int) – look back period + `scale`(str) – scale of time axis, can take value of seconds, or minutes + `time_col_name` (str) – name of time column + `time_axis_col_name` (str) – input array of datetime + `value_axis_col_name` (str) input array of values **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Typical price ``` aws.finspace.timeseries.spark.analytics.typical_price_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` Typical Price calculation defined as `(High + Low + Close) / 3`. **Parameters** + `tenor` (int) – look back window + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Ult oscillator ``` aws.finspace.timeseries.spark.analytics.ult_osc_indicator(time_col_name, price_col_name, high_col_name, low_col_name) ``` **Parameters** + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Weighted close price ``` aws.finspace.timeseries.spark.analytics.weighted_close_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` Weighted Close Price calculation defined as `(High+ Low + 2*Close) / 4` **Parameters** + `tenor` (int) – look back window + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar + `high_col_name` (str) – input array of high prices over bar + `low_col_name` (str) – input array of low prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Weighted linear regression ``` aws.finspace.timeseries.spark.analytics.weighted_linear_regression(tenor, time_col_name, input_arr1_col_name_, input_arr2_col_name_, weights_col_name) ``` Takes three arrays, the first is the independent axis, and the second is the value axis and third is weights and then produces slope and intercept. **Parameters** + `tenor`(str) – look back period + `time_col_name` (str) – name of time column + `input_arr1_col_name` – input array of independent variables + `input_arr2_col_name` – input array of dependent variables + `weights_col_name` (str) – input array of weights **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Weighted TS linear regression ``` aws.finspace.timeseries.spark.analytics.weighted_ts_linear_regression(tenor, scale, time_col_name, time_axis_col_name, value_axis_col_name, weights_axis_col_name) ``` Takes three arrays, the first is the time axis, and the second is the value axis and third is weights and then produces slope and intercept, where time axis is the independent variable and value\$1axis is the dependent variable, and weights. **Parameters** + `tenor`(str) – look back period + `scale` (str) – scale of time axis, can take value of seconds, or minutes + `time_col_name` (str) – name of time column + `time_axis_col_name` (str) – input array of datetime + `value_axis_col_name` (str) – input array of values + `weights_axis_col_name` (str) – input array of weights **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Welles wilder smoothing average (WWS) ``` aws.finspace.timeseries.spark.analytics.wilder_smoothing_indicator(tenor, time_col_name, price_col_name) ``` The Welles Wilder's Smoothing Average (WWS) was developed by J. Welles Wilder, Jr. and is part of the Wilder's RSI indicator implementation. This indicator smoothes price movements to help you identify and spot bullish and bearish trends. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of closing prices over bar **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## %R ``` aws.finspace.timeseries.spark.analytics.will_r_indicator(tenor, time_col_name, price_col_name, high_col_name, low_col_name) ``` Compute william %R indicator. The %R indicator was developed by Larry Williams and introduced in his 1979 book **How I Made \$11,000,000 Trading Commodities Last Year**. Williams %R is similar to a stochastic oscillator, as it normalizes price as a percentage between 0 and 100. It is basically an inverted version of the **Raw %K** value of a Fast Stochastic. **Parameters** + `tenor` (int) – look back period + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of asset prices, determined by user, default is close for intraday calculations + `high_col_name` (str) – input array of high asset prices + `low_col_name` (str) – input array of high asset price **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Rate of change (ROC100) ``` aws.finspace.timeseries.spark.analytics.ROC100_indicator(tenor, time_col_name, price_col_name) ``` The Rate of Change (ROC100) indicator compares the current price with the previous price from a selected number of periods ago. The current price is divided by the previous price and multiplied by 100. This indicator is also commonly known as a momentum indicator. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/Spark user defined scalar function ## Rate of change percentage (ROCP) ``` aws.finspace.timeseries.spark.analytics.ROCP_indicator(tenor, time_col_name, price_col_name) ``` The Rate of Change Percentage (ROCP) indicator compares the current price with the previous price from a selected number of periods ago. The current price is divided by the previous price. ROCP is not expressed as a percentage. This indicator is also commonly known as a momentum indicator. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/spark user defined scalar function ## Rate of change rate (ROCR) ``` aws.finspace.timeseries.spark.analytics.ROCR_indicator(tenor, time_col_name, price_col_name) ``` The Rate of Change Rate (ROCR) indicator compares the current price with the previous price from a selected number of periods ago. The current price is divided by the previous price. This indicator is also commonly known as a momentum indicator. **Parameters** + `tenor` (int) – window size + `time_col_name` (str) – name of time column + `price_col_name` (str) – input array of prices **Return type** `Callable[. . . , Column]` **Returns** pandas/spark user defined scalar function # Using the Amazon FinSpace library **Important** Amazon FinSpace Dataset Browser will be discontinued on *March 26, 2025*. Starting *November 29, 2023*, FinSpace will no longer accept the creation of new Dataset Browser environments. Customers using [Amazon FinSpace with Managed Kdb Insights](https://aws.amazon.com/finspace/features/managed-kdb-insights/) will not be affected. For more information, review the [FAQ](https://aws.amazon.com/finspace/faqs/) or contact [AWS Support](https://aws.amazon.com/contact-us/) to assist with your transition. This following section provides a step-by-step example on how to use the time series library across all stage in the framework, using the **US Equity TAQ 6 months, AMZN Symbol** dataset available with the [sample capital markets data bundle](sample-data-bundle.md) with Amazon FinSpace. ![\[time series library 1\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-1.png) **Events** – The Data View now loaded into the DataFrame contains raw data events. The DataFrame is filtered on `ticker`, `eventtype`, `datetime`, `price`, `quantity`, `exchange`, `conditions` fields. ![\[time series library 2\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-2.png) **Collect Bars** – In this stage, the FinSpace create\$1time\$1bars function is used to collect raw data events into 1-minute time bars. ![\[time series library 3\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-3.png) The window represents the 1-min time interval for the bar. The Activity count shows the number of events collected in each bar. Note that the data events collected inside the bar are not shown. ![\[time series library 4\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-4.png) **Summarize Bars** – In this stage, the FinSpace summarize functions are applied to calculate 1-minute summaries of events collected in bars. Summaries are created for two-point standard deviation, Volume Weighted Average Price, open(first), high, low, close(last) prices commonly referred as OHLC. ![\[time series library 5\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-5.png) The activity count shows the number of events summarized in a single summary bar. ![\[time series library 6\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-6.png) **Fill & Filter** – The resulting data set is filtered according to an exchange trading calendar. ![\[time series library 7\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-7.png) The schema is simplified to prepare a dataset of features. VWAP and standard deviation calculations are now displayed as well. ![\[time series library 8\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-8.png) **Apply Analytics** – FinSpace Bollinger Bands function is applied on the features dataset. Note that the tenor window to perform the calculation is 15 which means that the calculation is applied when 15 data events are available. As each event corresponds to a 1-min summary bar in the features dataset, the resulting dataset starts from timestamp 09:45 (see end column). ![\[time series library 9\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-9.png) ![\[time series library 10\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-10.png) You can plot the output into a chart using matplotlib. The chart shows the Bollinger Bands for the entire 3 month history for AMZN. ![\[time series library 11\]](http://docs.aws.amazon.com/finspace/latest/userguide/images/07-prepare-and-analyze-data/time-series-library-11.png) FinSpace time series library is provided with `aws.finspace.timeseries.spark` package used when working with data that will be processed using a FinSpace Spark cluster in the FinSpace notebook.