# Work with dataflows AWS Ground Station uses a *node* and *edge* relationship to construct *dataflows* to enable stream processing of your data. Each node is represented by a *config* which describes its expected processing. To illustrate this concept, consider a dataflow of `antenna-downlink` to a `s3-recording`. The `antenna-downlink` node represents the analog to digital transformation of the radio frequency spectrum per the defined parameters on the config. The `s3-recording` represents a compute node which will receive incoming data and store it in your S3 bucket. The resulting dataflow is an asynchronous data delivery of digitized RF data to an S3 bucket based on your specifications. Within your mission profile, you can create many dataflows to meet your needs. The following sections describe how to set up your other AWS resources to be used with AWS Ground Station and offers recommendations for constructing dataflows. For detailed information on how each node behaves, including if it is considered a source or destination node, please see [Use AWS Ground Station Configs](how-it-works.config.md). **Topics** + [AWS Ground Station data plane interfaces](#dataflows.interfaces) + [Use cross-region data delivery](dataflows.cross-region-data-delivery.md) + [Set up and configure Amazon S3](dataflows.s3-configuration.md) + [Set up and configure Amazon VPC](dataflows.vpc-configuration.md) + [Set up and configure Amazon EC2](dataflows.ec2-configuration.md) ## AWS Ground Station data plane interfaces The resulting data structure of your chosen dataflow depends on the source of the dataflow. Details of these formats are provided to you during the onboarding of your satellites. The following summarizes the formats used for each type of dataflow. + **antenna-downlink** + (Bandwidth less-than-or-equal-to 40MHz) data is delivered as [VITA-49 Signal Data/IP](https://www.vita.com/Standards) Format packets. + (Bandwidth greater-than 40MHz) data is delivered as AWS Ground Station Class 2 packets. + **antenna-downlink-demod-decode** + Data is delivered as Demodulated/Decoded Data/IP Format packets. + **antenna-uplink** + Data must be delivered as [VITA-49 Signal Data/IP](https://www.vita.com/Standards) Format packets. + **antenna-uplink-echo** + Data is delivered as [VITA-49 Signal Data/IP](https://www.vita.com/Standards) Format packets. # Use cross-region data delivery The AWS Ground Station cross-region data delivery feature gives you the flexibility to send your data from an antenna to any AWS Ground Station supported AWS Region. This means you can maintain your infrastructure in a single AWS Region and schedule contacts on any [AWS Ground Station Locations](aws-ground-station-antenna-locations.md) you are onboarded to. When receiving your contact data in an Amazon S3 Bucket, AWS Ground Station will manage all delivery aspects for you. To use cross-region data delivery to an Amazon EC2 instance (using either the AWS Ground Station Agent or a dataflow endpoint), the *dataflow-endpoint* must be created in your current AWS Region and your *dataflow-endpoint-config* must specify the same region. AWS Ground Station will manage delivering the data cross-region for you. # Set up and configure Amazon S3 You can utilize a Amazon S3 bucket to receive your downlink signals using AWS Ground Station. To create the destination *s3-recording-config*, you must be able to specify a Amazon S3 bucket and an IAM role which authorizes AWS Ground Station to write files to the bucket. See [Amazon S3 Recording Config](how-it-works.config.md#how-it-works.config-s3-recording) for restrictions on the Amazon S3 bucket, IAM role, or AWS Ground Station config creation. # Set up and configure Amazon VPC A full guide to set up a VPC is beyond the scope of this guide. For an in-depth understanding please refer to the [Amazon VPC User Guide](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html). In this section, it is described how your Amazon EC2 and dataflow endpoint may exist within a VPC. AWS Ground Station does not support multiple delivery points for a given dataflow - it is expected that each dataflow terminates to a single EC2 receiver. As we expect a single EC2 receiver, the configuration is not multi-AZ redundant. For full examples which will use your VPC, please see [Example mission profile configurations](examples.md). ## VPC Configuration with AWS Ground Station Agent ![\[AWS Ground Station architecture with VPC, private and public subnets, and Amazon EC2 instance.\]](http://docs.aws.amazon.com/ground-station/latest/ug/images/dataflows.vpc-gs-agent.png) Your satellite data is provided to an AWS Ground Station Agent instance that is proximate to the antenna. The AWS Ground Station Agent will stripe and then encrypt your data using the AWS KMS key you provide. Each stripe is sent to your [ Amazon EC2 Elastic IP (EIP) ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html) from the source antenna across the AWS Network backbone. The data arrives at your EC2 instance via the [Amazon EC2 Elastic Network Interface (ENI) ](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) attached. Once on your EC2 instance, the installed AWS Ground Station Agent will decrypt your data and perform forward error correction (FEC) to recover any dropped data, then forward it to the IP and port you specified in your setup. The below list calls out unique setup considerations when setting up your VPC for AWS Ground Station Agent delivery. **Security Group** - It is recommended you set up a security group dedicated to only AWS Ground Station traffic. This security group should allow UDP ingress traffic on the same port range you specify in your Dataflow Endpoint Group. AWS Ground Station maintains an AWS-managed prefix list to restrict your permissions to only AWS Ground Station IP addresses. See [AWS Managed Prefix Lists ](https://docs.aws.amazon.com/vpc/latest/userguide/working-with-aws-managed-prefix-lists.html) for details on how to replace the *PrefixListId* for your deployment regions. **Elastic Network Interface (ENI)** - You will need to associate the above security group with this ENI and place it in your public subnet. **Note** The default quota for number of security groups attached per ENI is 5. This is an adjustable limit up to 16, see [Amazon VPC Quotas](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html#vpc-limits-security-groups). The following CloudFormation template demonstrates how to create the infrastructure described in this section. ``` ReceiveInstanceEIP: Type: AWS::EC2::EIP Properties: Domain: 'vpc' InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: AWS Ground Station receiver instance security group. VpcId:YourVpcId SecurityGroupIngress: # Add additional items here. - IpProtocol: udp FromPort: your-port-start-range ToPort: your-port-end-range PrefixListIds: - PrefixListId: com.amazonaws.global.groundstation Description: "Allow AWS Ground Station Downlink ingress." InstanceNetworkInterface: Type: AWS::EC2::NetworkInterface Properties: Description: ENI for AWS Ground Station to connect to. GroupSet: - !Ref InstanceSecurityGroup SubnetId: A Public Subnet ReceiveInstanceEIPAllocation: Type: AWS::EC2::EIPAssociation Properties: AllocationId: Fn::GetAtt: [ ReceiveInstanceEIP, AllocationId ] NetworkInterfaceId: Ref: InstanceNetworkInterface ``` ## VPC configuration with a dataflow endpoint ![\[Diagram showing two VPCs with Amazon EC2 instances running endpoint applications.\]](http://docs.aws.amazon.com/ground-station/latest/ug/images/dataflows.vpc-dataflow-endpoint-application.png) Your satellite data is provided to a dataflow endpoint application instance that is proximate to the antenna. The data is then sent through cross-account [Amazon EC2 Elastic Network Interface (ENI)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html) from a VPC owned by AWS Ground Station. The data then arrives at your EC2 instance via the ENI attached to your Amazon EC2 instance. The installed dataflow endpoint application will then forward it to the IP and port you specified in your setup. The reverse of this flow occurs for uplink connections. The below list calls out unique setup considerations when setting up your VPC for dataflow endpoint delivery. **Note** The default quota for number of security groups attached per ENI is 5. This is an adjustable limit up to 16, see [Amazon VPC Quotas](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html#vpc-limits-security-groups). **IAM Role** - The IAM Role is part of the Dataflow Endpoint and is not shown in the diagram. The IAM role that is used to create and attach the cross-account ENI to the AWS Ground Station Amazon EC2 instance. **Security Group 1** - This security group is attached to the ENI which will be associated to the Amazon EC2 instance in your account. It needs to allow UDP traffic from Security Group 2 on the ports specified in your *dataflow-endpoint-group*. **Elastic Network Interface (ENI) 1** - You will need to associate Security Group 1 with this ENI and place it in a subnet. **Subnet** - You will need to ensure that there is at least one available IP address per dataflow for the Amazon EC2 instance in your account. For more details on subnet sizing see, [Subnet CIDR blocks](https://docs.aws.amazon.com/vpc/latest/userguide/subnet-sizing.html) **Security Group 2** - This security group is referenced in the Dataflow Endpoint. This security group will be attached to the ENI that AWS Ground Station will use to place data into your account. **Region** - For more information on the supported regions for cross-region connections, see [Use cross-region data delivery](dataflows.cross-region-data-delivery.md). The following CloudFormation template demonstrates how to create the infrastructure described in this section. ``` DataflowEndpointSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security Group for AWS Ground Station registration of Dataflow Endpoint Groups VpcId: YourVpcId AWSGroundStationSecurityGroupEgress: Type: AWS::EC2::SecurityGroupEgress Properties: GroupId: !Ref: DataflowEndpointSecurityGroup IpProtocol: udp FromPort: 55555 ToPort: 55555 CidrIp: 10.0.0.0/8 Description: "Allow AWS Ground Station to send UDP traffic on port 55555 to the 10/8 range." InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: AWS Ground Station receiver instance security group. VpcId: YourVpcId SecurityGroupIngress: - IpProtocol: udp FromPort: 55555 ToPort: 55555 SourceSecurityGroupId: !Ref DataflowEndpointSecurityGroup Description: "Allow AWS Ground Station Ingress from DataflowEndpointSecurityGroup" ReceiverSubnet: Type: AWS::EC2::Subnet Properties: # Ensure your CidrBlock will always have at least one available IP address per dataflow endpoint. # See https://docs.aws.amazon.com/vpc/latest/userguide/subnet-sizing.html for subent sizing guidelines. CidrBlock: "10.0.0.0/24" Tags: - Key: "Name" Value: "AWS Ground Station - Dataflow endpoint Example Subnet" - Key: "Description" Value: "Subnet for EC2 instance receiving AWS Ground Station data" VpcId: !Ref ReceiverVPC ``` # Set up and configure Amazon EC2 Properly configuring your Amazon EC2 instance is required for synchronous delivery of VITA-49 Signal/IP data or VITA-49 Extension data/IP to be delivered via the AWS Ground Station Agent or a dataflow endpoint. Depending on your specific needs, you may perform the Front End (FE) processor or Software Defined Radio (SDR) directly on the same instance, or you may need to utilize additional EC2 instances. Selection and installation of your FE or SDR is beyond the scope of this user guide. For more information on the specific data formats, see [AWS Ground Station data plane interfaces](dataflows.md#dataflows.interfaces). For information about our service terms, please see [AWS Service Terms](https://aws.amazon.com/service-terms). ## Supplied Common Software AWS Ground Station provides common software to ease setup of your Amazon EC2 instance. ### AWS Ground Station Agent The AWS Ground Station Agent receives Digital Intermediate Frequency (DigIF) downlink data and egresses decrypted data that enables the following: + DigIF downlink capability from 40 MHz to 400 MHz of bandwidth. + High rate, low jitter DigIF data delivery to any public IP (AWS Elastic IP) on the AWS network. + Reliable data delivery using Forward Error Correction (FEC). + Secure data delivery using a customer managed AWS KMS key for encryption. For more information, see [AWS Ground Station Agent User Guide](https://docs.aws.amazon.com/ground-station/latest/gs-agent-ug). ### Dataflow endpoint application A networking application that is used by AWS Ground Station to send and receive data between the AWS Ground Station antenna locations, and your Amazon EC2 instances. It can be used for the uplink and downlink of data. ### Software Defined Radio (SDR) A software defined radio (SDR) that can be used to modulate/demodulate the signal used to communicate with your satellite. ## AWS Ground Station Amazon Machine Images (AMIs) To reduce the build and configuration times of these installs, AWS Ground Station also offers preconfigured AMIs. The AMIs with a dataflow endpoint networking application and a software defined radio (SDR) are made available to your account after your onboarding is complete. They can be found in the Amazon EC2 console by searching for *groundstation* in private [Amazon Machine Images (AMIs)](https://console.aws.amazon.com/ec2/home?#Images:visibility=private;search=groundstation;sort=imageName). The AMIs with AWS Ground Station Agent are public and can be found in the Amazon EC2 console by searching for *groundstation* in public [Amazon Machine Images (AMIs)](https://console.aws.amazon.com/ec2/home?#Images:visibility=public-images;search=groundstation;sort=imageName).