Post-deployment tasks

Sign in to the AWS Glue console.

Select Crawlers on the left navigation pane.

Wait for all crawlers to show status as READY, and the Tables added column to each be 1 or larger.

Sign into the Amazon Omics console

Select Storage > Reference store in the left navigation page.

Verify the presence of hg38 under Reference Genomes with Import status set to Active.

Select Analytics > Variant stores in the left navigation page.

Verify that Variant store with name omicsvariantstore has status set to Active.

Choose omicsvariantstore and verify that there are two Job IDs with Import status set to Completed. These correspond to the import of the 1000 Genomes VCF and the example VCF.

Select Analytics > Annotation stores in the left navigation page.

Verify that Annotation store with the name omicsannotationstore has status set to Active.

Choose omicsannotationstore and verify that there is a Job ID with Import status set to Completed. This corresponds to the import of the ClinVar VCF.

Sign in to the AWS CloudFormation console.

Select Stacks in the left navigation pane, then select the Imaging stack.

Select the Outputs tab, then select the link shown in the CreateQuicksightLink output. This opens the CloudFormation Create Stack wizard with the stack name and project name pre-filled.

Choose Create stack to deploy the stack.

You can view the status of the stack in the AWS CloudFormation Console in the Status column. You should see a status of CREATE_COMPLETE in approximately 1 minute.

Open the QuickSight console from the AWS Management Console by searching for QuickSight.

In the Datasets tab, locate the new dataset with your selected project name and choose it to open.

Choose Create analysis.

Your analysis will begin with a pre-selected AutoGraph visual. Change this to a Horizontal Bar Chart by selecting the appropriate icon in the Visual types panel.

From the Fields list, drag the age_at_dx_numeric field to the Y axis field well at the top of the screen. Your visual will now show Count of Records by Age_at_dx_numeric.

The visual by default will zoom into the data, which you can navigate using the vertical scroll bar in the visual tile.

To view the entire data set, with the visual selected, choose the Format visual gear icon in the small context menu that appears to the upper right of the visual. Then, expand the Y-axis settings and uncheck Show data zoom.

Next, we will add a second visual to look at an additional field.

Choose the + Add button in the upper-left and select Add visual.

From the Visual types panel, choose the Heat map visual type.

Drag the egfr_mutation_status field from the Fields list to the Rows field well, and drag the eml4_alk_translocation_status field to the Columns field well.

The visual now shows a heat map of the number of patients who have had their tumors tested for EML4-ALK translocation and EGFR mutation status. Because there are a substantial number of patients without test results, let’s add some filters to the data set to clean this up.

Select the Filter tab and ensure that you have the newly created heat map visual selected.

Choose the + button in the Filters section to add a new filter and select eml4_alk_translocation_status as the field to filter.

Choose the newly created filter and change the following settings:

Choose Apply to see the effect of the new filter, and then choose Close.

Repeat steps 11 through 13, but this time select the egfr_mutation_status field.

Next, create a pivot table that shows the number of data records available for each category of patients in our filtered cohort.

Choose the + Add button in the upper-left and select Add visual.

From the Visual types panel, choose the Pivot table visual type.

Drag both the egfr_mutation_status and eml4_alk_translocation_status fields from the Fields list to the Columns fields well.

In the Values fields well, change it to “Row” mode by selecting Row, then drag all of the num_..._records fields into the Values field well.

For each entry in the Values field well, change the Aggregate function from Sum to Count by choosing the dropdown arrow, then selecting Count from the Aggregate menu.

Finally, create a table listing the patient IDs that are in the current analysis. This can be exported as a CSV file to create a cohort for further analysis.

Choose the + Add button in the upper-left and select Add visual.

From the Visual types panel, choose the Table visual type.

Drag the bcr_patient_barcode field into the Value field well. You can add additional fields such as those we have used in the filters or other visuals.

To export the list of patients as a CSV file, in the small context menu that appears to the upper right of the visual, choose the Menu options (…) icon, then select Export to CSV.

Navigate to AWS Lake Formation, you should see the variant store and annotation store resource shares listed under Data catalog > Tables with the names for the Omics stores. In this case, omicsvariantstore and omicsannotationstore.

Select omicsvariantstore, choose Actions and choose Create resource link. Give the resource link a name, such as variants. This will be the table name that will show up in Athena. This table can be added to the existing solution database >genomicsanalysis. Once created, repeat the process for the annotation store omicsannotationstoreand give it a name, such as annotations.

Once resource links are created, select the newly created table variants under Tables, choose Actions and select Grant on target. Select the user and role that you want to allow access.

Next, under LF-Tags or catalog resources, select Named data catalog resources and choose the resource link created for the database, such as <account-id>-<variant-store-id>-variant. Select All tables. Under Table permissions, check the Select and Describe box under Table permissions and Grantable permissions. Select All data access under Data permissions and choose Grant. Repeat the process for the annotations table.

After the previous step completes successfully, navigate to Amazon Athena in the console, select AwsDataCatalog under Data source and genomicsanalysis under Database. Verify that you are able to see the newly created tables variants and annotations under Tables and views. Also verify that you have selected Workgroup genomicsanalysis-aws-region. You are now ready to run a query against all the data in these databases.

Sign in to the Amazon SageMaker AI console.

Select Notebook Instances under the Notebook category on the left pane.

Select your notebook instance and choose Open Jupyter.

To explore cohort creation, clinical dataset enrichment, and imaging visualization, select cohort-creation.ipynb.

Run each step in the notebook.

To explore the creation of a drug response report, in the main Jupyter window, select runbook.ipynb.

Run each step in the notebook.

Sign in to the Amazon Athena console.

Under Data Source select the AwsDataCatalog data source on the left pane.

Under Database select the genomicsanalysis database on the left pane.

In the query window paste the query below.

Select Run Query.

SELECT  count(*)/cast(numsamples AS DOUBLE) AS genotypefrequency 
    ,cv.attributes['RS'] as rs_id
    ,cv.attributes['CLNDN'] as clinvar_disease_name
    ,cv.attributes['CLNSIG'] as clinical_significance
    ,sv.contigname
    ,sv.start
    ,sv."end"
    ,sv.referenceallele
    ,sv.alternatealleles
    ,sv.calls
        FROM {variant_table_name} sv 
        CROSS JOIN 
            (SELECT count(1) AS numsamples 
            FROM 
                (SELECT DISTINCT vs.sampleid 
                FROM {variant_table_name} vs
                WHERE vs.sampleid LIKE 'NA12%')) 
        JOIN {annotation_table_name} cv 
        ON sv.contigname = cv.contigname 
            AND sv.start = cv.start 
            AND sv."end" = cv."end" 
            AND sv.referenceallele = cv.referenceallele 
            AND sv.alternatealleles = cv.alternatealleles
            AND cv.attributes['CLNSIG'] LIKE '%response%' 
            AND sv.sampleid LIKE 'NA12%' 
        GROUP BY  sv.contigname 
                  ,sv.start 
                  ,sv."end" 
                  ,sv.referenceallele 
                  ,sv.alternatealleles
                  ,sv.calls
                  ,cv.attributes['RS']
                  ,cv.attributes['CLNDN']
                  ,cv.attributes['CLNSIG'] 
                  ,numsamples 
        ORDER BY genotypefrequency DESC LIMIT 50