Knowledge Layer

Knowledge Layer Overview

The Knowledge Layer serves as the semantic intelligence foundation of the architecture, enabling unified querying across heterogeneous data sources through a Virtual Knowledge Graph (VKG) approach. At its core, the layer exposes SPARQL query capabilities that allow applications, agents, and users to interact with knowledge as a unified graph — regardless of whether that knowledge resides in a dedicated graph database, relational databases, data warehouses, or data lakes. The VKG Service, deployed on Amazon ECS with Fargate or EC2 instances, orchestrates query translation and federation, enabling seamless access to both materialized semantic knowledge and virtualized data layer sources within a single SPARQL query. Ontop is a strong choice for implementing the VKG Service — it's an open-source platform purpose-built for OBDA (Ontology-Based Data Access) that translates SPARQL queries over ontology-defined knowledge graphs into optimized SQL executed directly against relational sources, using R2RML mappings and lightweight OWL ontologies, with no data duplication required.

Amazon Neptune Database serves as the persistent graph store for knowledge that cannot or should not remain in the data layer, including entities, relationships, and propositions extracted from unstructured documents (PDFs, images, reports) and mapped to ontology classes and properties via the Information Extraction pipeline, deduction results that infer new relationships or properties through symbolic reasoning, cross-domain associations linking entities across disparate data sources, and enriched semantic metadata. Together, these represent the materialized A-Box (instance data) of the knowledge graph, capturing both directly extracted and logically derived facts as first-class graph citizens. Neptune's native RDF and SPARQL support makes it the natural home for the T-Box (ontology/schema) and this materialized A-Box, which require graph-native reasoning and traversal. Neptune DB Streams provide near real-time change data capture for all create, update, and delete operations, enabling event-driven architectures where downstream processes, such as the Symbolic Reasoning Service, can react immediately to knowledge graph mutations, maintaining consistency and triggering inference workflows without polling.

The VKG architecture's key strength lies in its ability to query the Semantic Layer Graph Database (Neptune), relational databases and data warehouses (via JDBC-connected sources like Snowflake, Redshift, Athena), and data lakes (Iceberg/Parquet on S3) — either independently or federated together in a single query. This eliminates data duplication while preserving the semantic richness of the knowledge graph, allowing the Knowledge Layer to serve as a true abstraction over the enterprise's distributed data landscape. Ontology-driven mappings (R2RML) define how relational and tabular data sources are virtualized as RDF, enabling SPARQL queries to transparently span both materialized and virtual knowledge without requiring applications to understand the underlying physical storage topology.

The Symbolic Reasoning Service maintains knowledge graph consistency, validates data quality, and derives new knowledge automatically through event-driven workflows triggered by Neptune DB Streams. Deployed on Amazon ECS, the service implements a modular ensemble architecture supporting multiple reasoning engines including ELK (lightweight OWL reasoning), HermiT (full OWL 2 DL expressivity), SHACL validators (constraint enforcement), and SWRL engines (rule-based inference). When graph mutations occur, the Symbolic Reasoning Service identifies applicable rules, retrieves necessary sub-graphs from the VKG Service (federating across Neptune and Data Layer sources), performs reasoning, and persists inferred triples, validation results, and consistency flags back to Neptune with full provenance metadata. This closed-loop architecture ensures that ontology changes, new assertions, and constraint violations are automatically validated and that inferred knowledge is materialized in near real-time, supporting autonomous agents with verified, explainable knowledge.

The Knowledge Layer includes a Metric Store — a governed registry of reusable, ontology-grounded metric definitions persisted alongside the T-Box, R2RML mappings, and SHACL constraints in the Semantic Layer Graph Database.

Metric Definition: A reusable, pre-defined calculation bound to an ontology class and its datatype properties. Each definition specifies an aggregation function, the properties involved, optional filters, and a pre-compiled SPARQL/SQL expression resolved deterministically at query time — no LLM required.

ex:ProcessingTime a ex:MetricDefinition ;
    rdfs:label "Processing Time" ;
    ex:metricForClass ex:Claim ;
    ex:aggregationFunction "AVG" ;
    ex:metricExpression "AVG(DATEDIFF('day', ?submissionDate, ?approvalDate))" ;
    ex:metricUnit "days" .

Key properties:

Virtual Knowledge Graph (VKG) Query

Data Flow - Virtual Knowledge Graph (VKG) Query

1. SPARQL Client → Virtual Knowledge Graph Service A SPARQL Client (application, agent, or user interface) submits a SPARQL query to the Virtual Knowledge Graph (VKG) Service deployed on Amazon ECS within the VPC. The VKG Service endpoint accepts the query via HTTPS, receiving the JWT/OIDC bearer token propagated by the AgentCore Gateway. The VKG Service extracts user identity (sub), authorization groups (groups), and agent identity (act) from the token for row-level security enforcement and audit logging. The VKG Service (implemented using Ontop or similar OBDA platform) parses the SPARQL query and consults the loaded ontology (T-Box) and R2RML mappings to determine the query execution plan.

2. VKG Service Query Planning & Federation The VKG Service analyzes the SPARQL query structure and determines which data sources are required to satisfy the query — materialized knowledge in Neptune DB (Step 3), virtualized data from relational/data warehouse sources (Step 4), or a federated combination of both. For triple patterns that correspond to ontology-mapped relational tables, the VKG Service translates the SPARQL query into optimized SQL using the R2RML mappings. For triple patterns representing deduced knowledge, cross-domain relationships, or ontology axioms, the query is routed to Neptune DB via SPARQL endpoint. The VKG Service applies query optimization techniques including predicate pushdown, join reordering, and filter propagation to achieve locality optimization and minimize data transfer from sources.

3. VKG Service → Neptune DB (Semantic Layer Graph Database) For SPARQL query fragments targeting materialized knowledge, the VKG Service issues a SPARQL query to Amazon Neptune DB via the Neptune SPARQL endpoint. Neptune executes the query against the stored RDF graph, which includes the T-Box (ontology/schema), materialized A-Box (instance data), deduction results from reasoning workflows, and cross-domain entity linkages. Neptune enforces access control policies at query execution time, ensuring users only retrieve authorized data. Neptune returns RDF bindings (for SELECT queries) or RDF graphs (for CONSTRUCT/DESCRIBE queries) to the VKG Service. Neptune's native support for SPARQL 1.1, named graphs, and OWL inference ensures semantic fidelity in query results.

4. VKG Service → Data Layer (Virtualized Query Execution via JDBC) For SPARQL query fragments mapped to relational or data warehouse sources, the VKG Service translates the SPARQL into native SQL and dispatches queries to the appropriate Data Layer sources via JDBC connections. The VKG Service retrieves tabular result sets from these sources, converts rows and columns to RDF bindings using the R2RML mapping definitions, and integrates them with results from Neptune (Step 3) if the query is federated. SQL queries are executed directly against the source systems — no data is duplicated or moved into Neptune unless explicitly materialized by a separate ETL or reasoning process.

5. Data Layer Sources Execute SQL Queries Each data source in the Data Layer executes the translated SQL query against its native storage. Supported sources include:

VPC peering or AWS PrivateLink ensures secure connectivity back to the VKG Service. Query results are returned as standard JDBC result sets, which the VKG Service consumes, transforms to RDF using R2RML mappings, and merges with Neptune-sourced results for the final SPARQL response.

6. VKG Service Returns Unified SPARQL Results to Client The VKG Service merges results from Neptune (Step 3) and Data Layer sources (Steps 4–5), applies any final SPARQL query operators (e.g., UNION, OPTIONAL, FILTER), and returns a unified SPARQL result set to the SPARQL Client. Results are serialized in the requested format — SPARQL JSON, XML, or Turtle — based on the client's Accept header. The VKG Service caches query plans and frequently accessed R2RML mappings to improve performance for repeated queries. OpenTelemetry traces are emitted across all steps, capturing query latency, source attribution, and row counts for observability.

Cross-Cutting Concerns

Event-Driven Architecture

Data Flow - Event-Driven Architecture

1. SPARQL Client → Virtual Knowledge Graph Service (Data Manipulation Query) A SPARQL Client submits a SPARQL data manipulation query to the Virtual Knowledge Graph (VKG) Service running on Amazon ECS within the VPC. This could be an INSERT DATA, DELETE DATA, or CONSTRUCT query — including construct queries generated by reasoning processes that result in new knowledge or deductions. For example, a reasoning workflow may infer new relationships between entities based on ontology axioms and assert them as new triples. The VKG Service authenticates the request using JWT/OIDC bearer tokens propagated by the AgentCore Gateway and validates the SPARQL UPDATE syntax.

2. VKG Service → Neptune DB (Semantic Layer Graph Database) The VKG Service routes the SPARQL data manipulation query to Amazon Neptune DB via the Neptune SPARQL Update endpoint. Neptune persists the changes to the RDF graph — inserting new triples (CREATE), modifying existing statements (UPDATE), or removing triples (DELETE). Neptune enforces access control policies at write time, ensuring the requesting principal has authorization to modify the targeted named graphs or triples. The update is committed to Neptune's ACID-compliant transaction log, ensuring durability and consistency. Changes may target the T-Box (ontology/schema modifications) or A-Box (instance data updates, deduced relationships, cross-domain entity linkages).

3. Neptune DB → Neptune DB Streams → AWS Lambda (Change Data Capture) Upon successful commit of the SPARQL UPDATE, Neptune DB Streams automatically emit change data capture (CDC) events in near real-time. Each stream event captures the mutation type (CREATE, UPDATE, DELETE), the affected triples (subject, predicate, object), the named graph IRI, timestamp, and transaction metadata. Neptune DB Streams provide an ordered, durable event log with configurable retention (up to 7 days). An AWS Lambda function subscribes to the Neptune DB Streams and is invoked automatically as new change events arrive. The Lambda function acts as the stream event handler, deserializing the event payload and routing it to downstream subscribers based on event type and affected knowledge domain.

4. AWS Lambda → Amazon SNS (Event Publication to Subscribers) The Lambda function publishes the processed Neptune DB Streams event to Amazon SNS topics. SNS acts as a pub/sub messaging layer, enabling fan-out to multiple downstream subscribers. Subscribers receive CUD events and handle them appropriately based on their role:

SNS supports both in-account subscribers (Lambda, SQS, EventBridge) and cross-account or external subscribers (HTTPS webhooks, email) for enterprise-wide event distribution. Message filtering policies can route events based on metadata attributes (e.g., eventType, namedGraph, ontologyId).

Event Types & Symbolic Reasoning Triggers

Reasoning Outcome Handling - All reasoning actions produce one or more of the following outcomes, persisted to Neptune with full provenance metadata: (1) new inferred triples stored in dedicated named graphs, distinguishing inferred from asserted knowledge; (2) SHACL validation reports with severity levels (violation, warning, info), focus nodes, and remediation suggestions; (3) consistency flags annotating axioms or instances with their validation status; (4) invalidation signals sent via SNS to AgentCore (triggering cache refresh and reasoning chain re-evaluation), Search Index Service (updating embeddings), and Ontology Engineering UI (notifying knowledge engineers). All outcomes emit new CDC events via Neptune DB Streams, enabling cascading reasoning workflows and maintaining a complete audit trail for compliance and explainability.

Cross-Cutting Concerns

This architecture enables near real-time, event-driven knowledge graph workflows — ensuring that reasoning processes, validation engines, search indexes, and downstream agents react immediately to knowledge mutations without polling Neptune. The pattern is particularly valuable for maintaining consistency in federated knowledge graphs where materialized knowledge, deduced relationships, and ontology changes must trigger automated reasoning and validation workflows to preserve semantic integrity.

Symbolic Reasoning

Data Flow — Knowledge Layer Symbolic Reasoning

1. Amazon SNS → Amazon SQS (Graph Mutation Event Trigger) The Semantic Layer Graph Database (Neptune DB) emits change data capture events via Neptune DB Streams for all knowledge graph mutations — create, update, and delete operations on ontologies (T-Box), instance data (A-Box), or relationships. These mutation events are published to an Amazon SNS topic configured for reasoning triggers. The SNS topic implements a fan-out pattern, publishing to an Amazon SQS queue dedicated to the Symbolic Reasoning Service. This event-driven architecture ensures that ontology changes, new assertions, or data deletions automatically trigger reasoning workflows without manual intervention. The SQS message contains the mutation type (CREATE, UPDATE, DELETE), affected triples (subject, predicate, object), named graph IRI, transaction metadata, and references to the modified entities.

2. Amazon SQS → AWS Lambda (Reasoning Orchestrator) An AWS Lambda function polls the SQS queue and is triggered when graph mutation events arrive. The Lambda function acts as the reasoning orchestration layer, analyzing the event payload to determine which reasoning workflows to invoke based on the type and scope of change:

The Lambda function prepares the invocation payload for the Symbolic Reasoning Service, including references to affected named graphs, ontology versions, and reasoning policies to apply. The orchestrator invokes the Reasoning Service asynchronously, enabling parallel processing of independent reasoning tasks.

3. AWS Lambda → Symbolic Reasoning Service (Amazon ECS) The Lambda function invokes the Symbolic Reasoning Service deployed as containerized tasks on Amazon ECS (with Fargate or EC2 instances). The Reasoning Service implements a modular, ensemble architecture supporting multiple reasoning engines and techniques:

Ontology and Rules Management — The Symbolic Reasoning Service maintains near real-time synchronization with ontologies and rules stored in S3 Versioned Buckets or Git repositories. The service subscribes to ontology change events (via SNS/SQS) and automatically reloads updated ontologies and rule sets. This mechanism ensures that reasoning always operates against the current ontology version without manual intervention.

Rule Set Loading and Planning — Based on the graph mutation event details, the Symbolic Reasoning Service identifies the related rules that need to be evaluated. For example, if a new claim instance is asserted, only rules applicable to claims (e.g., eligibility validation, fraud detection rules) are loaded. The service then assembles a reasoning plan to determine what data is necessary to satisfy the rule set:

4. Symbolic Reasoning Service → Virtual Knowledge Graph (VKG) Service The Symbolic Reasoning Service sends requests to retrieve necessary sub-graphs to the Virtual Knowledge Graph (VKG) Service to satisfy the rule set. The VKG Service federates queries across:

The VKG Service translates SPARQL queries into optimized SQL (for virtualized sources) and native SPARQL (for Neptune), returning unified RDF bindings to the Symbolic Reasoning Service. This enables reasoning over both the Semantic Layer and live data from source systems without requiring full materialization.

Once all sub-graphs have been retrieved, the Symbolic Reasoning Service performs reasoning using the loaded rule set and selected reasoning engines:

5. Symbolic Reasoning Service → Neptune DB (Semantic Layer Graph Database) Any reasoning that results in updates (e.g., new knowledge via deductions, inferred relationships, validation results) to the Semantic Layer Graph Database are persisted and reflected in Neptune DB. The Symbolic Reasoning Service writes:

Neptune DB Streams then emit events for these newly asserted triples, potentially triggering additional downstream processing (e.g., notifying agents, updating search indexes, triggering cascading reasoning workflows). This creates a closed-loop, event-driven reasoning architecture where reasoning results can trigger further reasoning or validation cycles.

Reasoning Ensemble — Modular Architecture

Note: These are open-source implementation options. The modular architecture allows selection and combination of reasoning engines based on use case requirements.

Cross-Cutting Concerns

This Knowledge Layer Symbolic Reasoning architecture enables continuous, event-driven reasoning that maintains knowledge graph consistency, validates data quality, and derives new knowledge automatically as the Semantic Layer evolves — supporting autonomous agents with verified, explainable knowledge.