Well-architected AWS for LLM, RAG & agentic AI

1The reference architecture, drawn

The whole GenAI platform on one canvas, in the zoned left-to-right style: a request enters at the client, passes guardrails, and flows through ingestion, the knowledge/vector layer, the model, and (for agents) AgentCore — all sitting on a shared security foundation. Click any box for what it is, the AWS service, and a jump straight to the matching pipeline stage and pillar detail below.

2The original diagram — zoomable & clickable

Your reference architecture at full resolution. Scroll or use +/− to zoom, drag to pan, and click any region to read what it is and jump to the matching section. Double-click to reset.

+ − ↺

Zoom: scroll · Pan: drag · Click a region for detail

3Part 1 — the AWS foundation (no AI yet)

Before a single model is called, you need a secure, reliable place to run. This is the part of the diagram that would exist for any serious workload — the outer frame: identity, network, compute, data, observability, governance, and delivery. Get this right and the AI layer in Part 2 has somewhere safe to live.

4Part 2 — adding AI: LLM, RAG & agents

Now the AI/Data Platform layer that sits inside the Part-1 foundation. A request flows left to right through guardrails, retrieval, the model, and (for agents) tools. Click each stage for the AWS service, the pillars applied there, and the pitfalls.

A RAG/agentic platform is a pipeline: a user question flows through guardrails, retrieval, a model, and back. Click each stage to see the AWS service that owns it, how the Well-Architected pillars apply there specifically, and the problem that stage tends to create with its fix. This is the diagram everything else links back to.

5The Well-Architected Framework, applied to this diagram

The AWS Well-Architected Framework is six pillars — lenses you hold up to a design to check it is sound. AWS added a dedicated Generative AI Lens and a Responsible AI Lens for AI workloads. Here is what each pillar means, and exactly where it shows up in the reference architecture above.

6Choosing the vector store — the decision that shapes cost

The vector database is where RAG lives or dies on cost and latency. AWS gives you several, and the right answer depends on scale and how fast you need answers. Real tradeoffs, with the numbers people actually report.

7Practical build order

You do not build all six stages at once. This is the order that gets you to a working, well-architected system without over-engineering the prototype.

8Glossary

Every acronym and term used on this page, defined in plain language. Type to filter.

3-2-1-1-0 rule

Ransomware-aware backup rule: 3 copies of data, on 2 media types, 1 off-site, 1 immutable/offline, and 0 errors on restore verification.

Adversary-in-the-Middle (AiTM)

A phishing attack where a reverse proxy sits between the victim and the real login page, capturing the session token after a genuine login — defeating most MFA.

AgentCore

Amazon Bedrock's production platform for building and running AI agents — secure runtime, tool Gateway, memory, and observability.

Agentic AI

An LLM that can plan, reason, and take actions by calling tools/APIs — not just answer questions.

Availability Zone (AZ)

One of several physically separate data centers within an AWS Region, each with independent power, cooling, and networking. Spreading subnets and databases across AZs — "Multi-AZ" — is how a design survives a single data-center failure. The backbone of the Reliability pillar.

Region

A geographic AWS location (e.g. us-east-1) made up of multiple Availability Zones. Multi-AZ protects within a region; Multi-Region protects against a whole-region outage.

ALB (Application Load Balancer)

AWS load balancer that routes incoming requests to your application tier.

AMI (Amazon Machine Image)

A saved template of a server's disk used to launch identical instances — a 'golden AMI' is a hardened, approved baseline.

Aurora

AWS's managed relational database (MySQL/PostgreSQL-compatible); PostgreSQL Aurora can host vectors via pgvector.

Bedrock

Amazon's managed platform for building generative-AI applications — foundation models, RAG, guardrails, and agents behind one API.

BIA (Business Impact Analysis)

The exercise that determines how much downtime and data loss each system can tolerate — the input that sets RTO and RPO.

Canary token

A secret planted in the system prompt that must never appear in output; if it does, it signals a prompt-injection/exfiltration attempt.

CDK (Cloud Development Kit)

AWS's infrastructure-as-code tool that lets you define cloud resources in a programming language (TypeScript, Python, etc.).

Chunking

Splitting source documents into smaller passages so retrieval can return precise, relevant pieces to the model.

CloudFront

AWS's content delivery network (CDN) — caches content at the edge and absorbs traffic spikes.

CloudTrail

AWS's audit log of every API call in an account — who did what, when.

CloudWatch

AWS's monitoring service — metrics, logs, dashboards, and alarms.

Cognito

AWS's service for user sign-up, sign-in, and identity (authentication) at the application edge.

Conditional Access

Identity policy that allows or blocks a login based on context — device, location, risk score — rather than just the password and code.

Converse API

Bedrock's recommended, model-agnostic interface for chat-style calls — swap models without rewriting.

DevSecOps

Building security into the software delivery pipeline (CI/CD) rather than bolting it on afterward.

ECS / EKS

AWS container services — Elastic Container Service and Elastic Kubernetes Service — for running your orchestration code.

ElastiCache

AWS's managed in-memory cache (Redis/Valkey) — used for fast repeated answers and semantic caching.

Embedding

A list of numbers (a vector) that captures the meaning of a chunk of text, so similar meanings sit close together.

FIDO2 / passkey

Phishing-resistant authentication that does a cryptographic handshake bound to the real website's domain — it won't authenticate against a proxy, so it defeats AiTM.

Foundation model

A large, general-purpose model (Claude, Llama, Titan) accessed via API, used as the basis for generation.

GuardDuty

AWS's account-wide threat-detection service — flags anomalous or malicious activity.

Guardrails

Bedrock's safety layer that screens model input and output — content filtering, PII redaction, injection defense.

HNSW

Hierarchical Navigable Small World — the approximate-nearest-neighbor index algorithm most vector stores use to search vectors fast.

Hybrid search

Combining vector (semantic) search with keyword search to improve retrieval recall — catches both meaning and exact terms.

IaC (Infrastructure as Code)

Defining cloud infrastructure in version-controlled files (CDK, CloudFormation, Terraform) so it's repeatable and reviewable.

IAM (Identity and Access Management)

AWS's system of users, roles, and policies that controls who can do what — the foundation of least privilege.

IAM Identity Center

AWS's single-sign-on (SSO) service for workforce access across accounts.

KMS (Key Management Service)

AWS's service for creating and controlling encryption keys — encrypt data at rest with keys you own.

Knowledge Base

Bedrock's managed RAG service — point it at S3 and it handles chunking, embedding, vector storage, and retrieval.

Lake Formation

AWS's service for fine-grained governance and access control over data lakes.

Lambda

AWS's serverless compute — runs code on demand with no server to manage; scales to zero.

Least privilege

Granting each identity or tool exactly the permissions it needs and no more — the core security principle.

LLM (Large Language Model)

A model trained on vast text that generates language — answers, summaries, code — accessed here via Bedrock.

Macie

AWS's service that discovers and classifies sensitive data (like PII) in S3 automatically.

MFA (Multi-Factor Authentication)

Requiring a second factor beyond a password. Note: MFA protects the login moment, not the session that follows.

MFA fatigue

An attack that floods a user with push prompts (after stealing the password) until they approve one out of frustration.

Multi-AZ

Running across multiple Availability Zones (isolated datacenters) in a region for resilience against a single-zone failure.

NAT Gateway

Gives private-subnet resources controlled outbound-only internet access.

OpenSearch Serverless

AWS's managed search/vector engine — fast hybrid and k-NN vector search at scale; also used as a SIEM.

pgvector

A PostgreSQL extension that stores and searches vectors — the cheap option for smaller knowledge bases.

PII (Personally Identifiable Information)

Data that identifies a person — must be discovered, protected, and often redacted.

PITR (Point-in-Time Recovery)

Restoring a database to its exact state at a chosen second — gives a very low RPO (RDS/Aurora, DynamoDB).

PrivateLink

Private connectivity to AWS and SaaS services that keeps traffic off the public internet.

Prompt injection

An attack that hides instructions in user input or retrieved documents to make the model do something unintended — worse for agents, where it becomes an action.

RAG (Retrieval-Augmented Generation)

Grounding an LLM's answer in retrieved documents so it uses current, private, or domain-specific facts instead of only its training data.

RDS (Relational Database Service)

AWS's managed relational databases (PostgreSQL, MySQL, etc.).

Re-ranking

A second step that reorders retrieved chunks by relevance before sending the best to the model.

REL09-BP04

The AWS Well-Architected Reliability best practice that says: periodically restore backups to verify they actually work and meet your RTO/RPO.

RPO (Recovery Point Objective)

The maximum data loss you can tolerate, measured in time — it sets how often you must back up or replicate.

RTO (Recovery Time Objective)

The maximum downtime you can tolerate before impact is unacceptable — it sets how fast you must be able to restore or fail over.

S3 (Simple Storage Service)

AWS's object storage — the usual home for the document corpus a RAG system retrieves from.

S3 Vectors

Native vector storage in S3 — the cheapest option at huge scale, but slower; not a full database.

SBOM (Software Bill of Materials)

A list of every component in a piece of software — used to track and respond to vulnerabilities.

SCP (Service Control Policy)

An AWS Organizations guardrail that limits what accounts are allowed to do at all, regardless of IAM.

Security Hub

AWS's service that aggregates security findings and posture across the account.

Session token

The credential your browser holds after login that keeps you signed in — stealing it (via AiTM or malware) bypasses MFA entirely.

SIEM

Security Information and Event Management — central collection and analysis of security logs (here, OpenSearch).

SIM swap

Tricking a mobile carrier into moving a victim's number to the attacker's SIM, so SMS one-time codes go to the attacker.

SSO (Single Sign-On)

One authenticated identity used across many applications — convenient, but a high-value target.

Step Functions

AWS's service for orchestrating multi-step workflows reliably, with retries.

Terraform

A popular multi-cloud infrastructure-as-code tool.

Titan

Amazon's own family of foundation and embedding models on Bedrock.

Token theft

Stealing a valid session token (often via infostealer malware) to hijack a signed-in session without needing the password or MFA.

TOTP

Time-based One-Time Password — the 6-digit authenticator-app code; better than SMS but still phishable via AiTM.

Vector / vector store

A vector is the numeric representation of meaning; a vector store (pgvector, OpenSearch, S3 Vectors) holds them for similarity search.

VPC (Virtual Private Cloud)

Your isolated private network in AWS, divided into subnets you control.

VPC endpoint

A private door from your VPC to an AWS service, so traffic never traverses the public internet.

WAF (Web Application Firewall)

Filters and blocks malicious web traffic at the edge before it reaches your app. (Not to be confused with the Well-Architected Framework.)

Well-Architected Framework

AWS's six pillars — operational excellence, security, reliability, performance efficiency, cost optimization, sustainability — for reviewing a design; with dedicated Generative AI and Responsible AI lenses.

Zero Trust

A security model that trusts nothing by default — every request is authenticated and authorized, even inside the network.

9Implementation checklist — build it line by line

The reference architecture as an ordered, do-this-then-that checklist. Work top to bottom: each phase depends on the one before it. Tick items off as you go — your progress updates live. (Progress is per session; it resets if you reload.)

resource "aws_organizations_account" "prod" {
  name      = "workload-prod"
  email     = "aws-prod@example.com"
  parent_id = aws_organizations_organizational_unit.workloads.id
}

resource "aws_organizations_policy" "guardrails" {
  name = "baseline-guardrails"
  type = "SERVICE_CONTROL_POLICY"
  content = jsonencode({
    Version = "2012-10-17"
    Statement = [
      { Sid = "DenyCloudTrailStop", Effect = "Deny",
        Action = ["cloudtrail:StopLogging","cloudtrail:DeleteTrail"], Resource = "*" },
      { Sid = "DenyPublicS3", Effect = "Deny",
        Action = "s3:PutBucketPublicAccessBlock", Resource = "*",
        Condition = { StringNotEquals = { "s3:PublicAccessBlockConfiguration.BlockPublicAcls" = "true" } } }
    ]
  })
}

# In IAM Identity Center settings, set MFA to "always-on" and
# authenticator type to security keys / passkeys (WebAuthn).
resource "aws_ssoadmin_permission_set" "admin" {
  name             = "AdminAccess"
  instance_arn     = tolist(data.aws_ssoadmin_instances.this.arns)[0]
  session_duration = "PT1H"   # short sessions
}

resource "aws_iam_role" "app" {
  name                 = "rag-app-role"
  permissions_boundary = aws_iam_policy.boundary.arn
  assume_role_policy   = data.aws_iam_policy_document.assume.json
}
resource "aws_iam_policy" "boundary" {
  name   = "app-boundary"
  policy = data.aws_iam_policy_document.boundary.json  # caps max privilege
}

resource "aws_cloudtrail" "org" {
  name                          = "org-trail"
  s3_bucket_name                = aws_s3_bucket.log_archive.id
  is_organization_trail         = true
  is_multi_region_trail         = true
  enable_log_file_validation    = true
}

aws guardduty create-detector --enable
aws securityhub enable-security-hub \
  --enable-default-standards

resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
}
# subnets created per-AZ below (step 8)

locals { azs = ["us-east-1a","us-east-1b"] }

resource "aws_subnet" "public" {
  for_each          = toset(local.azs)
  vpc_id            = aws_vpc.main.id
  availability_zone = each.value
  cidr_block        = cidrsubnet(aws_vpc.main.cidr_block, 8, index(local.azs, each.value))
}
resource "aws_subnet" "app"  { for_each = toset(local.azs)  /* private, +10 offset */
  vpc_id = aws_vpc.main.id  availability_zone = each.value
  cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 8, 10 + index(local.azs, each.value)) }
resource "aws_subnet" "data" { for_each = toset(local.azs)  /* private, +20 offset */
  vpc_id = aws_vpc.main.id  availability_zone = each.value
  cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 8, 20 + index(local.azs, each.value)) }

resource "aws_internet_gateway" "igw" { vpc_id = aws_vpc.main.id }
resource "aws_route" "public_inet" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.igw.id
}

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.us-east-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_route_table.app.id, aws_route_table.data.id]
}

resource "aws_vpc_endpoint" "bedrock" {
  vpc_id              = aws_vpc.main.id
  service_name        = "com.amazonaws.us-east-1.bedrock-runtime"
  vpc_endpoint_type   = "Interface"
  private_dns_enabled = true
  subnet_ids          = [for s in aws_subnet.app : s.id]
  security_group_ids  = [aws_security_group.endpoints.id]
}

resource "aws_eip" "nat" { domain = "vpc" }
resource "aws_nat_gateway" "nat" {
  allocation_id = aws_eip.nat.id
  subnet_id     = aws_subnet.public["us-east-1a"].id
}
# Skip this whole resource if all egress is via VPC endpoints.

resource "aws_security_group" "app" {
  vpc_id = aws_vpc.main.id
  # no ingress rules = default deny
  egress { from_port = 443 to_port = 443 protocol = "tcp"
           cidr_blocks = ["0.0.0.0/0"] }   # tighten to endpoint SGs in prod
}

resource "aws_wafv2_web_acl" "edge" {
  name  = "edge-acl"
  scope = "REGIONAL"
  default_action { allow {} }
  rule {
    name     = "AWSManagedCommon"
    priority = 1
    override_action { none {} }
    statement { managed_rule_group_statement {
      name = "AWSManagedRulesCommonRuleSet"  vendor_name = "AWS" } }
    visibility_config { cloudwatch_metrics_enabled = true
      metric_name = "common"  sampled_requests_enabled = true }
  }
  visibility_config { cloudwatch_metrics_enabled = true
    metric_name = "edge"  sampled_requests_enabled = true }
}

resource "aws_s3_bucket" "corpus" { bucket = "rag-corpus-prod" }
resource "aws_s3_bucket_public_access_block" "corpus" {
  bucket = aws_s3_bucket.corpus.id
  block_public_acls = true  block_public_policy = true
  ignore_public_acls = true  restrict_public_buckets = true
}
resource "aws_s3_bucket_versioning" "corpus" {
  bucket = aws_s3_bucket.corpus.id
  versioning_configuration { status = "Enabled" } }
resource "aws_s3_bucket_server_side_encryption_configuration" "corpus" {
  bucket = aws_s3_bucket.corpus.id
  rule { apply_server_side_encryption_by_default {
    sse_algorithm = "aws:kms"  kms_master_key_id = aws_kms_key.data.arn } } }

resource "aws_db_instance" "app" {
  engine                = "postgres"
  instance_class        = "db.r6g.large"
  multi_az              = true
  storage_encrypted     = true
  kms_key_id            = aws_kms_key.data.arn
  db_subnet_group_name  = aws_db_subnet_group.data.name
  backup_retention_period = 7
}

resource "aws_backup_plan" "daily" {
  name = "daily-critical"
  rule {
    rule_name         = "daily-35d"
    target_vault_name = aws_backup_vault.immutable.name
    schedule          = "cron(0 5 * * ? *)"
    lifecycle { delete_after = 35 }
  }
}

resource "aws_backup_vault" "immutable" { name = "immutable-vault" }
resource "aws_backup_vault_lock_configuration" "lock" {
  backup_vault_name = aws_backup_vault.immutable.name
  min_retention_days = 7
  changeable_for_days = 3   # compliance mode after this window
}

resource "aws_backup_restore_testing_plan" "quarterly" {
  name                         = "quarterly-restore-test"
  schedule_expression          = "cron(0 6 1 */3 ? *)"
  recovery_point_selection { algorithm = "LATEST_WITHIN_WINDOW"
    include_vaults = ["*"]  recovery_point_types = ["SNAPSHOT"] }
}

resource "aws_lambda_function" "handler" {
  function_name = "rag-request"
  role          = aws_iam_role.app.arn
  runtime       = "python3.13"
  handler       = "app.handler"
  timeout       = 60
  vpc_config { subnet_ids = [for s in aws_subnet.app : s.id]
               security_group_ids = [aws_security_group.app.id] }
}

resource "aws_sfn_state_machine" "ingest" {
  name     = "doc-ingestion"
  role_arn = aws_iam_role.sfn.arn
  definition = jsonencode({
    StartAt = "Parse"
    States = {
      Parse = { Type = "Task", Resource = aws_lambda_function.parse.arn, Next = "Chunk" }
      Chunk = { Type = "Task", Resource = aws_lambda_function.chunk.arn, Next = "Embed" }
      Embed = { Type = "Task", Resource = aws_lambda_function.embed.arn, Next = "Index" }
      Index = { Type = "Task", Resource = aws_lambda_function.index.arn, End = true }
    }
  })
}

resource "aws_cloudwatch_event_rule" "on_upload" {
  event_pattern = jsonencode({
    source = ["aws.s3"]
    "detail-type" = ["Object Created"]
    detail = { bucket = { name = [aws_s3_bucket.corpus.id] } }
  })
}
resource "aws_cloudwatch_event_target" "to_sfn" {
  rule = aws_cloudwatch_event_rule.on_upload.name
  arn  = aws_sfn_state_machine.ingest.arn
  role_arn = aws_iam_role.events.arn
}

aws bedrock create-guardrail \
  --name rag-guardrail \
  --content-policy-config '{"filtersConfig":[
     {"type":"PROMPT_ATTACK","inputStrength":"HIGH","outputStrength":"NONE"}]}' \
  --sensitive-information-policy-config '{"piiEntitiesConfig":[
     {"type":"EMAIL","action":"ANONYMIZE"}]}' \
  --blocked-input-messaging "Blocked." \
  --blocked-outputs-messaging "Blocked."

# Enable the extension once, then create the table:
#   CREATE EXTENSION IF NOT EXISTS vector;
#   CREATE TABLE chunks (id bigserial, embedding vector(1024), body text);
#   CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
# (run via your migration tool against aws_db_instance.app)

resource "aws_cloudwatch_metric_alarm" "lambda_errors" {
  alarm_name          = "rag-request-errors"
  namespace           = "AWS/Lambda"
  metric_name         = "Errors"
  dimensions          = { FunctionName = aws_lambda_function.handler.function_name }
  statistic           = "Sum"
  period              = 60
  evaluation_periods  = 1
  threshold           = 1
  comparison_operator = "GreaterThanOrEqualToThreshold"
}

resource "aws_budgets_budget" "monthly" {
  name         = "genai-monthly"
  budget_type  = "COST"
  limit_amount = "5000"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"
  notification { comparison_operator = "GREATER_THAN"
    threshold = 80  threshold_type = "PERCENTAGE"
    notification_type = "ACTUAL"
    subscriber_email_addresses = ["finops@example.com"] }
}

aws wellarchitected create-workload \
  --workload-name "genai-rag-platform" \
  --description "LLM+RAG+agentic platform" \
  --environment PRODUCTION \
  --lenses "wellarchitected" "genai" \
  --review-owner "security@example.com"