App, identity and observability
Companion to the platform page and the code architecture page: what the app does, who can log in to what, which GraphQL features it shows, how it is observed with OpenTelemetry, and the build order. Prepared 2026-10-16. Phase 1 is built (the API, service kit, playground, CI and bootstrap infrastructure); later phases are design.
Decisions recorded 2026-10-16
- End state: a public demo of three things: the app (login + GraphQL calls), Argo CD, and the OpenTelemetry dashboards.
- Repo: the app lives in
eks-gitops-demo, structured as clean architecture on a shared service kit. - Environments: dev only. Promotion to prod is explained, not shown.
- Request flow: a participant raises a support request against their plan; agency staff approve or reject it; approval reserves budget as a service booking; the provider later claims against the booking. Nobody may decide their own request.
- Public hosts on one ALB and the wildcard certificate, all signing in through Keycloak at
auth.frankbluemind.com:app.,argocd.andgrafana.frankbluemind.com. The Keycloak admin console is not routed publicly. - Identities:
zhaolin.sun(admin of Argo CD and Grafana, and a participant who raises requests),case.officer(staff approver),demo.guest(shared read-only login for reviewers). - Passwords: random per user but memorable (word-word-number-symbol), kept in Secrets Manager and put into the realm at import time; never in Git (see Identity).
- Keycloak runs in production mode with its own database on RDS, so users persist; RDS therefore moves to phase 2.
- Dashboards: self-hosted Grafana as the UI over AWS-managed backends (X-Ray, Managed Prometheus, CloudWatch Logs).
- Terraform: applied from the laptop for the initial build; the IaC pipeline (plan on PR, apply on merge via GitHub OIDC) was added on 2026-10-16 and owns changes under
infra/from then on. - Prerequisites: an IAM Identity Center admin for running Terraform (not root), and the domain's contact-email verification.
1. App domain: a support-funding service
A participant receives a plan with a budget per support category. The participant (or a nominee acting for them) raises a support request: category, description, estimated amount and optionally a preferred provider. Agency staff approve or reject it with a reason; approval reserves part of the budget as a service booking for a provider. The provider delivers the service and claims payment against the booking. The plan's management type decides who may claim. A claim is a payment request against a budget, not an insurance claim.
Grey = people and organisations (also the callers); blue = records. Approving a support request creates the service booking. Every read and write passes the relationship checks in the use cases: nominee delegation, provider booking, plan management type, and separation of duties.
| Concept | Name | Note |
|---|---|---|
| Participant number | schemeNumber | Later deprecated in favour of memberNumber to show schema evolution |
| Staff role | AGENCY_STAFF in the GraphQL enum; agency_staff as the realm role in tokens | |
| Support categories | DAILY_LIVING / THERAPY / EQUIPMENT | |
| Realm, module | Realm demo, module eks-gitops-demo | Issuer https://auth.frankbluemind.com/realms/demo |
| Support request | SupportRequest with SUBMITTED / APPROVED / REJECTED | The request → decision → booking → claim workflow |
| Plan management | A management type on each plan | A real authorization rule: who may submit claims |
| Service booking | ServiceBooking as a first-class type | Created when a support request is approved |
2. GraphQL showcase
| Feature | Schema element | Status | Why it matters |
|---|---|---|---|
| Custom scalars | Date, Money | already | Validation at the edge; no float money |
| Directive-based auth | @auth(requires: [...]) on mutations | already | Coarse role gate declared in the contract |
| Nested resolvers + field masking | dateOfBirth null for providers; only booked budgets | already | Authorization on every path, not just the entry query |
| Relay cursor pagination | ClaimConnection, PageInfo | already | Stable paging for portal and mobile lists |
| Union payloads (errors as data) | SubmitClaimPayload = Claim | ValidationError | BudgetExceededError | already | Clients branch on typed outcomes, not error strings |
| Workflow with a domain-rule error | decideSupportRequest → SupportRequest | SelfApprovalNotAllowedError | ValidationError | built | A business rule (separation of duties) reaches the client as a typed outcome |
| Interfaces + global lookup | interface Node, node(id:), nodes(ids:) (at most 50) | built (Phase 5) | Client caches (Relay, Apollo) refetch any object by id. node reads through the same use cases as the typed queries: an id the caller may not read is null, exactly like participant(id:), and masked fields stay masked |
| Shared interface across types | interface Party { id displayName } (Participant, Provider) | built (Phase 5) | Polymorphic queries with fragments |
| Union search | search(term, first) → [Participant | Provider | Claim] | built (Phase 5) | One endpoint, many result types, still authorized per item: each type is searched inside the caller's scope. The term (2–100 characters) is a bind parameter and %/_ match literally; at most 50 results |
| Filter and sort inputs | claims(filter: ClaimFilter, orderBy: ClaimOrder), also on Plan.claims | built (Phase 5) | Typed query options instead of ad-hoc arguments. Keyset pagination in every order (submitted time or amount, ascending or descending, id as tie-break); the cursor records its order, and a cursor from another order is BAD_USER_INPUT |
| Error interface | interface UserError { message path } on ValidationError, BudgetExceededError, SelfApprovalNotAllowedError, IdempotencyKeyReusedError | built (Phase 5) | Generic error rendering in clients; the payload unions keep their members |
| Idempotent mutation | submitClaim(input: { idempotencyKey }) | built (Phase 5) | A client retry can never pay twice: the same key and input return the first claim; the same key with a different input is IdempotencyKeyReusedError. Keys are per user (primary key (subject, key), stored in the claim's transaction) and kept; concurrent retries store one claim |
| DataLoader batching | claims → provider, plan → budgets | built | Removes the N+1 query problem; visible in traces |
| Subscriptions | supportRequestStatusChanged(planId), claimStatusChanged(planId) over WebSocket (graphql-transport-ws), fanned out with Postgres LISTEN/NOTIFY | built (Phase 6) | The requester sees the decision live, whichever pod handled it; the ALB carries WebSockets. The bearer token goes in connection_init (invalid: close 4401; client not allowed: 4403); subscribing needs read access to the plan, and every update is re-read with the subscriber's access. At most 5 subscriptions per connection, a ping every 25 s (the ALB drops connections idle for 60 s), and the connection ends when its access token expires. Only subscriptions go over the socket; queries and mutations stay on POST, where the rate and time limits apply. |
| Schema evolution | Participant.schemeNumber @deprecated(reason: "Use memberNumber; removal after 2026-12-31") | built (Phase 5) | Versionless API lifecycle: add, deprecate, measure use (graphql_deprecated_field_usage_total{field}), remove. A CI check flags a pull request that breaks the schema unless it is labelled breaking-change-approved; advisory, since the free plan cannot require checks on a private repository (docs/schema-changelog.md) |
| Hardening | Depth and complexity limits; persisted queries: APQ (extensions.persistedQuery.sha256Hash) in the demo, a trusted-documents allowlist (PERSISTED_QUERIES=allowlist, persisted/queries.json) for first-party clients; introspection off outside dev; a strict Content-Security-Policy on every page | built (Phase 7) | Protects a public GraphQL endpoint from expensive or arbitrary queries; mobile clients send small hashes; pages run no inline code. Checked in Chrome: PKCE login and a GraphiQL query with 0 CSP violations |
| Integration adapter | approveClaim pays the claim through payments-svc with demo-api's own client-credentials token (demo-api-svc), the claim id as idempotency key; Claim.payment { status ref } is PENDING, PAID or FAILED | built (Phase 6) | The API as a consistent integration layer over back-end systems. The approval commits first; a failed payment leaves it standing (FAILED, audited as payment_failed). Two 2 s attempts with capped backoff (honouring Retry-After), one token refresh on a 401, and a circuit breaker that only unavailability can open. A replayed approval is refused before any payment, and concurrent approvals pay once (a conditional status update decides the winner). |
Try the showcase (in the playground)
# any object by id, through a shared interface
{ node(id: "P1") { __typename id ... on Party { displayName } ... on Participant { memberNumber } } }
{ nodes(ids: ["PR1", "PL1", "C1"]) { __typename id } }
# one search, three result types, each inside what the caller may read
{ search(term: "sun") { __typename ... on Party { id displayName } ... on Claim { id status amount } } }
# filtered and ordered keyset pagination: pass pageInfo.endCursor as after:
{ claims(first: 2, filter: { status: SUBMITTED }, orderBy: { field: AMOUNT, direction: DESC }) {
totalCount edges { node { id amount submittedAt } } pageInfo { hasNextPage endCursor } } }
# an idempotent retry (as riley.provider): run it twice, get the same claim id
mutation { submitClaim(input: { planId: "PL1", category: DAILY_LIVING, amount: "12.00",
serviceDate: "2026-04-20", idempotencyKey: "try-1" }) {
__typename ... on Claim { id } ... on UserError { message path } } }
SDL sketch of the new types (the design; the built schema is internal/adapter/graphql/schema/schema.graphqls)
interface Node { id: ID! }
interface Party { id: ID! displayName: String! }
interface UserError { message: String! path: [String!] }
enum PlanManagement { AGENCY PLAN_MANAGED SELF }
enum SupportCategory { DAILY_LIVING THERAPY EQUIPMENT }
enum SupportRequestStatus { SUBMITTED APPROVED REJECTED }
enum OrderDirection { ASC DESC }
enum ClaimOrderField { SUBMITTED_AT AMOUNT }
type Participant implements Node & Party {
id: ID!
displayName: String!
memberNumber: String!
schemeNumber: String! @deprecated(reason: "Use memberNumber; removal after 2026-12-31")
dateOfBirth: Date # null unless the caller may see it
plans: [Plan!]!
}
type Provider implements Node & Party { id: ID! displayName: String! bookings: [ServiceBooking!]! }
type ServiceBooking implements Node {
id: ID! plan: Plan! provider: Provider! category: SupportCategory! reserved: Money! remaining: Money!
}
type SupportRequest implements Node {
id: ID! plan: Plan! category: SupportCategory! description: String!
estimatedAmount: Money! preferredProvider: Provider
status: SupportRequestStatus! decisionReason: String
booking: ServiceBooking # set when approved
}
union SearchResult = Participant | Provider | Claim
input ClaimFilter { # submittedAfter is inclusive, submittedBefore exclusive (RFC 3339)
status: ClaimStatus category: SupportCategory providerId: ID submittedAfter: String submittedBefore: String
}
input ClaimOrder { field: ClaimOrderField! = SUBMITTED_AT direction: OrderDirection! = ASC }
input SubmitSupportRequestInput {
planId: ID! category: SupportCategory! description: String! estimatedAmount: Money! preferredProviderId: ID
}
input DecideSupportRequestInput { id: ID! approve: Boolean! reason: String! providerId: ID }
input SubmitClaimInput {
bookingId: ID! amount: Money! serviceDate: Date!
idempotencyKey: String # optional; same key + same input → the same Claim, never a second payment
}
type ValidationError implements UserError { field: String! message: String! path: [String!] }
type BudgetExceededError implements UserError { message: String! path: [String!] remaining: Money! requested: Money! }
type SelfApprovalNotAllowedError implements UserError { message: String! path: [String!] }
type IdempotencyKeyReusedError implements UserError { message: String! path: [String!] }
union SubmitSupportRequestPayload = SupportRequest | ValidationError | BudgetExceededError
union DecideSupportRequestPayload = SupportRequest | SelfApprovalNotAllowedError | ValidationError
union SubmitClaimPayload = Claim | ValidationError | BudgetExceededError | IdempotencyKeyReusedError
type Query {
node(id: ID!): Node
nodes(ids: [ID!]!): [Node]!
search(term: String!, first: Int = 10): [SearchResult!]!
claims(filter: ClaimFilter, orderBy: ClaimOrder, first: Int = 20, after: String): ClaimConnection!
}
type Mutation {
submitSupportRequest(input: SubmitSupportRequestInput!): SubmitSupportRequestPayload! @auth(requires: [PARTICIPANT, NOMINEE])
decideSupportRequest(input: DecideSupportRequestInput!): DecideSupportRequestPayload! @auth(requires: [AGENCY_STAFF])
submitClaim(input: SubmitClaimInput!): SubmitClaimPayload! @auth(requires: [PROVIDER])
}
type Subscription {
supportRequestStatusChanged(planId: ID!): SupportRequest!
claimStatusChanged(planId: ID!): Claim!
}
Plan, Claim and the connection types also implement Node; existing types are unchanged apart from the renames. The self-approval check is a domain rule (see the code page); the @auth directive only gates the role.
3. Identity and access
Public host (one ALB, *.eksgitops.click cert) | Serves | Login |
|---|---|---|
app.frankbluemind.com | SPA + /graphql (demo-api) | Keycloak, Authorization Code + PKCE |
argocd.frankbluemind.com | Argo CD UI | Keycloak SSO (OIDC client argocd); local admin disabled |
grafana.frankbluemind.com | Grafana dashboards | Keycloak SSO (generic OAuth); optional public dashboard link |
auth.frankbluemind.com | Keycloak realm demo: only /realms/* and /resources/* | The admin console is not routed publicly |
- Keycloak in production mode (
start, notstart-dev) withKC_DB=postgrespointing at its own database and user on the RDS instance, so users, credentials and sessions survive pod restarts.KC_HOSTNAME=https://auth.frankbluemind.com, forwarded headers trusted from the ALB, health on the management port. The bootstrap admin password is generated into Secrets Manager. - Realm as code: keycloak-config-cli runs as an Argo PostSync Job after Keycloak is healthy. It reads
config/keycloak/realm.yamlfrom the gitops repo and applies the difference idempotently on every sync: clients, roles, groups, users, mappers, brute-force protection. Keycloak's built-in--import-realmonly imports when the realm does not exist yet, so later Git changes would never apply. - Groups claim: a client scope
groupswith a group-membership mapper puts group names into the ID token; Argo CD and Grafana map groups to roles.
| User | App | Argo CD | Grafana | Purpose |
|---|---|---|---|---|
zhaolin.sun | realm role participant, linked to participant P1 | argocd-admins → role:admin | grafana-admins → Admin | Owner; raises support requests |
case.officer | agency_staff | no access | no access | Approves or rejects requests and claims |
demo.guest | participant, linked to demo participant P3 | argocd-readonly → role:readonly | grafana-viewers → Viewer | Shared reviewer login: raise requests, run GraphQL queries, look but not touch |
sam.nominee / riley.provider | nominee (acts for P1) / provider (PR1, booked on P1) | no access | no access | Show field masking per role |
| Keycloak client | Type | Used by | Notes |
|---|---|---|---|
spa | Public, Authorization Code + PKCE | The SPA in the browser | Redirect URI on app.frankbluemind.com; an audience mapper adds demo-api |
demo-api | Audience only (no login) | demo-api's token check | Access tokens must carry aud: demo-api |
demo-api-svc | Confidential, client credentials | demo-api → payments-svc | Audience payments-svc; secret generated in Secrets Manager, delivered by ESO |
argocd | Confidential OIDC | Argo CD | Groups claim; secret generated → ESO → argocd-secret |
grafana | Confidential OIDC | Grafana | Groups claim → role_attribute_path; secret generated → ESO |
Passwords. The five demo users get random passwords that stay memorable (word-word-number-symbol, from a CSPRNG). make secrets-seed writes them to Secrets Manager (demo/keycloak/users); an ExternalSecret hands them to the realm Job as environment variables and keycloak-config-cli substitutes them at import time, re-setting them on every import. Git and the realm ConfigMap hold only $(env:PW_<USERNAME>) placeholders, so Argo CD's read-only role, which can read ConfigMaps, learns nothing. The generated client secrets above take the same path and are never human-typed.
Accepted risk: a public site with shared demo logins. Cheap mitigations: Keycloak brute-force detection; guests are read-only in Argo CD and Grafana; only zhaolin.sun is an admin; the Keycloak admin console is not routed; optionally an ALB IP allowlist on argocd.frankbluemind.com; rotate the passwords after use (ROTATE_DEMO_PASSWORDS=1 make secrets-seed, then sync the realm).
Production path: no shared logins, and TOTP MFA required for admin groups.
config/keycloak/realm.yaml (keycloak-config-cli, excerpt)
realm: demo
enabled: true
bruteForceProtected: true
failureFactor: 10
groups:
- name: argocd-admins
- name: argocd-readonly
- name: grafana-admins
- name: grafana-viewers
roles:
realm:
- name: participant
- name: nominee
- name: provider
- name: agency_staff
users:
- username: zhaolin.sun
enabled: true
realmRoles: [participant]
groups: [/argocd-admins, /grafana-admins]
attributes: { participant_id: [P1] }
credentials:
- { type: password, temporary: false, value: "$(env:PW_ZHAOLIN_SUN)" } # Secrets Manager demo/keycloak/users
- username: case.officer
enabled: true
realmRoles: [agency_staff]
- username: demo.guest
enabled: true
realmRoles: [participant]
groups: [/argocd-readonly, /grafana-viewers]
attributes: { participant_id: [P3] }
clients:
- clientId: argocd
publicClient: false
redirectUris: ["https://argocd.frankbluemind.com/auth/callback"]
defaultClientScopes: [profile, email, groups]
Argo CD and Grafana SSO settings (excerpt)
# argocd-cm
url: https://argocd.frankbluemind.com
admin.enabled: "false" # after SSO is verified
oidc.config: |
name: Keycloak
issuer: https://auth.frankbluemind.com/realms/demo
clientID: argocd
clientSecret: $oidc.keycloak.clientSecret # key in argocd-secret, written by ESO
requestedScopes: ["openid", "profile", "email", "groups"]
# argocd-rbac-cm
policy.default: "" # no access unless a group grants it
policy.csv: |
g, argocd-admins, role:admin
g, argocd-readonly, role:readonly
# grafana.ini (Helm values)
[server]
root_url = https://grafana.frankbluemind.com
[auth]
disable_login_form = true
[auth.generic_oauth]
enabled = true
name = Keycloak
client_id = grafana
client_secret = $__file{/etc/secrets/grafana-oidc/client_secret}
scopes = openid profile email groups
auth_url = https://auth.frankbluemind.com/realms/demo/protocol/openid-connect/auth
token_url = https://auth.frankbluemind.com/realms/demo/protocol/openid-connect/token
api_url = https://auth.frankbluemind.com/realms/demo/protocol/openid-connect/userinfo
role_attribute_path = contains(groups[*], 'grafana-admins') && 'Admin' || contains(groups[*], 'grafana-viewers') && 'Viewer'
role_attribute_strict = true # no matching group → no access
4. Observability: OpenTelemetry metrics and traces
Decision: the app speaks only OpenTelemetry (OTLP). An in-cluster ADOT collector (AWS Distro for OpenTelemetry) forwards traces to AWS X-Ray and metrics to Amazon Managed Service for Prometheus; logs go to CloudWatch Logs through Fluent Bit. Self-hosted Grafana (one small pod, installed by Argo) is the UI over all three, with Keycloak SSO. The collector is the single seam for data, Grafana the single seam for viewing: changing a backend is config, never app code.
Grafana runs in the same cluster; it is drawn on the right because it only reads from the backends.
| Layer | Choice | Notes |
|---|---|---|
| App instrumentation (built) | OpenTelemetry Go SDK in the kit (internal/platform/otel), OTLP gRPC to the collector |
|
| Collector | ADOT collector (Helm chart, installed by Argo) | A Kubernetes object, so Argo owns it (the IaC rule). Pod Identity grants X-Ray write and AMP remote write. Memory limiter + batch processors. |
| Traces | AWS X-Ray | Viewed in Grafana (X-Ray data-source plugin) and in the CloudWatch trace map. Free tier covers demo volume. |
| Metrics | Amazon Managed Service for Prometheus (AMP) | PromQL, remote write with SigV4. Small cost at demo volume. |
| Logs | Fluent Bit → CloudWatch Logs | 7-day retention keeps the per-GB cost small. Grafana's CloudWatch data source queries them. |
| Dashboards + alerts | Grafana, self-hosted (Helm chart via Argo) |
|
| AWS resources | Terraform infra/eks/observability.tf | AMP workspace, CloudWatch log groups, Pod Identity roles for the collector and for Grafana, SNS topic for alerts. |
Alternatives
| Option | Good | Why not chosen |
|---|---|---|
| Amazon Managed Grafana as the UI | Zero-ops; best when the organisation already uses IAM Identity Center | Every viewer must sign in through IAM Identity Center or SAML, and it is billed per active user. That doesn't fit a public demo with shared reviewer logins. |
| CloudWatch Observability EKS add-on only (Container Insights + Application Signals) | Fewest pieces, CloudWatch-native, one add-on | Less of a Prometheus/Grafana story. KISS fallback if time runs short. |
| Self-hosted kube-prometheus-stack + Tempo/Loki | No licence cost, fully portable | Heavy for 2 nodes; you operate, upgrade and store the data |
| Third-party SaaS (Datadog, Grafana Cloud, Honeycomb) | Rich UX, no backend to run | Extra vendor and cost; easy later, because the same collector can export there |
What we measure and alert on
- RED per GraphQL operation (built): request rate, errors by
extensions.code, latency histogram (p50/p95/p99). - What counts as an error:
outcome="error"only when the server failed (INTERNAL,UNAVAILABLE, or an unknown code). The caller's mistakes and denials (BAD_USER_INPUT,FORBIDDEN,UNAUTHENTICATED, parse/validation/depth/complexity rejections,RATE_LIMITED) areoutcome="client_error"; business results returned as union members (e.g.BudgetExceededError) are data,outcome="ok". The error-rate alert counts server errors only, so a user typing a bad amount cannot page anyone. - Security signals: 401/403 counts, 429 rate-limit rejections, complexity and depth rejections.
- Dependencies: DB pool in-use/idle/max (built); payments-svc calls as client spans in the approval's trace (built); circuit-breaker state changes as warning logs (a breaker metric is not built).
- Business counters (planned): support requests raised, approved, rejected; claims submitted, approved; budget-exceeded outcomes.
- Runtime: Go heap, goroutines, GC pauses.
- Sampling: 100% in the demo; parent-based 10% in production.
- Alerts (Grafana alerting → SNS email): server error rate > 5% for 5 min (built); p95 > 500 ms and a spike in 401s (planned).
- One dashboard: RED per operation, HPA replica count, DB connections. The scaling load test is visible on it, and KEDA can later scale on the same AMP query (requests/s).
Go: wiring the kit (built)
// cmd/api/main.go: config comes from OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_SERVICE_NAME,
// OTEL_TRACES_SAMPLER_ARG; an empty endpoint keeps every provider a no-op.
shutdown, err := kitotel.Setup(context.WithoutCancel(ctx), cfg.Telemetry)
if err != nil {
return fmt.Errorf("telemetry: %w", err)
}
defer kitotel.Flush(log, shutdown, 5*time.Second) // after the drain, inside the grace period
verifier := oidc.New(authn.Config{ /* … */ Transport: kitotel.Transport(nil)})
gql := graphql.NewHandler(svc, graphql.Options{ /* … */
Telemetry: otelgql.New(otelgql.Options{ClientErrorCodes: []string{graphql.CodeDepthLimit}}),
})
// cmd/api/store.go: the adapter only sees pgx's QueryTracer interface.
pool, err := postgres.Open(ctx, cfg.DatabaseURL, postgres.PoolOptions{ /* … */ Tracer: otelpgx.NewTracer()})
err = otelpgx.RecordPoolStats(pool)
// cmd/api/router.go: the chain's tracing slot.
r.Use(httpmw.Recover(log), httpmw.RequestID, kitotel.HTTPMiddleware("demo-api"), httpmw.AccessLog(log, clientIP) /* … */)
ADOT collector config (minimal)
extensions:
sigv4auth: { region: ap-southeast-2, service: aps }
receivers:
otlp:
protocols:
grpc: { endpoint: 0.0.0.0:4317 }
processors:
memory_limiter: { check_interval: 1s, limit_percentage: 75, spike_limit_percentage: 20 }
batch: {}
exporters:
awsxray: { region: ap-southeast-2 }
prometheusremotewrite:
endpoint: https://aps-workspaces.ap-southeast-2.amazonaws.com/workspaces/<workspace-id>/api/v1/remote_write
auth: { authenticator: sigv4auth }
resource_to_telemetry_conversion: { enabled: true }
service:
extensions: [sigv4auth]
pipelines:
traces: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [awsxray] }
metrics: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [prometheusremotewrite] }
Cost at demo volume: Grafana is one small pod; Managed Prometheus and X-Ray cost cents (X-Ray free tier); CloudWatch Logs per GB ingested, kept small by 7-day retention.
5. Public-demo guardrails
- Least privilege for guests everywhere:
demo.guestis a participant in the app,role:readonlyin Argo CD (cannot sync, delete or see secrets), and Viewer in Grafana. - Admin only for
zhaolin.sunin Argo CD and Grafana; the Keycloak admin console is not routed publicly. - Keycloak brute-force detection locks an account after repeated failures.
- API protection: the rate-limit middleware (token bucket per IP and subject) plus GraphQL depth and complexity limits.
- Cost: the budget alert (~US$60/month) and
make eks-downbetween sessions. - Optional: AWS WAF managed rule groups on the ALB (about US$5/month per web ACL plus per rule group), and an IP allowlist for
argocd.frankbluemind.com. - After use: rotate every demo password and client secret.
6. Phased plan, app and infrastructure together
Each phase ends with its pass/fail checks run against the live system; the next phase starts only when every box is ticked.
| # | Build | Pass/fail checks |
|---|---|---|
| 1 | Repo eks-gitops-demo: the app on clean architecture and the platform kit; CI → ECR |
|
| 2 | EKS + Argo CD + RDS + Keycloak (production mode on RDS, realm as code with zhaolin.sun, case.officer, demo.guest; Argo SSO) + ALB/TLS for app., argocd., auth. |
|
| 3 | Postgres store (same RDS, own database and user) + DataLoaders + the support-request flow + plan management and service bookings; migration Job, HPA |
|
| 4 | OpenTelemetry: SDK + ADOT collector + X-Ray + Managed Prometheus + CloudWatch Logs (Fluent Bit) + self-hosted Grafana at grafana.frankbluemind.com |
|
| 5 | Schema showcase (interfaces/node, search union, filter/sort, UserError, idempotency, @deprecated) + secrets (SOPS → Secrets Manager → ESO → Reloader) |
|
| 6 | payments-svc on the same kit + subscriptions (LISTEN/NOTIFY) + integration resilience + PR security scans |
|
| 7 | Hardening (limits, persisted queries) + IaC pipeline; rotate demo passwords after use |
|