Hexus – Smart Log Manager
Multi-Tenant Observability & RCA Platform
A multi-tenant platform that turns noisy, high-volume logs into automated anomaly detection and AI root-cause analysis.
Multi-tenant FastAPI + React/TypeScript platform serving 8+ enterprise products, orchestrating log ingestion, ML anomaly detection, semantic retrieval and an agentic RCA pipeline, hardened with row-level security, OIDC SSO, secrets management and automated PII anonymization.
Engineering Brief
- My contribution
- The multi-tenant API, anomaly-detection lifecycle, agentic RCA pipeline, and security and deployment integration.
- Engineering difficulty
- Keep tenant boundaries and sensitive data intact while combining ingestion, model activation, retrieval and LLM reasoning.
- Outcome
- An internal observability platform with self-serve model workflows, anomaly detection, and automated RCA.
Engineering questions
The Problem
Incident triage across multi-tenant log streams was manual and slow, with root-cause analysis bottlenecked on tribal knowledge and no safe way to handle sensitive data.
System Architecture
Architecture design
Illustrative paths, not live telemetry.
Architecture overview. No node selected.
Full network reference
Operators
ITOps · SRE
Kind: client
ITOps and SRE users investigate incidents through the application interface.
Incoming (0)
None in this diagram
Outgoing (1)
- Micro-Frontendflow
Micro-Frontend
React · MFE
Kind: client
The federated React frontend exposes model management and training observability.
Incoming (1)
- Operatorsflow
Outgoing (1)
- FastAPI APIflow
FastAPI API
Multi-tenant
Kind: api
Coordinate ingestion, anomaly detection, semantic retrieval, training and activation through tenant-aware APIs.
Incoming (2)
- Micro-Frontendflow
- Securitydeploy
Outgoing (3)
- Agentic RCAflow / incident
- Anomaly MLflow / detect
- Observabilitytelemetry
Agentic RCA
LangGraph
Kind: agent
Run RCA as a guarded LangGraph workflow with typed tools and validation around model calls.
Incoming (2)
- FastAPI APIflow / incident
- Azure OpenAIfeedback
Outgoing (2)
- Azure OpenAIflow / reason
- PostgreSQLdata
Azure OpenAI
Reasoning
Kind: agent
Provide language-model reasoning within the RCA workflow; generated incident statements need grounding and validation.
Incoming (1)
- Agentic RCAflow / reason
Outgoing (1)
- Agentic RCAfeedback
Anomaly ML
IForest · LOF
Kind: ml
Use Isolation Forest and LOF for unsupervised anomaly detection, with supervised scoring as labels accumulate.
Incoming (1)
- FastAPI APIflow / detect
Outgoing (1)
- PostgreSQLdata
PostgreSQL
Row-Level Security
Kind: data
Store tenant data behind PostgreSQL Row-Level Security. Correct policies and privileged-access controls remain necessary.
Incoming (2)
- Anomaly MLdata
- Agentic RCAdata
Outgoing (0)
None in this diagram
Security
SSO · Vault · PII
Kind: ops
Combine Keycloak OIDC, Vault-managed secrets and ingestion-time Presidio/spaCy anonymization.
Incoming (0)
None in this diagram
Outgoing (1)
- FastAPI APIdeploy
Observability
Metrics · Logs
Kind: ops
Expose operational metrics and logs for the platform and its training workflows.
Incoming (1)
- FastAPI APItelemetry
Outgoing (0)
None in this diagram
Live Demo
Incident flow and evidence
Synthetic demonstration, not employer telemetry or real model execution. Each step maps a local fixture to the documented architecture; no remediation is executed.
Identify the signal
A service and severity give triage an initial scope. The documented API coordinates ingestion; database policies, not this parser, enforce tenant isolation.
Design decisionScore the fixture
Hexus describes unsupervised detection when labelled incidents are scarce, with supervised scoring added as labels accumulate.
Design decisionInspect redaction
The ingestion-redaction decision removes sensitive fields before persistence or model access, rather than relying on redaction at read time.
Design decisionInspect supporting signals
Context should remain inside tenant boundaries. The architecture documents API-to-agent and agent-to-database relationships, with RLS as a separate control.
Design decisionExplain the rule match
The documented LangGraph decision uses typed tools, timeouts and schema validation around model calls. The local fixture demonstrates a narrower boundary: known rule or human triage.
Design decisionReview the next step
An unmatched rule must remain unknown. The demo cannot confirm a root cause, approve a change, or act on infrastructure.
Design decision
What I Engineered
- Architected a multi-tenant FastAPI + React/TypeScript platform orchestrating ingestion, anomaly detection, semantic retrieval and root-cause analysis for 8+ enterprise products.
- Built an agentic RCA pipeline on LangGraph + Azure OpenAI with prompt-driven reasoning that minimises hallucinated incident statements.
- Engineered the full ML lifecycle: Isolation Forest / LOF unsupervised detection plus supervised scoring, with APIs for training, vectorization and model activation.
- Hardened persistence with PostgreSQL Row-Level Security and SQLAlchemy ORM across batch and streaming log collection.
- Shipped a federated micro-frontend (Vite module federation + Grommet) for model management and live training observability.
- Operationalised with Docker/Kubernetes, Keycloak OIDC SSO, HashiCorp Vault secrets, and Presidio + spaCy PII anonymization.
Impact & Scale
- Automated RCA + anomaly detection across multi-tenant log streams
- Tenant isolation designed around database Row-Level Security, with policy and privileged-access testing still required
- Automated PII detection and anonymization at ingestion; detection coverage still requires validation
- Self-serve ML lifecycle: train, vectorize and activate models via API
Evidence & Scope
The local RCA demo uses synthetic incidents, not employer telemetry. Component explanations and engineering decisions describe the platform workflow.
Source is HPE-internal and cannot be published. Architecture and trade-offs walked through on request.
Engineering Decisions
- 01
Enforced tenant isolation in PostgreSQL with Row-Level Security, not in the API layer.
Database policies reduce reliance on every API remembering a tenant filter, at the cost of query and operational flexibility. Row-Level Security still needs correct role configuration, policy tests and controls for privileged access; it does not replace a full threat model or security testing.
- 02
Led with unsupervised anomaly detection (Isolation Forest / LOF) before supervised scoring.
Labelled incident data is scarce and biased early on. Unsupervised detection delivers day-one value with no training set; supervised scoring folds in later as labels accumulate, avoiding a cold start where nothing is flagged until data exists.
- 03
Ran RCA as a guarded LangGraph state machine rather than a free-form prompt chain.
Typed tools, timeouts and schema validation around every model call keep reasoning auditable and isolate failures to a single node. That is what minimises hallucinated incident statements: the difference between a demo and something an SRE trusts at 3am.
- 04
Anonymised PII at ingestion with Presidio + spaCy, not on read.
Stripping sensitive data before it lands in storage or reaches the LLM is a stricter boundary than redacting at query time. The reprocessing overhead is far cheaper than the blast radius of a leak.
Stack
- FastAPI
- Expose ingestion, training, vectorization and model-activation workflows through APIs.
- PostgreSQL
- Centralize tenant policies in Row-Level Security instead of relying only on application filters.
- LangGraph
- Make RCA a guarded sequence of reasoning and tool steps.
- Presidio
- Detect and anonymize sensitive fields before storage and model access.
Source is HPE-internal and cannot be published. Architecture and trade-offs walked through on request.
Related work
Shared engineering patterns across separate systems, not runtime dependencies.
- HPE OpsNexus
Both place controls around model-assisted reasoning: OpsNexus proposes permission-aware graph and vector retrieval, while Hexus documents guarded RCA and ingestion-time redaction.
Research design; prospective targets, not measured outcomes.
- GTMScale
Both expose multi-tenant applications through APIs, with distinct documented trade-offs: Hexus centralizes tenant isolation in database policies; GTMScale separates service releases and tunes data access.
Enterprise Production
Open to AI Software Development & Platform Engineering roles
Need someone who ships systems like this into production?