Technology Stack

LogSys is built on a deliberately boring, operationally proven open-source stack. Every choice is justified in Design Principles.

---

Stack Overview

flowchart TB
  subgraph INGEST["Ingestion"]
    VEC[Vector<br/>Rust]
    COL[Collectors<br/>Python/FastAPI]
  end
  subgraph CORE["Core Runtime"]
    KAF[Kafka<br/>Go/JVM]
    PG[PostgreSQL<br/>C]
    RED[Redis<br/>C]
  end
  subgraph APP["Application Layer"]
    API[FastAPI<br/>Python]
    PIPE[Pipeline<br/>Python]
    ML[RoBERTa<br/>PyTorch]
    UI[React 19<br/>TypeScript]
  end
  VEC --> KAF
  COL --> KAF
  KAF --> PIPE
  PIPE --> PG
  PIPE --> RED
  API --> PG
  API --> RED
  PIPE --> ML
  UI --> API

---

Runtime & Languages

Component	Language	Version	Purpose
Kafka	Go/JVM	3.7 KRaft	Message broker (no ZooKeeper)
PostgreSQL	C	16	Primary store, partitioning, OLAP
Redis	C	7	Cache + dedup + pub/sub
Vector	Rust	latest	HTTP → Kafka router
Backend API	Python	3.12	FastAPI REST + WebSocket
Pipeline	Python	3.12	6-stage consumer pipeline
ML	Python	3.12 + PyTorch	RoBERTa sequence classification
Frontend	TypeScript	5.x	React 19 SPA

---

Key Libraries

Layer	Library	Why
API	FastAPI + ORJSON	Speed, OpenAPI, type safety
ORM	SQLAlchemy 2.x	Async + sync, PostgreSQL dialect
Migrations	Alembic	Versioned schema changes
Cache	Redis-py	Lua scripts, pub/sub
Parser	Drain3	Online log-template mining
DataFrames	Polars	Fast in-memory feature engineering
ML	transformers / torch	RoBERTa sequence classification
Schedule	APScheduler	Poll-based collectors
Config	pydantic-settings	Typed env config
Logging	structlog	Structured JSON logs
Rate limit	slowapi	Redis-backed limits
Tests	pytest + ruff	Unit + integration

---

Frontend Libraries

Library	Purpose
React 19	UI framework
TanStack Router	Type-safe routing
TanStack Query	Server-state caching
TanStack Table	Data grids
Recharts	Charts
Tailwind CSS	Styling
Zustand	Client state
WebSocket	Real-time KPI stream

---

Infrastructure

Tool	Role
Docker Compose	Single-file full stack (dev + prod parity)
Docker	Container runtime
Alembic	Schema migrations
GitLab CI (planned)	Pipeline, build, push
Netlify	Frontend hosting (static)

---

Versioning & Code Quality

Tool	Use
Ruff	Lint + format
pytest	Test runner
knip	Dead-code detection (frontend)
pnpm	Frontend package manager
Git	Monorepo with dev branch

---

Related

• Architecture Overview
• Design Principles