Template Cache

The Template Cache is a three-tier caching system that stores ML priority predictions for Drain3 templates to avoid repeated RoBERTa inference.

---

Architecture

%%{init: {'theme':'default','themeVariables':{'primaryColor':'#1a73e8'}}}%%
graph TB
  subgraph PIPELINE["Pipeline Process"]
    L2[L2: In-Memory Dict<br/>~μs lookup]
  end

  subgraph SHARED["Shared Infrastructure"]
    L3[(L3: Redis<br/>ml:priority:{tid})]
    L1[(L1: PostgreSQL<br/>template_priority)]
  end

  EVENT[Event with template_id] --> L2
  L2 -->|Miss| L3
  L3 -->|Miss| L1
  L1 -->|Miss| ROBERTA[RoBERTa Inference]
  ROBERTA -->|Store| L1
  ROBERTA -->|Store| L3
  ROBERTA -->|Store| L2

---

Three Tiers

Tier	Technology	Scope	Latency	Persistence	Write Policy
L1	PostgreSQL (template_priority)	Global, durable	~5 ms	Permanent	INSERT ... ON CONFLICT
L2	Python dict (in-process)	Per replica	< 1 μs	Process lifetime	Direct dict write
L3	Redis (ml:priority:{tid})	Cluster-wide	< 1 ms	TTL 24h	SETEX

---

Cache Interface

# template_cache.py
class TemplateCache:
    def __init__(self, dsn: str, redis_client, model_version: str):
        self.pg_pool = psycopg_pool.AsyncConnectionPool(dsn)
        self.redis = redis_client
        self.model_version = model_version
        self.local_cache: dict[str, CachedPriority] = {}

    async def load_from_postgres(self):
        """Load all cached priorities at startup."""
        async with self.pg_pool.connection() as conn:
            rows = await conn.execute("""
                SELECT template_id, priority, confidence, model_version
                FROM template_priority
            """)
            for row in rows:
                self.local_cache[row.template_id] = CachedPriority(...)

    def get(self, template_id: str) -> CachedPriority | None:
        # L2: In-memory
        if template_id in self.local_cache:
            return self.local_cache[template_id]

        # L3: Redis (async in real impl, sync here for simplicity)
        cached = self.redis.get(f"ml:priority:{template_id}")
        if cached:
            self.local_cache[template_id] = parse_cached(cached)
            return self.local_cache[template_id]

        return None

    def set(self, template_id: str, template_text: str, 
            priority: str, confidence: float):
        # Only cache if confidence >= threshold
        if confidence < settings.ml_min_confidence:
            return

        cached = CachedPriority(
            priority=priority,
            confidence=confidence,
            model_version=self.model_version,
        )

        # L2
        self.local_cache[template_id] = cached

        # L3: Redis (TTL 24h)
        self.redis.setex(
            f"ml:priority:{template_id}",
            86400,
            orjson.dumps(cached.model_dump())
        )

        # L1: PostgreSQL (async, fire-and-forget)
        asyncio.create_task(self._persist_to_postgres(template_id, cached))

---

Conditional Storage

Critical: Only cache predictions with confidence ≥ 0.80

def set(self, template_id, template_text, priority, confidence):
    if confidence < settings.ml_min_confidence:  # 0.80
        return  # Don't cache low-confidence predictions

    # ... store in all tiers

Why? Low-confidence predictions pollute cache, reduce hit rate, propagate errors.

---

PostgreSQL Schema

-- migration 0011
CREATE TABLE template_priority (
    template_id TEXT PRIMARY KEY REFERENCES templates(template_id),
    template_text TEXT NOT NULL,
    priority VARCHAR(2) NOT NULL CHECK (priority IN ('P1','P2','P3','P4')),
    confidence REAL NOT NULL CHECK (confidence >= 0 AND confidence <= 1),
    model_version VARCHAR(100) NOT NULL,
    prediction_count INTEGER DEFAULT 0,
    trained_at DATE DEFAULT CURRENT_DATE,
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX ix_template_priority_priority ON template_priority (priority);

---

Cache Invalidation

Trigger	Action
Manual priority correction (PATCH /api/logs/{id}/priority)	DELETE FROM template_priority WHERE template_id = ? + redis.delete() + local_cache.pop()
Model version change	Full cache clear on startup (model_version mismatch)
TTL expiry	Redis auto-evicts (24h); L2 cleared on process restart

# API endpoint invalidates cache
async def update_event_priority(event_id, priority, reason):
    # ... update event ...
    if event.template_id:
        await template_cache.invalidate(event.template_id)

---

Metrics

Metric	Target	Source
Hit Rate	≥ 95%	ml_cache_hits / (ml_cache_hits + ml_predictions)
Known Templates	Growing	SELECT COUNT(*) FROM template_priority
Avg Confidence	> 0.85	AVG(confidence) FROM template_priority
Low Confidence	< 5%	COUNT(*) WHERE confidence < 0.80

# Via API
curl -s http://localhost:8000/api/ai-insights/stats | jq .ml_status
# "healthy" if hit_rate >= 95% and low_conf_count == 0

---

Troubleshooting

Issue	Diagnosis	Fix
Hit rate < 90%	Many new templates, or cache cleared	Check ml_predictions_total rate; wait for cache warmup
Stale predictions	Model updated but cache not cleared	Restart pipeline (clears L2), or manual redis.flushdb
PostgreSQL growing	Old templates never expire	Add retention job: DELETE WHERE updated_at < NOW() - INTERVAL '90 days'
Redis memory high	Too many keys	Reduce TTL, or add maxmemory-policy allkeys-lru

# Check cache stats
curl -s http://localhost:8000/api/ai-insights/engine | jq .ml