ML Service

Standalone FastAPI service (:8001) that serves the RoBERTa model for priority inference. Used by the pipeline for template classification.

---

Endpoints

Method	Path	Description
GET	/health	Service + model status
POST	/predict	Classify template text

---

Health Check

curl http://localhost:8001/health
# {
#   "status": "healthy",
#   "model_loaded": true,
#   "model_version": "roberta_v1.1",
#   "device": "cpu"
# }

---

Inference

curl -X POST http://localhost:8001/predict \
  -H "Content-Type: application/json" \
  -d '{"text": "Database connection pool exhausted"}'

# Response
# {
#   "priority": "P1",
#   "confidence": 0.92,
#   "model_version": "roberta_v1.1"
# }

Request:

{
  "text": "string (template text, max 512 tokens)"
}

Response:

{
  "priority": "P1|P2|P3|P4",
  "confidence": 0.0-1.0,
  "model_version": "roberta_v1.1"
}

---

Architecture

flowchart LR
  PIPE[Pipeline] -->|HTTP POST /predict| MLSVC[ML Service :8001]
  MLSVC --> MODEL[(RoBERTa Model<br/>/app/model/log_priority_roberta)]
  MLSVC --> TOKENIZER[Tokenizer]

• Single process (FastAPI + Uvicorn)
• CPU inference (no GPU required)
• Model loaded at startup (~2s cold start)
• Thread-safe (transformers supports concurrent requests)

---

Configuration

Env Var	Default	Description
ML_MODEL_PATH	/app/model/log_priority_roberta	Model directory
ML_DEVICE	cpu	cpu or cuda
ML_MAX_LENGTH	512	Max tokens
ML_BATCH_SIZE	32	Inference batch size

---

Docker

ml-service:
  build:
    context: .
    dockerfile: backend/app/Ml_Model/Dockerfile
  volumes:
    - ./backend/app/Ml_Model/log_priority_roberta:/app/model/log_priority_roberta:ro
  deploy:
    resources:
      limits:
        memory: 2g

Model mounted read-only from host (built during training).

---

Model Details

Property	Value
Architecture	RobertaForSequenceClassification
Base	roberta-base (125M params)
Classes	4 (P1, P2, P3, P4)
Hidden Size	768
Layers	12
Attention Heads	12
Vocab Size	50,265
Max Position	514
Format	PyTorch + Safetensors

---

Training Pipeline

# 1. Label templates (devtools)
export NVIDIA_API_KEY=...
python devtools/label_with_llm.py

# 2. Fine-tune (notebook)
jupyter devtools/IaOps.ipynb
# → exports to /app/model/log_priority_roberta

# 3. Deploy
docker compose build ml-service consumer-pipeline
docker compose up -d ml-service consumer-pipeline

---

Monitoring

# Health
curl http://localhost:8001/health

# Test inference
curl -X POST http://localhost:8001/predict \
  -d '{"text": "Out of memory kill detected"}'

# Logs
docker compose logs -f ml-service

---

Troubleshooting

Issue	Check
model_loaded: false	Model dir missing, corrupt weights, OOM
High latency	CPU contention, increase ML_BATCH_SIZE
500 on predict	Input too long (>512 tokens), malformed JSON
Pipeline falls back to rule	ML service down, confidence < 0.80