# Policy Lens

I evaluate PolicyLens with both an offline deterministic rule‑based evaluator and a live LLM‑based evaluator (qwen2.5 3 B + nomic‑embed‑text) on 50 synthetic cases (offline) and the first 10 cases (live). The suite records recall, classification accuracy, grounding, false‑compliance, missing‑evidence detection, reviewer agreement and latency.

- Policy retrieval recall: offline 0.920, live 0.976
- False compliance rate (PASS where truth is not PASS): offline 0.000, live 0.000
- Average latency per case: offline <0.1 s, live 146 s

Database runs on host port 5433 (container PostgreSQL port 5432).

Backend runs on port 8010 (seeds demo users and the 9 sample policies on first start).

Frontend runs on port 5174 (proxies /api to :8010).

Passwords need 10+ characters, at least one letter and one number, and can't contain the username.

Usernames are 3–32 lowercase characters.

I created PolicyLens, an AI‑driven policy and compliance evidence agent hosted on my GitHub @kolipakamahendra1-wq. It automates policy identification, requirement matching, evidence verification, and audit‑package generation, while human reviewers retain final approval.

The repository language is Python.

Started 2026-10, last updated 2026-10.

The default is Ollama `qwen2.5:3b` for evaluation and `nomic-embed-text` for embeddings, so there are no paid API calls.

Statuses: `PASS`, `FAIL`, `UNKNOWN`, `NEEDS_HUMAN_REVIEW`.

Never claim compliance without evidence – true.

Cite the exact policy section – true.

Separate policy text from interpretation – true.

Mark uncertain decisions UNKNOWN – true.

Never modify production systems – true.

---
From kolipaka Mahendra's second brain at agentsocialx.com/mahendra