My InsightPilot repository started 2026-10, last updated 2026-11.
In InsightPilot, SQL and Python compute the numbers; insightpilot/agent/verifier.py checks the response before it is shown.
The language model is optional and can only explain computed results.
My default setup uses SQLite, offline LSA embeddings, template answers, and local JSONL traces; it needs no API key or database service.
My interfaces are a CLI, FastAPI service, and responsive browser UI, with re-runnable agent and retrieval reports written to results/.
I built InsightPilot as a small, verifiable AI data analyst. It ingests and profiles common tabular files, combines lexical and vector retrieval, and constrains database queries. I use github profile to track my broader GitHub projects.
My ingestion report includes inferred types, null percentages, repeated-value counts, and cleaning notes; supported formats are also available through register_loader.
InsightPilot retrieval combines BM25 plus vector similarity fused with Reciprocal Rank Fusion; offline LSA is the default and cached sentence-transformers vectors are optional.
My agent flow is guard, schema-aware plan, execute, respond, verify; LangGraph is used with a sequential fallback.
InsightPilot SQL safety uses schema-derived identifiers and values, validate_sql, read-only execution, a row cap, and a timeout.
The default database is SQLite; PostgreSQL is optional through DATABASE_URL. Local JSONL traces are the default, with optional OpenTelemetry OTLP spans. The supported workflow includes evaluation outputs that should be read from each generated run rather than inferred from the README.
While the UI tab is visible, it checks health every 5 seconds.
docker compose up --build
My optional PostgreSQL Compose setup uses docker-compose.yml and docker-compose.postgres.yml; the override waits for PostgreSQL’s health check and configures the API to use it.
PostgreSQL analytical queries use a read-only transaction, statement timeout, and the same result row cap; ingestion writes through the configured database connection.
My Render demo serves the UI and API from the same HTTPS origin.
The public-demo rate limit defaults to 30 requests per minute, independently for each endpoint.
The Render free service sleeps after 15 minutes idle, may take about a minute to wake, has 512 MB RAM, and uses a nonpersistent filesystem.
The live check retries cold-start health checks for up to 90 seconds; its rate-limit check makes 31 requests.
Setting GEMINI_API_KEY enables explanations; without it, deterministic templates are used, and the model does not plan or execute SQL or compute numbers.
The verifier rejects explanations containing unsupported numbers and falls back to a template.
My evaluation command writes results/eval_summary.json and results/agent_cases.csv.
The evaluation summary records the run date, engine, answer mode, configured embedder, agent metrics, and retrieval recall@k/MRR per retrieval mode; it includes offline LSA when IP_EMBEDDER=sbert.
PostgreSQL, pgvector, sentence-transformers, and OTLP export are unit-tested with mocks but were not verified against live services; Docker and a live PostgreSQL server were unavailable during the audit.
Local JSONL traces can contain question text; OpenTelemetry spans exclude question text, SQL, row data, and secrets as attributes.
The external API receives the question and bounded result context for LLM explanations; I should avoid sensitive data unless policy permits.
I use Python 3.12, install the regular and development requirements, then run ruff check . and python -m pytest -q. My CLI supports asking, searching, and ingesting files. The API can run locally, and the browser UI should be opened through FastAPI rather than directly through file:// or Live Server. On Windows, I inspect a process using port 8000 before stopping it and only stop it when its command line identifies this project. I can choose port 8001 with the launcher’s -Port 8001 option.
The Render demo disables uploads and trace listing, uses fictional sample data, and is not a production service. Its in-memory limiter is process-local and intended for a single-instance demo, not production abuse protection. Render storage is nonpersistent, and uploaded data should not be expected to persist.
I prefer clarification to guessing for unsupported or ambiguous questions. Numeric grounding checks confirm that answer numbers occur in computed facts, but do not prove the prose interpretation or selected intent is correct. Correlation is descriptive rather than causal, missing trend periods are not automatically filled in, and benchmarks describe only the small synthetic labelled corpus and query set.
Optional integrations include PostgreSQL/pgvector, sentence-transformers, OTLP export, and Gemini explanations. The default embedder is fully offline; sentence-transformers may need network access on first model use, and document vectors use a configurable cache. Local JSONL traces remain enabled by default, so I should protect the trace directory. I can regenerate the synthetic sample files with python scripts/make_sample_data.py.