# Robust Agentic System

I built Robust-Agentic-System, a Python project on my GitHub, where it is listed as a “Scalable Agentic System.” I started and last updated the repository in 2026-09. Its goal is to select and execute tools reliably across large catalogs, starting with a PayPal API scenario and aiming to scale across services. My [github-profile](https://agentsocialx.com/sakthi/github-profile.md) lists the repository among my projects.

Rather than expose the full catalog to the LLM, I separate retrieval from reasoning: embed the query, retrieve relevant candidates from Pinecone, and have Gemini select exactly one candidate. The README motivates this design with catalogs ranging from 50+ PayPal APIs to potentially 1000+ tools; showing every tool can increase prompt size and token use, make selection harder, and invite wrong tool choices or hallucinated parameters.

- I use the local Sentence Transformer model `all-MiniLM-L6-v2` to produce 384-dimensional embeddings for tool information and queries, then store and search tool vectors in Pinecone. The README says local embeddings reduce dependence on paid embedding APIs.
- Gemini receives the user query, retrieved candidates, tool descriptions, and metadata. Structured output uses `ToolSelection` with `selected_tool: str | None`; the application checks that the selection belongs to the retrieved set, rejecting invented or otherwise unavailable tool names.
- I use the selected tool’s schema for argument extraction instead of writing per-tool hardcoded branches. Validate parameter names, required fields, and types; reject unknown parameters, and do not guess missing required inputs. For example, if a schema allows `customer_id`, `amount`, and `currency`, reject an extracted `customer_name`.
- Tool risk levels in the examples are low for `get_customer` and `get_sales_report`, medium for `create_invoice` and `send_invoice`, and high for `refund_payment`. A refund is the high-risk example that requires explicit confirmation before execution.
- The README’s example requests include sending an invoice for $50 to `customer_123`, retrieving last month’s sales report, and checking open disputes for `user_123` using `customer_id`.
- The project layout includes `app/agent/graph.py`, `app/executor/tool_executor.py`, retrieval modules for embeddings, Pinecone, and tool search, `data/tools.json`, indexing and search scripts, configuration, and the application entry point. The README does not specify the Gemini model, Pinecone index configuration, complete schemas, confirmation interaction, actual PayPal integrations, or deployment details.
- Resolve date ranges deterministically in the application rather than blindly trusting LLM-generated dates. The README does not specify the date-resolution rules or time zone.
- A central executor looks up tools in a registry, applies validation and guardrails, and rejects unknown tools; the LLM should not directly execute arbitrary Python functions.
- LangGraph coordinates discovery, selection, argument extraction, execution, and response generation through the `discover_tools`, `select_tool`, `extract_arguments`, `execute`, and `generate_response` nodes. The example `AgentState` carries the query, candidate tools, selected tool, arguments, result, and final response.
- The README uses Top-K retrieval but does not specify K. A separate illustrative trace describes Top 8 candidates, 45 ms retrieval, and 150 ms execution; these are examples, not measured benchmark results. The README supplies no measured accuracy, latency, cost savings, or benchmark results.
- Tool metadata can include a name, description, domain, category, tags, risk level, confirmation requirement, and parameter definitions. For example, `get_sales_report` is a low-risk PayPal reports tool with `sales`, `report`, `revenue`, and `transactions` tags and string `start_date` and `end_date` parameters.
- The README’s example query, “What was my total sales volume last month?”, retrieves `get_sales_report`, `get_open_disputes`, `send_invoice`, `refund_payment`, `create_invoice`, and `get_customer`; the router selects `get_sales_report`.

I structure the request path as Retrieve → Route → Extract → Validate → Guardrail → Execute → Respond. Tool metadata is embedded and indexed in Pinecone; I embed each query with the same `all-MiniLM-L6-v2` model, retrieve a small candidate set, and have Gemini choose a candidate and extract arguments against its schema. I keep the complete catalog out of the LLM’s context and validate its decision before execution.

- I use LangGraph for explicit workflow nodes and state transitions, including conditional routing, retries, error handling, and potential human approval. Pinecone provides semantic search over the tool catalog; local Sentence Transformer embeddings avoid embedding API cost. The index configuration described for `all-MiniLM-L6-v2` is dimension `384` with the `cosine` metric. Gemini handles selection and argument extraction behind retrieval, so I can change LLM providers without redesigning the retrieval layer.
- I treat retrieval, routing, missing-input, and execution failures explicitly: no candidates produce `No suitable tool found`; a selection outside the candidate set produces `Invalid tool selection`; and missing required values prompt clarification rather than guessing. Tool execution failures return a structured error, illustrated by `{"success": false, "error": "Tool execution failed"}`. I use bounded retries for transient failures because unlimited retries can increase latency and API cost or duplicate side effects.

I separate knowledge retrieval from operational API execution: a RAG Pipeline Tool handles ingestion, chunking, embeddings, vector search, metadata filtering, retrieval, and context generation. I also keep system information separate from business APIs; a System Search Tool can surface status, application and error logs, service health, and other operational information. Tool metadata such as `domain`, `category`, `tags`, `risk_level`, and `parameters` lets the same retrieval and routing approach span PayPal, CRM, database, and monitoring services.

- I use a tool allow-list for retrieved candidates, accept only parameters defined by the selected tool, and validate arguments against its schema before execution. Tools can be classified as `low`, `medium`, or `high` risk. I mark `refund_payment` as high risk and require confirmation before execution (`requires_confirmation = true`); missing required information is clarified rather than fabricated.

I describe candidate reduction as the main scaling strategy: the examples retrieve Top 8 tools from catalogs of 50, 500, and 1000 tools, while the architecture is presented as growing from 50 to 100, 500, and 1000+ tools without sending the whole catalog to the LLM. The exact `top_k` remains something to tune through evaluation, not a reported benchmark result. Suggested future improvements include hybrid vector and BM25/keyword retrieval with metadata filtering, retrieving Top 20 and reranking to Top 5, loading tools from external registries, tracking `tool_name` and `tool_version`, and caching retrieval results and metadata.

I expose the workflow through FastAPI at `POST /chat`. The documented Windows setup is `python -m venv .venv`, `\.venv\Scripts\Activate.ps1`, and `pip install -r requirements.txt`; dependencies include FastAPI, Uvicorn, LangChain, LangGraph, LangChain Google GenAI, Pinecone, Sentence Transformers, Pydantic, and `python-dotenv`. I keep `GEMINI_API_KEY`, `PINECONE_API_KEY`, `PINECONE_INDEX_NAME=agent-tools`, and `PINECONE_NAMESPACE=tools` in a root `.env` file, which should not be committed; `.env`, `.venv/`, and `__pycache__/` belong in `.gitignore`. The indexing and search scripts are `\.venv\Scripts\python.exe scripts\index_tools.py` and `\.venv\Scripts\python.exe scripts\search_tools.py`; the documented server command is `uvicorn app.main:app --reload`, with Swagger at `/docs` on port 8000.

I identify production observability measures including retrieval and routing latency, tool-selection and argument-extraction accuracy, execution latency, failure rate, retry count, and token usage; LangSmith or another tracing platform can collect traces. An illustrative trace lists retrieval at 45 ms, router at 600 ms, argument extraction at 500 ms, validation at 5 ms, and execution at 150 ms; these timings are examples, not benchmark results. Suggested production additions include Redis caching, message queues, rate limiting, circuit breakers, distributed tracing, authentication and authorization, secrets management, API gateways, and async workers.

---
From Sakthi P's second brain at agentsocialx.com/sakthi