Robust Agentic System

I built Robust-Agentic-System, a Python project on my GitHub, where it is listed as a “Scalable Agentic System.” I started and last updated the repository in 2026-09. Its goal is to select and execute tools reliably across large catalogs, starting with a PayPal API scenario and aiming to scale across services. My github profile lists the repository among my projects.

Rather than expose the full catalog to the LLM, I separate retrieval from reasoning: embed the query, retrieve relevant candidates from Pinecone, and have Gemini select exactly one candidate. The README motivates this design with catalogs ranging from 50+ PayPal APIs to potentially 1000+ tools; showing every tool can increase prompt size and token use, make selection harder, and invite wrong tool choices or hallucinated parameters.

• I use the local Sentence Transformer model all-MiniLM-L6-v2 to produce 384-dimensional embeddings for tool information and queries, then store and search tool vectors in Pinecone. The README says local embeddings reduce dependence on paid embedding APIs.
• Gemini receives the user query, retrieved candidates, tool descriptions, and metadata. Structured output uses ToolSelection with selected_tool: str | None; the application checks that the selection belongs to the retrieved set, rejecting invented or otherwise unavailable tool names.
• I use the selected tool’s schema for argument extraction instead of writing per-tool hardcoded branches. Validate parameter names, required fields, and types; reject unknown parameters, and do not guess missing required inputs. For example, if a schema allows customer_id, amount, and currency, reject an extracted customer_name.
• Tool risk levels in the examples are low for get_customer and get_sales_report, medium for create_invoice and send_invoice, and high for refund_payment. A refund is the high-risk example that requires explicit confirmation before execution.
• The README’s example requests include sending an invoice for $50 to customer_123, retrieving last month’s sales report, and checking open disputes for user_123 using customer_id.
• The project layout includes app/agent/graph.py, app/executor/tool_executor.py, retrieval modules for embeddings, Pinecone, and tool search, data/tools.json, indexing and search scripts, configuration, and the application entry point. The README does not specify the Gemini model, Pinecone index configuration, complete schemas, confirmation interaction, actual PayPal integrations, or deployment details.
• Resolve date ranges deterministically in the application rather than blindly trusting LLM-generated dates. The README does not specify the date-resolution rules or time zone.
• A central executor looks up tools in a registry, applies validation and guardrails, and rejects unknown tools; the LLM should not directly execute arbitrary Python functions.
• LangGraph coordinates discovery, selection, argument extraction, execution, and response generation through the discover_tools, select_tool, extract_arguments, execute, and generate_response nodes. The example AgentState carries the query, candidate tools, selected tool, arguments, result, and final response.
• The README uses Top-K retrieval but does not specify K. A separate illustrative trace describes Top 8 candidates, 45 ms retrieval, and 150 ms execution; these are examples, not measured benchmark results. The README supplies no measured accuracy, latency, cost savings, or benchmark results.
• Tool metadata can include a name, description, domain, category, tags, risk level, confirmation requirement, and parameter definitions. For example, get_sales_report is a low-risk PayPal reports tool with sales, report, revenue, and transactions tags and string start_date and end_date parameters.
• The README’s example query, “What was my total sales volume last month?”, retrieves get_sales_report, get_open_disputes, send_invoice, refund_payment, create_invoice, and get_customer; the router selects get_sales_report.

I structure the request path as Retrieve → Route → Extract → Validate → Guardrail → Execute → Respond. Tool metadata is embedded and indexed in Pinecone; I embed each query with the same all-MiniLM-L6-v2 model, retrieve a small candidate set, and have Gemini choose a candidate and extract arguments against its schema. I keep the complete catalog out of the LLM’s context and validate its decision before execution.

• I use LangGraph for explicit workflow nodes and state transitions, including conditional routing, retries, error handling, and potential human approval. Pinecone provides semantic search over the tool catalog; local Sentence Transformer embeddings avoid embedding API cost. The index configuration described for all-MiniLM-L6-v2 is dimension 384 with the cosine metric. Gemini handles selection and argument extraction behind retrieval, so I can change LLM providers without redesigning the retrieval layer.
• I treat retrieval, routing, missing-input, and execution failures explicitly: no candidates produce No suitable tool found; a selection outside the candidate set produces Invalid tool selection; and missing required values prompt clarification rather than guessing. Tool execution failures return a structured error, illustrated by {"success": false, "error": "Tool execution failed"}. I use bounded retries for transient failures because unlimited retries can increase latency and API cost or duplicate side effects.

I separate knowledge retrieval from operational API execution: a RAG Pipeline Tool handles ingestion, chunking, embeddings, vector search, metadata filtering, retrieval, and context generation. I also keep system information separate from business APIs; a System Search Tool can surface status, application and error logs, service health, and other operational information. Tool metadata such as domain, category, tags, risk_level, and parameters lets the same retrieval and routing approach span PayPal, CRM, database, and monitoring services.

• I use a tool allow-list for retrieved candidates, accept only parameters defined by the selected tool, and validate arguments against its schema before execution. Tools can be classified as low, medium, or high risk. I mark refund_payment as high risk and require confirmation before execution (requires_confirmation = true); missing required information is clarified rather than fabricated.

I describe candidate reduction as the main scaling strategy: the examples retrieve Top 8 tools from catalogs of 50, 500, and 1000 tools, while the architecture is presented as growing from 50 to 100, 500, and 1000+ tools without sending the whole catalog to the LLM. The exact top_k remains something to tune through evaluation, not a reported benchmark result. Suggested future improvements include hybrid vector and BM25/keyword retrieval with metadata filtering, retrieving Top 20 and reranking to Top 5, loading tools from external registries, tracking tool_name and tool_version, and caching retrieval results and metadata.

I expose the workflow through FastAPI at POST /chat. The documented Windows setup is python -m venv .venv, \.venv\Scripts\Activate.ps1, and pip install -r requirements.txt; dependencies include FastAPI, Uvicorn, LangChain, LangGraph, LangChain Google GenAI, Pinecone, Sentence Transformers, Pydantic, and python-dotenv. I keep GEMINI_API_KEY, PINECONE_API_KEY, PINECONE_INDEX_NAME=agent-tools, and PINECONE_NAMESPACE=tools in a root .env file, which should not be committed; .env, .venv/, and __pycache__/ belong in .gitignore. The indexing and search scripts are \.venv\Scripts\python.exe scripts\index_tools.py and \.venv\Scripts\python.exe scripts\search_tools.py; the documented server command is uvicorn app.main:app --reload, with Swagger at /docs on port 8000.

I identify production observability measures including retrieval and routing latency, tool-selection and argument-extraction accuracy, execution latency, failure rate, retry count, and token usage; LangSmith or another tracing platform can collect traces. An illustrative trace lists retrieval at 45 ms, router at 600 ms, argument extraction at 500 ms, validation at 5 ms, and execution at 150 ms; these timings are examples, not benchmark results. Suggested production additions include Redis caching, message queues, rate limiting, circuit breakers, distributed tracing, authentication and authorization, secrets management, API gateways, and async workers.