Predictive Digital Twin Multi Uav Agriculture

I use this page to record the README’s feature matrix, academic documentation, license, and stated goals for my multi-UAV precision-agriculture framework. These are README descriptions; the supplied material does not independently verify that every capability is implemented or operational.

My feature matrix lists 25 components. The Multispectral Ingestion entry is Rasterio · NumPy (4-band GeoTIFF) | ✅ Verified.

My Vegetation Indices entry is NDVI · NDRE · NDWI · SAVI · EVI · MSAVI2 · CIRE | ✅ Verified.

My Semantic Segmentation entry is PyTorch DeepLabV3+ (4-channel backbone) | ✅ Verified.

My Explainable AI (XAI) entry is Grad-CAM pixel-level saliency | ✅ Verified.

My Photogrammetry entry is ORB+RANSAC stitching · Stereo DSM/CHM | ✅ Complete.

My Disease Forecasting entry is Fisher-Kolmogorov PDE · Directed GNN | ✅ Active.

My Treatment Optimizer entry is Q-Learning MDP · Knapsack · Monte Carlo VaR | ✅ Verified.

My Yield Forecasting entry is Monteith LUE · GDD · Heat-sterility index | ✅ Verified.

My Swarm Path Planning entry is Potential-field · Serpentine sweep · Stress waypoints | ✅ Verified.

My Stress-Proportional Dosing entry is Per-waypoint dosing ∝ local stress score | ✅ Active.

My GPU Spray Physics entry is PyTorch CUDA particle drift simulation | ✅ Active.

My GIS Mapping entry is Folium satellite basemap · K-Means zone clustering | ✅ Complete.

My Weather Risk entry is Open-Meteo REST · Spray feasibility windows | ✅ Active.

My Ground Control Station entry is MAVLink telemetry · QoS · mission map/replay | ✅ Complete.

My Live Mode entry is Real-time field state machine · alert engine | ✅ Active.

My WebRTC Video Streaming entry is aiortc · STUN/TURN · lock-free frame queue | ✅ Complete.

My Sensor Fusion entry is Extended Kalman Filter (GPS+IMU+baro) | ✅ Active.

My RGB-Only AI entry is Stage classification · disease · stress · weed | ✅ Complete.

My Mobile PWA entry is WebRTC · offline IndexedDB · service worker | ✅ Complete.

My UE5 / Unity Bridge entry is FastAPI · C# HISM material update client | ✅ Active.

My Digital Twin API entry is 20×20 NDVI grid · anomaly detection · heatmap | ✅ Active.

My PDF Reports entry is FPDF2 offline deterministic report compiler | ✅ Complete.

My Research Pipeline entry is 7-stage reproducible ML pipeline | ✅ Complete.

My Test Suite entry is 8 Pytest modules — all passing | ✅ Verified.

My Theme Sync entry is Auto dark/light mode on UI + matplotlib | ✅ Complete.

My academic_paper.tex is a Full IEEE double-column LaTeX paper covering all theoretical formulations, system architecture, experiments, and results.

I also list academic_paper_draft.md as a Markdown draft for collaborative editing and academic_paper_outline.md as a structured section-by-section outline.

My project states: MIT License — see LICENSE for full terms.

I describe the project as built for sustainable precision agriculture, with goals of reducing chemical inputs, maximising crop yield, and enabling autonomous multi-UAV field intelligence. The README does not provide detailed algorithms, model configurations, evaluation metrics, experimental results, or deployment details for these feature descriptions.

I document the repository’s installation and run workflow here, alongside its existing architecture and research notes. The README gives setup commands, dashboard and API launch instructions, a Unity/UE5 bridge procedure, classifier-training settings, and the intended regression-test checks; these are documented instructions, not independently verified operational results.

I list Python 3.11 / 3.12 / 3.13 as supported versions, Windows, Linux, or macOS as supported systems, and CUDA as optional with CPU fallback supported everywhere.

git clone https://github.com/Kev-seb/Predictive-Digital-Twin-Framework-for-Multi-UAV-Swarm-Precision-Agriculture.git
cd Predictive-Digital-Twin-Framework-for-Multi-UAV-Swarm-Precision-Agriculture
python -m venv .venv
.venv\Scripts\activate
source .venv/bin/activate
pip install -r requirements.txt

I launch the Streamlit dashboard with streamlit run src/dashboard/dashboard.py and open it at http://localhost:8501.

• My dashboard lists Vegetation Analytics for NDVI/NDRE/EVI maps and zone statistics; Segmentation & XAI for DeepLabV3+ stress segmentation and Grad-CAM; Disease Forecasting for PDE spread maps and a GNN transmission graph; and Treatment Optimizer for Q-Learning prescriptions and Knapsack scheduling.
• My dashboard also lists Swarm Operations for multi-UAV path planning and spray dispatch; Yield Forecast for a biomass model and GDD growth-stage tracker; Weather Risk for Open-Meteo spray-window feasibility; and a GIS Map with satellite basemap and zone overlays.
• My dashboard lists Ground Station panels for live telemetry, AI results, video, and QoS; Live Mode for real-time field state, alerts, and swarm dispatch; and a PDF Report offline mission-report compiler.

I start the Digital Twin API with python digital_twin_api.py; its documented Swagger UI address is http://<redacted>:8008/docs, so the actual host is not specified in the guide.

python -m uvicorn src.mobile_api.mobile_api_server:app --host <redacted> --port 8080

I instruct users to scan the dashboard QR code to open the companion app on a phone.

• For my UE5/Unity bridge, I start digital_twin_api.py on the same machine as UE5, add src/unity_integration/UnityBridgeClient.cs or its Blueprint equivalent to a level Actor, and set ApiBaseUrl = "http://<redacted>:8008".

My C# bridge polls /field/status every 500 ms and updates HISM material parameters NDVI, StressLevel, and GrowthStage per instance on the GPU.

python research/01_build_data_manifest.py
python research/02_extract_patches_with_provenance.py
python research/03_create_flight_disjoint_splits.py
python research/04_train_spectral_ablation.py
python research/05_stress_proxy_generator.py
python research/06_radiometric_normalization.py
python research/07_train_stress_classifier.py

I document that the seven research scripts are run in order from uav-crop-stress-intelligence/; they build the data manifest, extract patches with provenance, create flight-disjoint splits, train spectral ablations, generate stress proxies, normalize radiometry, and train the stress classifier.

python src/classification/train_classifier.py \
--task stage \
--data_dir data/processed/classification \
--epochs 50 \
--batch_size 16

I document the test command as .venv\Scripts\pytest tests/ -v.

• My test descriptions say test_ai_optimizer.py checks that Q-Learning converges and net reward exceeds baseline, while test_flight_physics.py checks that potential-field forces maintain d_ij > d_0 = 6 m across trajectories.
• My test descriptions say test_indices.py compares NDVI, SAVI, and NDWI outputs with reference arrays, and test_segmentation.py checks that the DeepLabV3+ forward pass produces the correct output shape.
• My test descriptions say test_swarm_coordination.py checks waypoint completion without collision, test_temporal.py checks plausible change-detection difference maps, and test_weather.py checks for a valid spray window.

My test_yield.py description says the biomass model outputs should fall within [0.0, 12.0] t/ha.

The disease-spread PDE is specified as $$\frac{\partial S}{\partial t} = \nabla \cdot (\mathbf{D} \nabla S) + rS\left(1 - \frac{S}{K}\right) - \vec{v}_{\text{wind}} \cdot \nabla S$$.

The treatment-optimization reward is $$R(s,a) = \text{Yield}(s') \cdot P_{\text{crop}} - \sum_i \text{Input}i \cdot C{\text{chem}} - \lambda \cdot \text{Stress}(s')$$.

Radiometric normalization uses Empirical line calibration + histogram matching across flights.

The disease-spread formulation is an Anisotropic advection-diffusion-reaction PDE with wind-skewed diffusion, humidity- and temperature-coupled growth, and NDVI-limited carrying capacity.

The directed GNN uses a Wind-direction alignment term in directed GNN edge weights.

The Q-learning reward balances Yield value minus chemical-input costs and a stress penalty.

Monte Carlo risk estimation uses N = 1000 rollouts for VaR and CVaR at 95th percentile.

The UAV planner models a Potential-field repulsive force between drones using inter-drone distance.

The spray rate scales with local stress, and the sweep path is a Boustrophedon path minimises re-crossing and total flight distance.

The spray engine uses a Droplet position update combines drone velocity, wind, turbulent velocity, and gravity; implementation specified as PyTorch CUDA.

Grad-CAM computes a Channel importance weight from gradient global average pool.

The Grad-CAM saliency map is ReLU of weighted sum of feature maps.

Biomass is estimated with a Monteith Light Use Efficiency model using PAR, fPAR, maximum LUE, temperature response, and water response.

GDD is the Daily mean of maximum and minimum temperature minus base temperature, summed across days.

The model's yield calculation is: Heat-adjusted harvest index counts days above 38°C; yield equals biomass times harvest index.

Sensor fusion uses a 9-element state [x, y, z, v_x, v_y, v_z, φ, θ, ψ]ᵀ fuses GPS, barometer, and 6-DoF IMU with per-sensor tuned process and measurement noise.

The manifest reports 20 flights, 8 244 image sets.

Patch extraction uses 224 × 224 patches; max 250 patches/flight; float16 .npy; ~1.6 GB total.

The dataset is 20 UAV flights across 2 paddy fields, 5 487 patches total, 4-band multispectral (G/R/RE/NIR) data.

The field areas are Field001 3 acres, Field002 2 acres.

The spectral ablation compares inputs using 5 seeds; RGB vs 3-band vs 4-band vs 5-channel; leakage quantification.

The research uses a 7-stage pipeline: 01_build_data_manifest.py, 02_extract_patches_with_provenance.py, 03_create_flight_disjoint_splits.py, 04_train_spectral_ablation.py, 05_stress_proxy_generator.py, 06_radiometric_normalization.py, 07_train_stress_classifier.py.

The pipeline provides 4 split types: temporal holdout, cross-field, LOFO, random patch; classifier validation is flight-disjoint.

Weakly supervised learning uses NDVI-threshold proxy label generation for weakly-supervised learning.

Reproducibility tooling at research/reproducibility/capture_environment.py snapshots full conda/pip environment, Git commit hash, and hardware specs.

I list the treatment optimizer as: Tabular Q-Learning MDP (ε-greedy), Knapsack multi-resource scheduling, Monte Carlo rollouts, VaR/CVaR risk estimation; 36 KB.

I describe the treatment recommender as a Rule-based prescription engine; outputs variable-rate fertilizer, fungicide, and irrigation recommendations per zone.

I list the yield predictor as: Monteith Light Use Efficiency biomass model + Growing Degree Days (GDD) thermal accumulation + heat-sterility harvest index.

I describe disease modeling as SIR/SEIR compartmental disease state evolution with weather coupling; Anisotropic Fisher-Kolmogorov PDE spread + Directed GNN transmission routing.

I list the digital-twin simulation components as: Persistent digital twin state manager; synchronises field JSON state to UE5 via REST; Full field physics simulator: crop growth, stress propagation, nutrient cycles; UAV 6-DoF kinematics, potential-field collision avoidance, serpentine sweep, stress-proportional waypoint generation; PyTorch GPU particle spray engine: droplet advection, wind drift, gravity, turbulence; Synthetic simulated UAV camera feed with configurable noise and altitude.

I describe the ground-station capabilities as Live field monitoring with real-time NDVI heatmap refresh; MAVLink telemetry display; real-time AI inference results streamed from edge UAV; WebRTC live video; link quality, latency, packet loss, and RSSI indicators; mission PDF reports with flight logs; interactive Folium map with live UAV position and waypoints; post-flight mission replay with timeline scrubbing.

I describe live operations as a State machine managing live UAV dispatch, field update cycles, and swarm coordination triggers; Threshold-based alert system: stress exceedance, battery low, geofence breach, link loss.

I list mission lifecycle components as Mission data model: waypoints, spray parameters, crop zones, flight plan; records flight telemetry, AI inferences, and images to disk; SQLite-backed mission persistence, query, and replay.

I describe the mobile app as a Fully offline-capable smartphone companion app; device GPS, accelerometer, gyroscope, compass; WebRTC peer connection; IMU and compass calibration; IndexedDB caching; offline usage and background sync.

I describe RGB inference as AI analysis even without multispectral data (standard camera); growth stage classification; visual disease symptom detection; canopy stress estimation from colour features; weed detection for precision herbicide application.

I list segmentation components as PyTorch DeepLabV3+ with 4-channel multispectral input backbone; Grad-CAM spatial attribution; softmax class probability maps; mIoU, class F1, confusion matrix computation; training with validation, checkpointing, LR scheduling.

I describe streaming as an aiortc-based signalling server with STUN/TURN support; GStreamer / OpenCV camera-source abstraction; Lock-free frame queue for sub-100 ms end-to-end latency.

I describe telemetry processing as Converts smartphone GPS/IMU data to MAVLink messages; Extended Kalman Filter fusing GPS, accelerometer, and barometer.

I list the digital-twin API as a FastAPI backend serving the 20 × 20 paddy field state grid with endpoints for per-instance updates, anomaly detection, NDVI heatmap, and full state dump.

I list the remote-sensing indices as NDVI $(NIR-R)/(NIR+R)$ for chlorophyll density & canopy vigour; NDRE $(NIR-RE)/(NIR+RE)$ for leaf nitrogen content (deep canopy); NDWI $(G-NIR)/(G+NIR)$ for canopy water content & waterlogging; all indices are vectorised with NumPy for zero-copy band operations.

I record the listed SAVI specification: | SAVI | $(NIR-R)(1+L)/(NIR+R+L)$ | Soil-corrected vegetation (L=0.5) |

The model directories list DeepLabV3+ weights in .pth format and EfficientNet-B0 weights in .pt format.

The RL optimizer benchmark script is labeled 33 KB.

The PWA companion app lists GPS/IMU/compass sensor access, WebRTC video streaming, sensor calibration, and IndexedDB offline storage.

The repository lists a Streamlit multi-tab dashboard with dark/light mode and interactive maps.

The utility scripts include parameter sensitivity analysis, a Unity HTTP API simulation client, end-to-end digital-twin validation, NDVI reference validation, a software-in-the-loop telemetry check, batch GeoTIFF processing, synthetic test-data generation, and dataset preparation.

The README lists treatment optimization using Q-Learning MDP, Knapsack, and Monte Carlo methods.

The listed yield predictor uses Monteith LUE for biomass and a GDD yield model.

The README lists SIR/SEIR disease evolution and a spatiotemporal epidemiology engine.

The repository tree lists an FPDF2 offline PDF report compiler.

The mobile companion app has a FastAPI server listed for its API.

The listed weather components are an Open-Meteo REST client and a spray-feasibility assessment engine.

The repository tree lists a full IEEE-format LaTeX paper, a Markdown paper draft, and a structured paper outline.

I describe this Python repository as an end-to-end framework for autonomous multi-UAV precision agriculture, connecting multispectral imagery, crop-stress analysis, swarm dispatch, ground-station telemetry, and a 3D digital twin. These are README descriptions; the supplied material does not independently verify that every capability is implemented or operational.

My repository is described as Python, with 2 forks.

The repository started 2026-05 and was last updated 2026-10.

The README lists Rasterio ingestion of 4-band GeoTIFF (G/R/RE/NIR) and support for NDVI, NDRE, NDWI, SAVI, EVI, MSAVI2, and CIRE vegetation indices.

The README describes semantic segmentation using PyTorch DeepLabV3+ with Grad-CAM XAI.

The listed disease-spread approach combines a Fisher-Kolmogorov PDE and Directed GNN.

Treatment optimization is described as Tabular Q-Learning MDP with Monte Carlo VaR/CVaR.

Swarm coordination is described with potential-field path planning, serpentine sweeps, and stress waypoints.

The listed live-telemetry technologies are MAVLink, WebRTC video, and WebSocket streaming.

The digital-twin description lists a FastAPI backend, UE5 / Unity C# bridge, and 20 × 20 HISM grid.

The README describes a 7-stage reproducible ML pipeline with spectral ablation.

The README claims 24 modules in src/.

The patch manifest is described as containing 5 487 patches.

The README describes the tests as a Pytest regression suite with 8 modules.

The architecture also lists EfficientNet-B0 classifier weights, flight-disjoint train/validation/test splits, data-manifest and patch-provenance scripts, radiometric normalization, stress-proxy generation, classifier training, and environment-capture tooling. The README provides no measured model accuracy, yield improvement, latency, flight performance, or other experimental results, and does not explain the rationale behind its stated design choices.

• Other described components include Q-Learning/Knapsack/Monte Carlo treatment optimization; Monteith LUE and GDD yield modeling; SIR/SEIR and spatiotemporal epidemiology; SQLite mission and telemetry storage; Streamlit dashboards; field and crop-growth simulation; UAV kinematics; GPU particle-spray simulation; Folium mapping; and threshold-based live alerts.
• The README also lists a mobile PWA with GPS/IMU/compass access, WebRTC, calibration, and IndexedDB offline storage; RGB crop-stage, disease, stress, and weed inference; segmentation metrics and training tools; ORB stitching, DSM and canopy CHM reconstruction; sensor fusion with an Extended Kalman Filter; weather-based spray-feasibility assessment; and a FastAPI mobile API.

The repository tree contains source-format line-break anomalies: “engines” appears separately after the digital_twin comment, service_worker.js and its comment appear on a separate line, and the academic_paper_outline.md line ends with “#” without a description. I preserve these as formatting anomalies rather than infer corrected structure.

Open questions remain about which listed capabilities are operational, the datasets and flight details behind the patch count, model training and measured results, model equations and validation, RL reward design and benchmarks, live-operation hardware and safety, supported engine versions and deployment, and test coverage and outcomes. The supplied README content ends after the architecture section separator, so later sections in its table of contents are not available here.