Across aerial drone analytics, the comparison between vision‑LLMs and classical CNN/YOLO detectors is beginning to look like a trade‑off between structured efficiency and semantic flexibility rather than a simple accuracy leaderboard battle. YOLO’s evolution from v1 through v8 and into transformer‑augmented variants has been driven by exactly the kinds of requirements that matter in urban aerial scenes—real‑time detection, small object robustness, and deployment on constrained hardware. The comprehensive YOLO survey by Terven and Cordova‑Esparza systematically traces how each generation improved feature pyramids, anchor strategies, loss functions, and post‑processing to balance speed and accuracy, and emphasizes that YOLO remains the de facto standard for real‑time object detection in robotics, autonomous vehicles, surveillance, and similar settings. Parking lots in oblique or nadir drone imagery—dense, small, often partially occluded cars—fit squarely into the “hard but well‑structured” regime these models were built for.

Vision‑LLMs enter this picture from a different direction. Rather than optimizing a single forward pass for bounding boxes, they integrate large‑scale image–text pretraining and treat detection as one capability inside a broader multimodal reasoning space. The recent review and evaluation of vision‑language models for object detection and segmentation by Feng et al. makes that explicit: they treat VLMs as foundational models and evaluate them across eight detection scenarios—including crowded objects, domain adaptation, and small object settings—and eight segmentation scenarios. Their results show that VLM‑based detectors have clear advantages in open‑vocabulary and cross‑domain cases, where the ability to reason over arbitrary text labels and semantically rich prompts matters. However, when we push them into conventional closed‑set detection benchmarks, especially with strict localization requirements and dense scenes, specialized detectors like YOLO and other CNN‑based architectures still tend to outperform them in raw mean Average Precision and efficiency. In other words, VLMs shine when we want to say “find all the areas that look like improvised parking near stadium entrances” even if we never trained on that exact label, but they remain less competitive if the task is simply “find every car at 0.5 IoU with millisecond latency.”

A qualitative comparison study of vision and vision‑language models in object detection underscores this pattern from a different angle. Rather than only reporting mAP values, Rakic and Dejanovic analyze how vision‑only and vision‑language detectors behave when confronted with ambiguous, cluttered, or semantically nuanced scenes. They note that VLMs are better at leveraging contextual cues and language priors—understanding that cars tend to align along marked lanes, or that certain textures and shapes co‑occur in parking environments—but can suffer from inconsistent localization and higher computational overhead, especially when used in zero‑shot or text‑prompted modes. CNN/YOLO detectors, by contrast, exhibit highly stable behavior under the same conditions once they are trained on the relevant aerial domain: their strengths are repeatability, tight bounding boxes, and predictable scaling with resolution and hardware. For an analytics benchmark that cares about usable detections in urban parking scenes, this suggests that YOLO‑style models will remain our baseline for “hard numbers,” while VLMs add a layer of semantic interpretability and open‑vocabulary querying on top.

The VLM review goes further by explicitly varying finetuning strategies—zero‑prediction, visual fine‑tuning, and text‑prompt tuning—and evaluating how they affect performance across different detection scenarios. One of their core findings is that visual fine‑tuning on domain‑specific data significantly narrows the gap between VLMs and classical detectors for conventional tasks, while preserving much of the open‑vocabulary flexibility. In a drone parking‑lot scenario, that means a VLM fine‑tuned on aerial imagery with car and parking‑slot annotations can approach YOLO‑like performance for “find all cars” while still being able to answer richer queries like “highlight illegally parked vehicles” or “find under‑utilized areas in this lot” by combining detection with relational reasoning. But this comes at a cost: model size, inference time, and system complexity are higher than simply running a YOLO variant whose entire architecture has been optimized for single‑shot detection.

For aerial drone analytics stacks like the ones we are exploring, the emerging consensus from these surveys is that vision‑LLMs and CNN/YOLO detectors occupy complementary niches. YOLO and related CNN architectures provide the backbone for high‑throughput, high‑precision object detection in structured scenes, with well‑understood trade‑offs between mAP, speed, and parameter count. Vision‑LLMs, especially when lightly or moderately fine‑tuned, act as semantic overlays: they enable open‑vocabulary detection, natural‑language queries, and richer scene understanding at the cost of heavier computation and less predictable performance on dense, small‑object detection. The qualitative comparison work reinforces that VLMs are most compelling when the question isn’t just “is there a car here?” but “what does this pattern of cars, markings, and context mean in human terms?”. In a benchmark for urban aerial analytics that includes tasks like parking occupancy estimation, illegal parking detection, or semantic tagging of parking lot usage, treating YOLO‑style detectors as the quantitative ground‑truth engines and VLMs as higher‑level interpreters and judges would be directly aligned with what the current research landscape is telling us.

 In the aerial drone analytics space, the comparison between plain vision‑LLMs, ReAct‑style agents, and broader agentic frameworks is about how we want our system to behave under pressure: do we want a single powerful model that “understands” scenes, or an ensemble of agents that can plan, probe, and correct themselves over time. The recent UAV‑CodeAgents work is a clean illustration of the second camp. It builds a multi‑agent framework on top of large language and vision‑language models, using the ReAct (Reason + Act) paradigm to interpret satellite imagery, ground high‑level natural language instructions, and collaboratively generate UAV trajectories. A vision‑grounded pixel‑pointing mechanism lets agents refer to precise locations on aerial maps, while a reactive thinking loop supports iterative reflection and dynamic goal revision in evolving environments. Evaluated on large‑scale fire detection missions, this ReAct+agentic stack achieves a 93% mission success rate with an average mission creation time of 96.96 seconds at a low decoding temperature, demonstrating that structured, multi‑step reasoning and tool use can deliver high reliability without blowing up latency.  

 By contrast, pure vision‑LLMs, even when multimodal and fairly large, tend to be evaluated on perceptual or question‑answer tasks rather than mission‑level outcomes. Sapkota and colleagues’ broader work on multimodal LLMs in domains like agriculture underscores the pattern: general‑purpose or domain‑adapted vision‑language models excel at flexible perception and instruction following, but their performance is typically reported as accuracy on classification, detection, or description benchmarks, not as end‑to‑end success in complex workflows. dblp In a benchmarking context like ezbenchmark, which is inspired by TPC‑H’s workload‑centric philosophy, that distinction matters. A vision‑LLM can certainly answer “What structures do we see in this tile?” or “Which parcels are likely orchards?” with impressive zero‑shot competence, but those answers are rarely tied directly to operational metrics like “Did the mission achieve its analytic goal without re‑flight?” or “How many follow‑up queries or human corrections were needed?” The agentic literature, especially around UAVs, starts from those operational questions and works backward to architecture.  CatalyzeX

 The Agentic UAVs survey by Sapkota, Roumeliotis, and Karkee makes that shift explicit. They define Agentic UAVs as systems that integrate perception, decision‑making, memory, and collaborative planning to operate adaptively in real environments, with goal‑driven behavior and contextual reasoning as first‑class design targets.  CatalyzeX In their taxonomy, vision‑LLMs and other multimodal models are enabling technologies inside a larger agentic stack rather than the entire solution. Perception components transform aerial imagery and other sensor data into structured representations; cognitive agents plan and replan missions; control agents execute actions; and communication agents manage interaction with humans and other UAVs across domains like precision agriculture, construction, disaster response, environmental monitoring, and inspection.  From an effectiveness standpoint, the survey argues that these agentic stacks surpass traditional UAV autonomy by improving mission flexibility, learning capacity, and system‑level robustness, but they also incur more architectural complexity. For a benchmark like ezbenchmark or a spatio‑temporal query engine like SpatialSky, this implies that evaluating “just the vision‑LLM” only tells part of the story; we also want metrics that capture how an agentic wrapper uses perception, memory, and planning to deliver reliable analytics over time.  CatalyzeX

 UAV‑CodeAgents sits at the intersection of these ideas and gives we quantitative hooks to work with. It exemplifies a multi‑agent ReAct framework where each agent is powered by an LLM or VLM but constrained by a structured action space: interpret imagery, reference map locations, synthesize mission code, revise plans, and coordinate with peers. The authors show that fine‑tuning Qwen2.5‑VL‑7B on 9,000 annotated satellite images substantially improves spatial grounding, which is a direct nod to the strength of vision‑LLMs as perception cores.  Yet the headline numbers—93% success rate, roughly 97‑second planning times—are achievements of the full agentic system, not the VLM alone.  If we imagine swapping that ReAct framework into an ezbenchmark workload, the effectiveness metrics we would record are not only pixel‑ or object‑level accuracies but also how many reasoning–action iterations the agents need to converge, how often they recover from ambiguous instructions without human help, and how consistently they satisfy constraints akin to TPC‑H’s query semantics when operating over a drone scenes catalog.

 The broader survey of Agentic LLMs reinforces why that ReAct pattern has become so central. It distinguishes between “plain” LLM use—where the model simply maps prompts to outputs—and agentic use, where LLMs plan, call tools, manage memory, and interact with other agents in pursuit of goals. alphaxiv.org UAV‑CodeAgents is explicitly cited as an example of this agentic turn in UAV mission planning: multi‑agent ReAct plus vision‑language grounding yields scalable, autonomous mission generation with minimal supervision. alphaxiv.org When we transfer that lens back to benchmarking, we get a natural three‑way comparison. Pure vision‑LLMs are cost‑effective for single‑step perception and natural language querying; ReAct frameworks wrap those models in explicit “think–act–observe–think” loops that can interrogate data and tools; full agentic UAV architectures, as surveyed by Sapkota et al., extend this further by embedding ReAct‑like cycles into a distributed system that includes collaboration, persistent memory, and multi‑mission learning.  alphaxiv.org  Each step up the ladder tends to increase implementation cost and complexity but also improves mission‑level robustness and adaptability in domains that look a lot like the use cases in SpatialSky and what we are sketching in ezbenchmark—multi‑tile analytics, evolving spatio‑temporal queries, and feedback‑driven missions over large areas.

 For the specific kinds of workloads in ezbenchmark and SpatialSky—workload chains over a spatial schema, spatio‑temporal pattern detection, and comparative evaluation of alternative pipelines—the existing literature suggests a division of labor rather than a straight winner. Vision‑LLMs, especially when domain‑tuned like the Qwen2.5‑VL‑7B variant in UAV‑CodeAgents, serve as powerful perception and explanation modules, mapping imagery and schema‑level metadata into natural language and structured hints.  dblp ReAct frameworks, exemplified by UAV‑CodeAgents, convert that perception into iterative planning and tool use, achieving high mission success and bounded planning time.  Agentic UAV architectures, as surveyed by Sapkota and colleagues, frame everything as part of a larger ecosystem where agents can accumulate experience, coordinate across missions, and adapt to new tasks and domains.  CatalyzeX If we encode those three regimes as configurations in ezbenchmark—vision‑LLM only, vision‑LLM+ReAct controller, and full agentic stack—we can attach metrics that reflect what the literature actually measures: task‑level accuracy and descriptive quality for the VLM, convergence behavior and mission‑success rates for ReAct, and cross‑mission adaptability and system‑level robustness for the agentic frameworks.  alphaxiv.org  CatalyzeX

 In that sense, incorporating ReAct and agentic metrics into ezbenchmark is less about chasing a trend and more about turning the UAV and agentic AI survey results into concrete benchmark dimensions. UAV‑CodeAgents gives us  model of how to quantify ReAct‑based mission planning performance in aerial scenarios, including success rates and planning time under different reasoning temperatures.  The Agentic UAVs survey gives us a taxonomy of capabilities—goal‑driven behavior, contextual reasoning, collaborative planning—that we can translate into workloads and evaluation criteria at the analytics level.  CatalyzeX And the broader Agentic LLMs perspective explains why simply swapping in a bigger or better vision‑LLM will not give us the same system‑level behavior as a ReAct or agentic framework; what matters is how the model is embedded in a loop of reasoning, action, and feedback. alphaxiv.org Together, they give us a roadmap for evolving ezbenchmark from a TPC‑H‑inspired catalog of queries into a testbed that can meaningfully compare vision‑LLMs, ReAct controllers, and full agentic UAV stacks on the very kinds of aerial analytics workloads embodied in our own repository and in systems like SpatialSky. 

#codingexercise: CodingExercise-01-07-2026.docx 

 The ReAct family of frameworks, where agents interleave explicit reasoning with concrete actions, has become one of the most natural ways to structure aerial drone analytics once we move beyond static perception into mission‑ and workflow‑level intelligence. In the UAV literature, we see this most clearly in the distinction between traditional “sense–plan–act” autonomy and what Sapkota and colleagues call Agentic UAVs: systems that integrate perception, decision‑making, memory, and collaborative planning into goal‑driven agents that can adapt to context and interact with humans and other machines in a loop, not just execute precomputed trajectories. ReAct‑style agents fit neatly into this picture as the cognitive core: they look at aerial data and task context, think through possible interpretations and actions in natural language or a symbolic trace, then call tools, planners, or control modules, observe the results, and think again. For drone image analytics, that “reason + act” cycle is where scene understanding, query planning, and mission evaluation start to blur together.

Sapkota et al.’s survey is useful precisely because it doesn’t treat this as a single pattern but as a spectrum of agentic architectures for UAVs. At one end are perception‑heavy agents where the “act” step is little more than calling specialized detectors or segmenters over imagery; the ReAct loop becomes a way to sequence image analytics: detect, reflect on the result, refine the region of interest, detect again, and so on. In the middle are cognitive planning agents that take higher‑level goals—“inspect all bridges in this corridor,” “prioritize hotspots near critical infrastructure”—and use ReAct loops to decompose them into analyzable subproblems, continuously grounding their reasoning in visual and geospatial feedback. At the far end are fully multi‑agent systems where different agents specialize in perception, planning, communication, and oversight, coordinating via shared memories and negotiation; here, ReAct is no longer a single loop but a pattern repeated inside each agent’s internal deliberation and in their interactions with each other. Across these types, aerial image analytics is both the substrate (what perception agents operate on) and a source of constraints (what planning and oversight agents must respect).

UAV‑CodeAgents by Sautenkov et al. is arguably the clearest instantiation of a multi‑agent ReAct framework for aerial scenarios. Built explicitly on large language and vision‑language models, it uses a team of agents that interpret satellite imagery and natural‑language instructions, then iteratively generate UAV missions via a ReAct loop: agents “think” in natural language about what they see and what the instructions require, “act” by emitting code, waypoints, or tool calls, observe the updated plan and environment, and then continue reasoning. A key innovation is their vision‑grounded pixel‑pointing mechanism, which lets agents refer to precise locations on aerial maps, ensuring that each act step is anchored in real spatial structure rather than abstract tokens. This is not just a conceptual nicety; in their large‑scale fire‑detection scenarios, the combination of multi‑agent ReAct reasoning and grounded actions yields a reported 93% mission success rate with an average mission creation time of 96.96 seconds at a lower decoding temperature, showing that we can get both reliability and bounded planning latency when the ReAct loop is carefully constrained.

If we step back and treat these works as a de facto “survey” of ReAct variants for drone analytics, a few archetypes emerge. Single‑agent ReAct patterns appear when a single vision‑language model is responsible for both scene understanding and action selection, often in simpler or more scripted environments. Multi‑agent ReAct, as in UAV‑CodeAgents, distributes reasoning and action across specialized agents—one may focus on interpreting imagery, another on code synthesis for trajectory generation, another on constraint checking—with the ReAct loop dictating both their internal thought and their coordination. Sapkota et al. broaden this further by embedding ReAct‑like cycles into a layered cognitive architecture where perception, cognition, and control agents all perform their own micro “reason + act” sequences, coordinated through shared memory and communication protocols. In all cases, the ReAct pattern is what allows these systems to treat aerial imagery not as a static input to a one‑shot model, but as a dynamic environment that agents can interrogate, test, and respond to.

For ezbenchmark, which is inheriting TPC‑H’s workload sensibility into drone image analytics, these ReAct variants suggest natural metrics to encode into the benchmark. UAV‑CodeAgents already gives u two: mission success rate and mission creation time under a multi‑agent ReAct regime. Sapkota et al. implicitly add dimensions like adaptability to new tasks and environments, collaborative efficiency among agents, and robustness under partial observability, all tied to how effectively agents can close the loop between reasoning and action in complex aerial scenarios. Translating that into ezbenchmark means we can, for each workload, not only measure traditional analytics metrics (accuracy, latency, cost) but also evaluate how different ReAct configurations perform when used as controllers or judges over the same scenes: how many reasoning–action iterations are needed to converge on a correct analytic conclusion, how sensitive mission‑level outcomes are to the agent’s decoding temperature, and how multi‑agent versus single‑agent ReAct architectures trade off between planning time and success rate.

Framing ReAct this way turns it into a first‑class axis in our benchmark rather than a hidden implementation detail. A workload in ezbenchmark could specify not just “find all overloaded intersections in this area” but also the agentic regime under test: a single vision‑LLM performing a ReAct loop over tools, a UAV‑CodeAgents‑style multi‑agent system, or a layered Agentic UAV architecture where oversight and planning are separated. The metric is then not only whether the analytic answer matches ground truth, but how the ReAct dynamics behave: convergence speed, stabiliuty under repeated runs, and resilience to minor perturbations in input or prompt. The survey‑style insights from Agentic UAVs and the concrete results from UAV‑CodeAgents together give u enough structure to define those metrics in a principled way, letting ezbenchmark evolve from a static TPC‑H‑inspired harness into a testbed that can actually compare ReAct frameworks themselves as part of the drone analytics stack.

#Codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/IQBk3cia2bM4TY8StfsC2aAPASY17d3Z1rjw2-3b6Mr9rFo?e=HsHA3H

 Cost–effectiveness is where the romantic idea of “just use a giant vision‑LLM” runs into the hard edges of drone operations. When we look for explicit economic comparisons between vision‑LLMs used directly on aerial imagery and more structured agentic frameworks, we quickly discover that the literature is still thin: most papers report computational and operational efficiency (latency, success rate, mission duration), but stop short of a full dollar‑per‑mission analysis. Still, the numbers they do provide already hint at how the trade‑offs play out when we try to build something like ezbenchmark into a realistic pipeline.

UAV‑CodeAgents is a useful anchor because it is unambiguously an agentic framework: a team of language and vision‑language model–driven agents using a ReAct loop to interpret satellite imagery, ground natural language instructions, and generate detailed UAV missions in large‑scale fire detection scenarios. Rather than asking a single vision‑LLM to go from pixels to trajectories, the system delegates: one agent reads the task and context, another reasons about waypoints in map space, and others refine plans through iterative “think–act” cycles, all grounded by a pixel‑pointing mechanism that can refer to precise locations on aerial maps. From a cost perspective, this is clearly heavier than a single forward pass through a monolithic VLM, but the paper quantifies why developers might accept that overhead: at a relatively low decoding temperature, UAV‑CodeAgents achieves a 93% mission success rate with an average mission creation time of 96.96 seconds for complex industrial and environmental fire scenarios. Those two numbers—success rate and planning latency—are effectively stand‑ins for mission‑level cost: fewer failed missions and sub‑two‑minute planning windows translate into fewer re‑flights and less human babysitting.

In contrast, work that relies on vision‑LLMs alone for aerial or satellite reasoning generally reports per‑task accuracy and qualitative flexibility, but not system‑level success metrics. A vision‑LLM that can answer “Where are the highest‑risk areas in this scene?” or “Which roofs look suitable for solar?” in a single forward pass is computationally attractive in isolation: one model, one call, no orchestration overhead. However, without an agentic layer to manage tools, refine outputs, and correct itself, any errors must be caught either by humans or by additional guardrail logic that is usually not part of the evaluation. What UAV‑CodeAgents implicitly shows is that we can treat the additional compute for multi‑agent reasoning as a kind of insurance premium: more tokens and more calls per mission, but dramatically higher odds that the resulting trajectory actually satisfies operational constraints. When we factor in the cost of failed missions—wasted flight time, re‑runs, delayed detection—the agentic system’s 93% success rate looks less expensive than it first appears.

None of this means that agentic frameworks are always cheaper in a narrow cloud‑bill sense. A pure vision‑LLM approach keeps our architecture simple and our per‑call overhead low. We can batch images, run them through a single VLM, and get scene descriptions or coarse analytics with predictable latency. If our benchmark only cares about perception‑level accuracy on static tasks, that simplicity is compelling. But once we move toward workload‑level benchmarking—chains of queries, mission‑like sequences, or “LLM‑as‑a‑judge” roles—errors propagate. A cheap VLM judgment that nudges a pipeline in the wrong direction can incur downstream costs far larger than the initial savings. UAV‑CodeAgents’ design, where agents iteratively reflect on observations and revise mission goals, is essentially an explicit acknowledgement that paying for more reasoning steps up front can reduce expensive mistakes later.

For ezbenchmark, which inherits TPC‑H’s focus on whole workloads rather than micro‑tasks, this suggests a specific way to think about cost‑effectiveness studies. Instead of trying to price each VLM token or GPU second in isolation, we treat the combination of “analytics accuracy + mission success + human oversight time” as our cost metric, and then compare three regimes: vision‑LLM alone, vision‑LLM embedded as a component in an agentic judge, and a full multi‑agent ReAct‑style framework like UAV‑CodeAgents wrapped around our catalog and tools. The existing literature gives us at least one anchor point on the agentic side—around 97 seconds of planning with 93% success for complex missions—while the vision‑LLM‑only side gives us per‑task accuracy but typically omits mission‑level reliability. A genuine cost‑effectiveness study in our setting would fill that gap, measuring not only GPU minutes but also re‑runs, operator interventions, and time to trustworthy insight over a suite of benchmark workloads

What’s missing in current research, and where ezbenchmark could be genuinely novel, is a systematic, TPC‑H‑style analysis that treats agentic frameworks and vision‑LLMs as first‑class design choices and quantifies their end‑to‑end economic impact on drone image analytics. UAV‑CodeAgents proves that multi‑agent ReAct with vision‑language reasoning can deliver high mission success with bounded planning time; our benchmark can extend that logic to analytics and judging: how many agentic reasoning steps, how many tool calls, and how many vision‑LLM passes are worth spending to get one unit of “better decision” from a drone scene. Framed that way, cost‑effectiveness stops being an abstract question about model sizes and becomes something our framework can actually measure and optimize.

 The emerging survey literature on agentic AI for UAVs makes it clear that “AI agents for drone image analytics” is no longer a single pattern but a family of architectures, each carving up perception, reasoning, and control in different ways. Sapkota et al. introduce the term “Agentic UAVs” to describe systems that integrate perception, cognition, control, and communication into layered, goal-driven agents that operate with contextual reasoning and memory, rather than fixed scripts or reactive control loops. arXiv.org arXiv.org alphaxiv.org In their framework, aerial image understanding is only one layer in a broader cognitive stack: perception agents extract structure from imagery and other sensors; cognitive agents plan and replan missions; control agents execute trajectories; and communication agents coordinate with humans and other UAVs. This layered view is useful when we start thinking about agentic frameworks as “judges” for benchmarking: the judging capability can itself be an agent, sitting in the cognition layer, consuming outputs from perception agents and workload metadata rather than raw pixels alone. arXiv.org alphaxiv.org 

Within this broader landscape, vision–language–driven agents are a distinct subclass. Sapkota et al. explicitly highlight vision–language models and multimodal sensing as key enabling technologies for Agentic UAVs, noting that they allow agents to parse complex scenes, follow natural-language instructions, and ground symbolic goals in visual context. arXiv.org alphaxiv.org These agents differ from traditional planners in that they can reason over image and text jointly, which makes them natural candidates for roles like “mission explainer,” “anomaly triager,” or, in our case, “benchmark judge” for aerial analytics workloads. Instead of judging purely from numeric metrics, a vision–language agent can look at a drone scene, read a workload description, inspect candidate outputs, and form a qualitative judgment about which pipeline better captures the intended analytic semantics. 

UAVCodeAgents by Sautenkov et al. provides a concrete multi-agent realization of this vision–language–centric approach for UAV mission planning. arXiv.org arXiv.org arXiv.org Their system uses a ReAct-style architecture where multiple agents, powered by large language and vision–language models, interpret satellite imagery and high-level natural language instructions, then collaboratively generate UAV trajectories. arXiv.org arXiv.org arXiv.org A core feature is a vision-grounded pixel-pointing mechanism that lets agents refer to precise locations on aerial maps, and a reactive thinking loop that enables iterative reflection, goal revision, and coordination as new observations arrive. arXiv.org arXiv.org In evaluation on large-scale fire detection missions, UAVCodeAgents reaches a 93% mission success rate with an average mission creation time of about 97 seconds when operated at a lower decoding temperature, illustrating that a team of reasoning-and-acting agents, anchored in visual context, can deliver robust, end-to-end behavior. arXiv.org arXiv.org arXiv.org While their agents are designed to plan rather than judge, the architecture is the same kind we would co-opt for an evaluative role: a vision–language agent that can “look,” “think,” and “act” by querying tools or recomputing metrics before rendering a verdict. 

Across these works, we can roughly distinguish three archetypes of agents relevant to drone image analytics. First are perception-centric agents, effectively wrappers around detection, segmentation, or classification models that expose their capabilities as callable tools within an agentic framework. arXiv.org alphaxiv.org Second are cognitive planning agents, like those in UAVCodeAgents, which translate goals and visual context into action sequences, refine them through ReAct loops, and manage uncertainty through deliberation. arXiv.org arXiv.org arXiv.org Third—more implicitly in the surveys—are oversight or monitoring agents that track mission state, constraints, and human guidance, and intervene or escalate when anomalies arise. arXiv.org arXiv.org For ezbenchmark, the “judge” fits best in this third category: an oversight agent that does not control drones directly, but evaluates analytic pipelines and their outputs against goals, constraints, and visual evidence, possibly calling perception tools or re-running queries to validate its own judgment before scoring. 

Agentic surveys also emphasize the role of multi-agent systems and collaboration, which is directly relevant to how we might structure an evaluative framework. arXiv.org alphaxiv.org Instead of a single monolithic judge, we can imagine a committee of agents: one agent specialized in geospatial consistency (checking object counts, extents, and spatial relations); another focused on temporal coherence across flights; another on narrative quality and interpretability of generated reports; and a final arbiter that aggregates their recommendations into a final ranking of pipelines. Sapkota et al. note that multi-agent coordination enables UAV swarms to share partial observations, negotiate tasks, and adapt to dynamic environments more effectively than single-agent systems. arXiv.org alphaxiv.org Translated into benchmarking, multi-agent evaluation would let different judges stress-test different aspects of a pipeline, with the ensemble acting as a richer, more discriminative “LLM-as-a-judge” than any single model pass. 

What makes this particularly attractive for an ezbenchmark-style adaptation of TPCH is that the agentic literature already leans heavily into reproducibility and benchmarking. UAVCodeAgents, for example, is explicitly released with plans for an open benchmark dataset for vision–language-based UAV planning, making their evaluation setup a template for standardized mission-level tasks and metrics in an agentic setting. arXiv.org arXiv.org Sapkota et al. argue for a “foundational framework” for Agentic UAVs that spans multiple domains—precision agriculture, construction, disaster response, inspection—and call out the need for system-level benchmarks that assess not only perception accuracy but also decision quality, mission flexibility, and human–AI interaction quality. arXiv.org arXiv.org This is very close in spirit to a TPCH-style workload benchmark, except operating at the level of missions and workflows rather than isolated queries. If we treat each ezbenchmark workload as a “mission” over a drone scenes catalog, an agentic judge can be evaluated on how consistently its preferences align with human experts when comparing alternative pipeline implementations for the same mission. 

In practice, using these agent types as judges means giving them access to more than just model outputs. An evaluative agent would see raw or tiled imagery, structured detections from classical or neural perception models, SQL outputs over our catalog, and the natural-language description of the analytic intent. It could then behave much like a planning agent, but in reverse: instead of generating a mission, it generates probes—additional queries, spot checks on specific tiles, sanity checks on object distributions—that help it decide which pipeline better fulfills the workload semantics. This is exactly the kind of “Reason + Act” loop that UAVCodeAgents demonstrates, only the action space is benchmark tooling instead of flight waypoints. arXiv.org arXiv.org The survey of Agentic UAVs suggests such introspective, tool-using behavior is central to robust autonomy in the field; using it in a judging capacity extends the same philosophy to benchmarking, pushing ezbenchmark beyond static metrics toward a living, agent-mediated evaluation process. arXiv.org arXiv.org alphaxiv.org 

Seen through this lens, enhancing ezbenchmark with an agentic judge is less about bolting on a new feature and more about aligning with where UAV autonomy research is already heading. Agentic UAV surveys formalize the components we need—perception tools, cognitive controllers, communication layers—and UAVCodeAgents shows how multi-agent ReAct with vision–language reasoning can reach high reliability on complex aerial tasks. arXiv.org arXiv.org arXiv.org arXiv.org Our benchmark can exploit those same design patterns: treat specialized detectors and SQL workloads as tools, wrap them in agents that can look, think, and act over drone imagery and metrics, and then measure how well those agents serve in an evaluative role. In doing so, ezbenchmark evolves from a TPCH adaptation into a testbed for agentic judgment itself, letting us benchmark not only pipelines, but also the very agents that will increasingly mediate how humans and UAVs reason about aerial imagery. 

Our References besides citations above: