Agentic judges for drone image analytics

Andrew Ng’s agentic workflow pattern—reflection, tool use, planning, and multi-agent collaboration—applies to drone-vision benchmarking. Reflection lets an agent critique and revise detections, captions, SQL answers, or mission reports. Tool use grounds reasoning in retrieval, code execution, geospatial operators, detector APIs, and database queries. Planning decomposes a workload into explicit steps before execution and supports replanning when a probe fails. Multi-agent orchestration assigns specialized roles: one agent checks geospatial consistency, another evaluates temporal coherence, another audits semantic alignment, and an arbiter aggregates evidence into a score.

Memory has been a design constraint. Loops let agents think; graphs let agents remember. A reflection loop can improve a single answer, but without persistent state the agent forgets why it inspected a frame, which detector disagreed, or which workload constraint failed. A graph turns those transient observations into reusable memory: frames, objects, captions, detections, SQL results, tool calls, critiques, and final judgments become linked evidence rather than buried transcript text. In ezbenchmark, this converts an agentic judge from a one-pass evaluator into a stateful audit system.

A practical build path is incremental. First, add one critique call after every generated answer or score; this is the highest-return change because it catches unsupported claims, missing visual evidence, and weak workload alignment before output is finalized. Second, expose the judge to tools: vector retrieval over frame evidence, SQL over the scene catalog, detector re-runs, spatial predicates, and temporal-neighbor comparisons. Third, require the judge to emit a structured plan before execution, such as JSON steps with expected evidence, tool calls, and failure conditions. Fourth, split evaluation across specialized agents and connect them through a shared graph store. The result is not merely a stronger prompt; it is an architecture in which weak models can outperform stronger single-pass models because the workflow supplies iteration, grounding, decomposition, and durable memory.

Agents are usually dedicated to perception, reasoning, and control in different ways. Sapkota et al. introduce the term “Agentic UAVs” to describe systems that integrate perception, cognition, control, and communication into layered, goal-driven agents that operate with contextual reasoning and memory, rather than fixed scripts or reactive control loops [1]. In their framework, aerial image understanding is only one layer in a broader cognitive stack: perception agents extract structure from imagery and other sensors; cognitive agents plan and replan missions; control agents execute trajectories; and communication agents coordinate with humans and other UAVs. This layered view is useful when we start thinking about agentic frameworks as “judges” for benchmarking: the judging capability can itself be an agent, sitting in the cognition layer, consuming outputs from perception agents and workload metadata rather than raw pixels alone [1].

Vision–language–driven agents are a distinct subclass. Sapkota et al. explicitly highlight vision–language models and multimodal sensing as key enabling technologies for Agentic UAVs, noting that they allow agents to parse complex scenes, follow natural-language instructions, and ground symbolic goals in visual context [1]. These agents differ from traditional planners in that they can reason over image and text jointly, which makes them natural candidates for roles like “mission explainer,” “anomaly triager,” or, in our case, “benchmark judge” for aerial analytics workloads. Instead of judging purely from numeric metrics, a vision–language agent can look at a drone scene, read a workload description, inspect candidate outputs, and form a qualitative judgment about which pipeline better captures the intended analytic semantics [1].