These are some performance improvement considerations for drone sensing applications when querying aerial flyover images from drones as studied from the explanations of the case study in previous posts. The following table outlines some of the comparisons made in terms of precision and recall.

For example, with the query “red car”, a query response limit of 50 images, vector dimensions of 1536 to enable embedding models and a baseline precision and recall from multimodal search the table outlines the relative improvements for various features:

From the comparisons, it seems the ranking of the images plays a significant role in precision, but the variety of images recalled significantly improves with query rewrites.

The query is vectorized and the descriptions of the objects detected in the images are part of the semantic configuration, the recall is healthy to suit many drone sensing applications providing a chat like interface to retrieve images only from the drone world proves sufficient. But the real gain in improvement happens with agentic retrieval when the responses to the queries from the drone sensing applications are merged and re-ranked. Many of the images retrieved across various approaches had red cars in them and some displayed the images with the greatest number of red cars from aerial shots as the first few results even when the size of the object in the image was less than 5% of the overall aerial image size in terms of pixels.

Caching of responses so that the store does not get hit for query re-use certainly improves performance as well as cost. Re-indexing operations are not counted in the comparisons above because they were completed prior to the comparisons. Re-indexing can be avoided if we setup the vectorizer with the openai embedding models on the algorithms used with the vector search and the dimensions of the vectors during upsert agrees with that needed by the embeddings model.

Token usage increases linearly with agentic retrieval as each agent leverages an LLM for its task and toolset. Token usage can be limited to keep the costs low and by reducing the response size.

This case study clearly shows the suitability of agentic retrieval for drone sensing applications.