Query planning and reranking are becoming first class features in cloud services. Take Azure, for example, and question decomposition for RAG based pipeline now consist of three steps: decomposition into sub-queries, retrieval per sub-query, and re-ranking against the original question. Azure AI search’s retrieval activity shows a similar multi-stage process: a ModelQueryPlanning step, multiple AzureSearchQuery calls, and an AzureSearchSemanticRanker stage. Azure AI Search supports continuous indexing of documents, enabling real-time updates to the search index as new data is ingested. It can connect to various data sources, such as Azure Blob Storage, SQL databases, or Cosmos DB, to ingest documents continuously. Indexers are configured to monitor these sources for changes and update the search index accordingly. The indexer scans the data source for new, updated, or deleted documents. The time taken to index new documents depends on factors like the size of the data, complexity of the schema, and the indexing tier. For large datasets, indexing may take longer, especially if the indexer is resource starved. Once documents are indexed, they are available for querying. However, query latency can vary based on the size of the index, query complexity, and service tier. The minimum interval for indexer runs is 5 minutes. If this pull from data source is not sufficiently fast enough, individual data item can be indexed by directly pushing to index using the index client. This query planning and reranking example shows how decomposition and reranking can be layered on top of vector search to both widen and narrow the search space in a controlled way, without changing the underlying index.

This pattern of exposing two intertwined trends is now common: AI applications treat retrieval as a multi step, structured process, and they increasingly lean on storage engineering ideas to narrow the scope of search without sacrificing recall. Vector indexes also support widening the search space to ensure coverage, then narrowing it intelligently to keep latency and noise under control.

Traditional systems like Facebook Presto unify structured and unstructured data through federated queries, in memory processing, and pipelined execution, allowing petabyte scale analytics across HDFS, Cassandra, and relational stores. Presto's ability to perform federated queries allowed users to join and analyze data from diverse sources, such as Hadoop Distributed File System (HDFS), Apache Cassandra, and relational databases, in real-time. This kind of federation is essentially a way of narrowing scope at query time: the engine chooses which sources and partitions to touch based on the query predicates, rather than re indexing everything into a single monolithic store. The same idea reappears in modern vector databases, where hybrid queries combine structured metadata with vector similarity, and the system uses filters to select collections, partitions, or segments before running ANN search.

Azure AI Search as used by Azure Foundry Agents come configured with AzureAISearchTool using VECTOR_SEMANTIC_HYBRID queries, a top_k parameter, and an optional filter expression. The retrieval client issues multiple search calls—one for “number of empty parking lots” and another for “total number of parking lots”—and then a semantic ranker processes the results. “Notice the automatic query decomposition in Case 1 for both vacant and total number of parking lots.” Here, decomposition is not only about breaking the question into sub queries; it is also about narrowing the effective search space for each sub query to the subset of documents that match a specific facet of the problem. The semantic ranker then acts as a narrowing construct, scoring and selecting only the most relevant candidates to ground the agent’s answer.

Others such as Milvus, Weaviate, Redis Search/Redis VSS, Qdrant, and Azure Cosmos DB, a few common constructs emerge that support dynamic narrowing of vector search. First, hybrid indexing is pervasive: structured metadata is indexed with traditional mechanisms, while vectors are indexed with ANN structures. In Milvus, hybrid searches combine scalar filters with vector similarity, so a query can restrict by metadata (for example, tenant or object type) and then run ANN search only within the filtered subset. In Weaviate and Qdrant, metadata filters are part of the query language, and the engine uses them to select collections or segments before computing nearest neighbors. Redis VSS and Redis Search follow a similar pattern, using hash or sorted set indexes for metadata and vector indexes for embeddings, allowing high speed lookups that touch only the relevant keys. Azure Cosmos DB allows vectors to be stored directly within documents alongside schema-free data. This colocation simplifies data management and enhances the efficiency of vector-based operations. By colocating vectors with document fields and supporting hybrid queries, Cosmos DB can narrow the search to documents that satisfy metadata predicates and then apply vector similarity within that subset.

Second, partitioning and segment level pruning are used to align physical layout with common filters. In traditional databases, table partitioning by time, tenant, or region allows queries to skip entire partitions. In vector databases, collections, namespaces, and partitions play a similar role. Milvus partitions collections; Qdrant and Weaviate organize data into segments; cloud services like Azure AI Search expose index level routing and filters. In your drone image index, the schema includes an id field, a JSON description field, and a vector field. The retrieval client can specify target_index_params with reranker_threshold and include_reference_source_data, and could also add filter add ons to restrict search to images with certain tags or metadata. Using the data source as Azure AI Search resource, filters such as “tagsResult contains ‘car’ and ‘aerial’” and thereby narrow the vector search to aerial car scenes, reducing the number of candidates before semantic ranking.

Third, query planning and reranking are becoming first class features in cloud services. Your blog post on question decomposition for RAG describes a three step pipeline: decomposition into sub queries, retrieval per sub query, and reranking against the original question. Azure AI Search’s retrieval activity logs show a similar multi stage process: a ModelQueryPlanning step, multiple AzureSearchQuery calls, and an AzureSearchSemanticRanker stage. In my aerial drone vision image multimodal vector search, the system issues a query for “red cars near building with circular roof” and another for “building with circular roof structure,” then the semantic ranker evaluates the candidates and surfaces a document whose description mentions “a building with a circular roof and a circular structure with cars parked on the side of the road” and tags such as “car,” “urban design,” and “aerial.” The agent then grounds its answer on this reference, citing 015982 as the relevant id. This pipeline shows how decomposition and reranking can be layered on top of vector search to both widen and narrow the search space in a controlled way, without changing the underlying index.

From a performance standpoint, narrowing the scope of vector search via metadata and query planning tends to reduce the number of distance computations and the amount of data touched, which in turn lowers latency and resource usage. Presto’s in memory processing and pipelined execution significantly reduc[e] end-to-end latency compared to traditional systems like Hive3. The same principle applies when a vector database or cloud service can avoid scanning irrelevant partitions or segments. If a query filter reduces the candidate set from millions of vectors to thousands, and the index structure can exploit that reduction by probing only the relevant partitions, then both CPU and memory traffic decrease. In Azure Cosmos DB, DiskANN based quantization is used to build efficient vector indexes; when combined with hybrid filters, the system can achieve single digit millisecond response times for many workloads. Azure Cosmos DB offers automatic scalability and single-digit millisecond response times, ensuring high performance at any scale. This kind of performance is a function of the ANN algorithm; it also depends on how effectively the system narrows the search scope before invoking the algorithm.

The constructs needed to support these gains are largely extensions of familiar storage engineering tools. There must be a way to attach and index metadata alongside vectors, using B trees, inverted indexes, or bitmaps. There must be a physical layout that groups data according to common filters, such as collections per application, partitions per tenant, or segments per time range. There must be a query planner that can orchestrate multi step retrieval, including decomposition, hybrid search, and reranking. And there must be an execution engine that can push filters down to the earliest possible stage, whether that is partition selection, segment loading, or ANN probing. My Azure AI Search example shows how these pieces can be wired together in practice: an agent with instructions that enforce id citation, a search tool configured for vector semantic hybrid queries, a function tool that performs agentic retrieval over the last few messages, and a retrieval client that logs planning, search, and ranking activity. The sample output for 015982 demonstrates that the system can retrieve dense captions, tags, and metadata for a drone image and then use them to answer a question about red cars near a circular roof, even when the data does not explicitly mention red cars.

In conclusion, Azure Cosmos DB integrates vector search into a general purpose database, supporting hybrid queries and DiskANN indexing. Azure AI Search provides vector semantic hybrid search, semantic ranking, and agentic retrieval tools that can be composed with language models. Other clouds offer analogous capabilities: managed vector stores, hybrid search APIs, and RAG oriented services that combine embeddings, filters, and rerankers. Treating retrieval as a structured, multi step process and borrowing partitioning, federation, and query planning ideas from storage engineering, is a trend that seems to be shaping how these services evolve. Instead of re indexing all vectors whenever the query pattern changes, they expose constructs that let engineers narrow the search dynamically: metadata filters, collections and namespaces, partitioned indexes, query decomposition, and semantic reranking. When designing vector indexes and retrieval pipelines that are not only accurate, one must be aware of filters, multi hop structure, and the realities of cloud scale storage.

#Codingexercise: Codingexercise-09-19-2026.pdf