#Codingexercise: Codingexercise-09-19-2026.pdf

 Query planning and reranking are becoming first class features in cloud services. Take Azure, for example, and question decomposition for RAG based pipeline now consist of three steps: decomposition into sub-queries, retrieval per sub-query, and re-ranking against the original question. Azure AI search’s retrieval activity shows a similar multi-stage process: a ModelQueryPlanning step, multiple AzureSearchQuery calls, and an AzureSearchSemanticRanker stage. Azure AI Search supports continuous indexing of documents, enabling real-time updates to the search index as new data is ingested. It can connect to various data sources, such as Azure Blob Storage, SQL databases, or Cosmos DB, to ingest documents continuously. Indexers are configured to monitor these sources for changes and update the search index accordingly. The indexer scans the data source for new, updated, or deleted documents. The time taken to index new documents depends on factors like the size of the data, complexity of the schema, and the indexing tier. For large datasets, indexing may take longer, especially if the indexer is resource starved. Once documents are indexed, they are available for querying. However, query latency can vary based on the size of the index, query complexity, and service tier. The minimum interval for indexer runs is 5 minutes. If this pull from data source is not sufficiently fast enough, individual data item can be indexed by directly pushing to index using the index client. This query planning and reranking example shows how decomposition and reranking can be layered on top of vector search to both widen and narrow the search space in a controlled way, without changing the underlying index.

This pattern of exposing two intertwined trends is now common: AI applications treat retrieval as a multi step, structured process, and they increasingly lean on storage engineering ideas to narrow the scope of search without sacrificing recall. Vector indexes also support widening the search space to ensure coverage, then narrowing it intelligently to keep latency and noise under control.

Traditional systems like Facebook Presto unify structured and unstructured data through federated queries, in memory processing, and pipelined execution, allowing petabyte scale analytics across HDFS, Cassandra, and relational stores. Presto's ability to perform federated queries allowed users to join and analyze data from diverse sources, such as Hadoop Distributed File System (HDFS), Apache Cassandra, and relational databases, in real-time. This kind of federation is essentially a way of narrowing scope at query time: the engine chooses which sources and partitions to touch based on the query predicates, rather than re indexing everything into a single monolithic store. The same idea reappears in modern vector databases, where hybrid queries combine structured metadata with vector similarity, and the system uses filters to select collections, partitions, or segments before running ANN search.

Azure AI Search as used by Azure Foundry Agents come configured with AzureAISearchTool using VECTOR_SEMANTIC_HYBRID queries, a top_k parameter, and an optional filter expression. The retrieval client issues multiple search calls—one for “number of empty parking lots” and another for “total number of parking lots”—and then a semantic ranker processes the results. “Notice the automatic query decomposition in Case 1 for both vacant and total number of parking lots.” Here, decomposition is not only about breaking the question into sub queries; it is also about narrowing the effective search space for each sub query to the subset of documents that match a specific facet of the problem. The semantic ranker then acts as a narrowing construct, scoring and selecting only the most relevant candidates to ground the agent’s answer.

Others such as Milvus, Weaviate, Redis Search/Redis VSS, Qdrant, and Azure Cosmos DB, a few common constructs emerge that support dynamic narrowing of vector search. First, hybrid indexing is pervasive: structured metadata is indexed with traditional mechanisms, while vectors are indexed with ANN structures. In Milvus, hybrid searches combine scalar filters with vector similarity, so a query can restrict by metadata (for example, tenant or object type) and then run ANN search only within the filtered subset. In Weaviate and Qdrant, metadata filters are part of the query language, and the engine uses them to select collections or segments before computing nearest neighbors. Redis VSS and Redis Search follow a similar pattern, using hash or sorted set indexes for metadata and vector indexes for embeddings, allowing high speed lookups that touch only the relevant keys. Azure Cosmos DB allows vectors to be stored directly within documents alongside schema-free data. This colocation simplifies data management and enhances the efficiency of vector-based operations. By colocating vectors with document fields and supporting hybrid queries, Cosmos DB can narrow the search to documents that satisfy metadata predicates and then apply vector similarity within that subset.

Second, partitioning and segment level pruning are used to align physical layout with common filters. In traditional databases, table partitioning by time, tenant, or region allows queries to skip entire partitions. In vector databases, collections, namespaces, and partitions play a similar role. Milvus partitions collections; Qdrant and Weaviate organize data into segments; cloud services like Azure AI Search expose index level routing and filters. In your drone image index, the schema includes an id field, a JSON description field, and a vector field. The retrieval client can specify target_index_params with reranker_threshold and include_reference_source_data, and could also add filter add ons to restrict search to images with certain tags or metadata. Using the data source as Azure AI Search resource, filters such as “tagsResult contains ‘car’ and ‘aerial’” and thereby narrow the vector search to aerial car scenes, reducing the number of candidates before semantic ranking.

Third, query planning and reranking are becoming first class features in cloud services. Your blog post on question decomposition for RAG describes a three step pipeline: decomposition into sub queries, retrieval per sub query, and reranking against the original question. Azure AI Search’s retrieval activity logs show a similar multi stage process: a ModelQueryPlanning step, multiple AzureSearchQuery calls, and an AzureSearchSemanticRanker stage. In my aerial drone vision image multimodal vector search, the system issues a query for “red cars near building with circular roof” and another for “building with circular roof structure,” then the semantic ranker evaluates the candidates and surfaces a document whose description mentions “a building with a circular roof and a circular structure with cars parked on the side of the road” and tags such as “car,” “urban design,” and “aerial.” The agent then grounds its answer on this reference, citing 015982 as the relevant id. This pipeline shows how decomposition and reranking can be layered on top of vector search to both widen and narrow the search space in a controlled way, without changing the underlying index.

From a performance standpoint, narrowing the scope of vector search via metadata and query planning tends to reduce the number of distance computations and the amount of data touched, which in turn lowers latency and resource usage. Presto’s in memory processing and pipelined execution significantly reduc[e] end-to-end latency compared to traditional systems like Hive3. The same principle applies when a vector database or cloud service can avoid scanning irrelevant partitions or segments. If a query filter reduces the candidate set from millions of vectors to thousands, and the index structure can exploit that reduction by probing only the relevant partitions, then both CPU and memory traffic decrease. In Azure Cosmos DB, DiskANN based quantization is used to build efficient vector indexes; when combined with hybrid filters, the system can achieve single digit millisecond response times for many workloads. Azure Cosmos DB offers automatic scalability and single-digit millisecond response times, ensuring high performance at any scale. This kind of performance is a function of the ANN algorithm; it also depends on how effectively the system narrows the search scope before invoking the algorithm.

The constructs needed to support these gains are largely extensions of familiar storage engineering tools. There must be a way to attach and index metadata alongside vectors, using B trees, inverted indexes, or bitmaps. There must be a physical layout that groups data according to common filters, such as collections per application, partitions per tenant, or segments per time range. There must be a query planner that can orchestrate multi step retrieval, including decomposition, hybrid search, and reranking. And there must be an execution engine that can push filters down to the earliest possible stage, whether that is partition selection, segment loading, or ANN probing. My Azure AI Search example shows how these pieces can be wired together in practice: an agent with instructions that enforce id citation, a search tool configured for vector semantic hybrid queries, a function tool that performs agentic retrieval over the last few messages, and a retrieval client that logs planning, search, and ranking activity. The sample output for 015982 demonstrates that the system can retrieve dense captions, tags, and metadata for a drone image and then use them to answer a question about red cars near a circular roof, even when the data does not explicitly mention red cars.

In conclusion, Azure Cosmos DB integrates vector search into a general purpose database, supporting hybrid queries and DiskANN indexing. Azure AI Search provides vector semantic hybrid search, semantic ranking, and agentic retrieval tools that can be composed with language models. Other clouds offer analogous capabilities: managed vector stores, hybrid search APIs, and RAG oriented services that combine embeddings, filters, and rerankers. Treating retrieval as a structured, multi step process and borrowing partitioning, federation, and query planning ideas from storage engineering, is a trend that seems to be shaping how these services evolve. Instead of re indexing all vectors whenever the query pattern changes, they expose constructs that let engineers narrow the search dynamically: metadata filters, collections and namespaces, partitioned indexes, query decomposition, and semantic reranking. When designing vector indexes and retrieval pipelines that are not only accurate, one must be aware of filters, multi hop structure, and the realities of cloud scale storage.

#Codingexercise: Codingexercise-09-19-2026.pdf

 Continued from previous two posts:

Traditional storage products already embody many of these ideas, even if they were not originally designed for vector search. Relational databases use table partitioning, index organized tables, and partial indexes to restrict scans to relevant partitions. Search engines like Elasticsearch and OpenSearch use index level routing, shards, and filtered queries to narrow the set of documents before applying scoring. When these systems added k NN or vector search capabilities, they naturally reused the existing partitioning and filtering machinery: k NN queries can be combined with term filters, range filters, and routing keys, and the engine can limit ANN search to shards that satisfy the filters. This reuse is one reason why hybrid search (metadata plus vectors) in such systems can be efficient without re indexing all vectors for each new filter.

Contemporary vector databases and cloud services extend these constructs with more built-in support for dynamic narrowing. Milvus exposes collections and partitions, supports hybrid search with scalar filters, and uses adaptive execution strategies to balance recall and latency under filtered workloads. pgvector integrates with PostgreSQL’s planner and allows filtered vector queries to benefit from relational indexes and partitioning. Commercial services such as Pinecone, Weaviate, and others expose namespaces, metadata filters, and per collection indexes, encouraging users to design schemas where common filters align with collection boundaries. Some services also support “metadata aware” routing, where vectors are placed into pods or shards based on attributes, so that queries with those attributes automatically narrow the search to a subset of pods.

These product features still suggest plausible directions for further work. One direction is to make partitioning and metadata aware routing more adaptive: instead of static partitions, the system could monitor query patterns and GLS like metrics, and reorganize partitions to maximize the overlap between common filters and vector clusters. Another direction is to integrate learned indexes or neural partitioners that map metadata and vectors jointly into buckets, so that both semantic similarity and filter constraints are captured in the same coarse index. A third direction is to expose more control to the optimizer, allowing users to specify policies such as “prefer exact scans when filter selectivity exceeds a threshold” or “limit ANN search to partitions with high GLS for this filter,” which would make the narrowing behavior more predictable in production.

Overall, narrowing the scope of vector search via metadata without re indexing the entire corpus is already a recognized problem, and both research and products are converging on a set of constructs—partitioned indexes, hybrid metadata/vector indexing, segment level pruning, and cost based optimization—that make this narrowing effective. The efficiency gains come from reducing the number of candidates and the amount of data touched, while keeping retrieval semantics intact by ensuring that all vectors satisfying the filter within the chosen partitions are considered. The interesting space lies in formalizing the interaction between filters and vector distributions, designing index structures that exploit that interaction, and building optimizers that can dynamically choose the right narrowing strategy for each query workload.

Query planning and reranking are becoming first class features in cloud services. Question decomposition for RAG based pipeline now consist of three steps: decomposition into sub-queries, retrieval per sub-query, and re-ranking against the original question. For example, Azure AI search’s retrieval activity shows a similar multi-stage process: a ModelQueryPlanning step, multiple AzureSearchQuery calls, and an AzureSearchSemanticRanker stage. Azure AI Search supports continuous indexing of documents, enabling real-time updates to the search index as new data is ingested. It can connect to various data sources, such as Azure Blob Storage, SQL databases, or Cosmos DB, to ingest documents continuously. Indexers are configured to monitor these sources for changes and update the search index accordingly. The indexer scans the data source for new, updated, or deleted documents. The time taken to index new documents depends on factors like the size of the data, complexity of the schema, and the indexing tier. For large datasets, indexing may take longer, especially if the indexer is resource starved. Once documents are indexed, they are available for querying. However, query latency can vary based on the size of the index, query complexity, and service tier. The minimum interval for indexer runs is 5 minutes. If this pull from data source is not sufficiently fast enough, individual data item can be indexed by directly pushing to index using the index client. This query planning and reranking example shows how decomposition and reranking can be layered on top of vector search to both widen and narrow the search space in a controlled way, without changing the underlying index.

 Graph based ANN structures such as HNSW can also be adapted to support narrowing. One approach is to maintain multiple graphs keyed by metadata values or ranges, effectively creating per label or per bucket graphs. Another is to embed metadata into the graph topology, for example by constraining edges so that nodes with incompatible metadata are not reachable from each other within a small number of hops. In both cases, the query engine uses the filter to choose which graph or subgraph to traverse. The FANNS taxonomy distinguishes pre filtering (restricting the candidate set before ANN search), runtime filtering (applying filters during graph traversal), and post filtering (filtering after ANN search), and evaluates their impact on performance and recall. Pre filtering and partition based designs tend to benefit most from narrowed scope, because they avoid exploring irrelevant regions of the index altogether.

From a performance perspective, the gains from narrowing the search scope arise from several layers. At the algorithmic level, ANN search cost is roughly proportional to the number of candidates examined and the number of distance computations performed. If metadata filtering can reduce the candidate set from N vectors to M≪N, and the index structure can exploit this reduction by probing only the partitions or segments that contain those M vectors, then both CPU and memory traffic decrease. At the system level, fewer partitions or segments need to be loaded into memory, which reduces cache misses and disk I/O. At the optimizer level, the engine can choose cheaper plans, such as exact scans over small filtered subsets instead of approximate scans over the full table, when selectivity is high. The GLS metric proposed in the FANNS work formalizes how strongly the filter correlates with the query vector distribution, and the experiments show that high GLS (strong correlation) allows more aggressive pruning without hurting recall, while low GLS (weak correlation) requires more cautious strategies. 

To support these gains, several constructs need to be present in the storage and query stack. There must be a way to attach metadata to each vector and to index that metadata with structures that support fast filtering—B trees for ranges, inverted indexes for terms, bitmaps for categorical attributes, or specialized partition maps. There must be a physical layout that groups vectors in a way that aligns with common filters: partitions by tenant or time, shards by routing key, IVF lists by coarse centroid, segments by collection. There must be an execution engine that can push filters down to the partition or segment selection stage, rather than applying them only after ANN search. And there must be a query optimizer that can estimate filter selectivity and choose between alternative plans: pre filtering plus ANN, ANN plus post filtering, or exact scans over filtered subsets.

 There is a growing body of work, both academic and commercial, that treats “narrowing the scope” of vector search via metadata as a first class design concern rather than an afterthought. This is sometimes referred to as, filtered approximate nearest neighbor search (FANNS), where a similarity query is combined with predicates over structured attributes, and the system attempts to avoid scanning the entire vector index while still returning neighbors consistent with the filter. One study analyzes how such filtered search behaves in FAISS, Milvus, and pgvector, and introduces a taxonomy of filtering strategies and a Global Local Selectivity (GLS) metric to capture how strongly the filter correlates with the query vector distribution. 

In practice, systems tend to expose a small set of recurring constructs that allow dynamic narrowing without re indexing the entire corpus. One construct is partitioned or clustered indexes, where the vector space is divided into coarse regions—IVF lists in FAISS, partitions in Milvus, shards or routing keys in Elasticsearch/OpenSearch, or namespaces/indexes/collections in commercial vector services. At ingestion time, each vector is assigned to one or more partitions based on metadata or a coarse quantizer. At query time, the engine uses the filter to select a subset of partitions and then runs ANN search only within those partitions. Because the partition boundaries are stable, the system does not need to rebuild the global index when the filter changes; it only chooses which partitions to probe. The FANNS study reports that partition based indexes such as IVFFlat can outperform graph based indexes like HNSW for low selectivity filtered queries, which suggests that this partitioning construct is particularly effective when the filter significantly reduces the candidate set. 

A second construct is hybrid indexing, where metadata is indexed with traditional structures (B trees, inverted indexes, bitmap indexes) and vectors are indexed with ANN structures (graphs, product quantization, IVF). The query planner first uses the metadata index to identify a candidate subset of rows or segments, and then applies vector similarity search only to those candidates. Milvus is described as using a hybrid approximate/exact execution strategy for filtered vector search, combining relational filtering with ANN search to stabilize recall under varying filter selectivity. In relational environments such as pgvector on PostgreSQL, the cost based optimizer can choose between a sequential scan with exact distance computation on a filtered subset, or an ANN index scan over the full table, depending on estimated costs. The same study notes that pgvector’s optimizer sometimes prefers approximate index scans even when exact sequential scans over a filtered subset would yield perfect recall at similar latency, which highlights how important the optimizer is in exploiting narrowed scope efficiently. 

A third construct is segment level pruning and tiered storage. Systems like Milvus, Weaviate, and some commercial services organize data into segments or collections that can be independently indexed and placed on different storage tiers. Metadata such as tenant, time range, or document type is used to route vectors into segments. At query time, filters are pushed down to select segments, and only those segments are loaded and searched. This reduces memory footprint and I/O, especially when segments can be kept cold until relevant filters appear. The underlying ANN index within each segment remains unchanged; the narrowing happens at the segment selection layer. This idea echoes long standing practices in columnar stores and time series databases, where partitioning by time or tenant allows queries to skip large portions of data without re indexing

 The history of artificial intelligence in medicine has often been told through benchmarks. Systems are presented with a clinical vignette, a collection of symptoms, laboratory findings, and imaging results, and are asked to produce a diagnosis. Over time, language models have become remarkably proficient at this form of evaluation, achieving scores that rival or exceed those of medical professionals on many structured medical reasoning tasks. Yet such benchmarks conceal an important aspect of clinical practice. Diagnosis is rarely the act of selecting an answer from a fully revealed problem. Instead, it is a process of discovering the problem itself.

Real-world diagnosis unfolds as a sequence of decisions under uncertainty. A clinician begins with incomplete information, formulates hypotheses, asks questions, orders tests, revises beliefs, and gradually narrows a differential diagnosis. Every action has consequences. Some tests are invasive, some are expensive, some consume scarce resources, and some provide little information relative to their cost. Expertise therefore consists not merely in reaching the correct conclusion but in determining the most informative next step. Clinical reasoning is fundamentally an information-gathering problem.

This view motivates a different way of thinking about both artificial intelligence and medical evaluation. Rather than judging a system solely by its final answer, the more important question becomes whether it can navigate uncertainty in the same way an expert clinician would. The challenge is not simply to know medicine but to know what information is worth acquiring, when enough evidence has been gathered, and when further investigation is unnecessary. Diagnosis becomes a dynamic decision-making process rather than a static prediction task.

An interactive framework for studying this problem begins with a patient case summarized in only a few sentences. From that starting point, a diagnostic agent must actively explore the case through questions and tests, much as a physician would. Information is not freely available. It is revealed only when explicitly requested. Each request imposes a cost, and every additional piece of evidence must justify its value. The resulting environment transforms diagnosis from a retrospective exercise into a prospective one, requiring planning, curiosity, skepticism, and resource management. The process resembles a search problem in which information itself is the primary resource.  

Such a framework shifts attention away from memorized medical facts and toward the structure of reasoning. It exposes weaknesses that conventional benchmarks often overlook. A system may rush toward an early diagnosis and become anchored on an initial hypothesis. It may order excessive testing because the costs are invisible. It may gather information indiscriminately without understanding which observations would meaningfully change the probability of a disease. By forcing an agent to choose each diagnostic step, these shortcomings become measurable.

The computational architecture that emerges from this perspective is notable because it does not rely exclusively on raw model capability. Instead, it treats diagnosis as a form of orchestrated reasoning. Rather than asking a single language model to solve a case end-to-end, the system distributes responsibility across multiple reasoning roles. One role maintains and updates diagnostic hypotheses. Another asks which test would best discriminate among competing explanations. A third challenges assumptions and searches for contradictory evidence. A fourth considers resource stewardship and cost. A fifth performs consistency checking and error detection. Together they form a virtual deliberative process whose objective is not merely correctness but disciplined reasoning.  

This structure reflects an important insight in artificial intelligence research. Many difficult reasoning tasks benefit from internal disagreement. Human cognition is susceptible to confirmation bias, anchoring, premature closure, and overconfidence. Language models exhibit analogous tendencies. Introducing specialized agents that argue from different perspectives transforms reasoning into a form of internal debate. The result is not a search for consensus from the outset but a controlled process of hypothesis generation, criticism, and revision.

What is especially interesting from a computer science perspective is that the architecture improves performance without modifying model parameters. No retraining is required. The gains arise from process rather than representation. This distinction has broad implications. Much discussion of AI capability assumes that progress depends primarily on larger models, larger datasets, and larger computational budgets. Here, however, substantial improvements emerge through improved organization of reasoning itself. The architecture functions as a kind of cognitive operating system layered above a foundation model, shaping how information is gathered and how uncertainty is managed.

The framework also introduces a richer conception of evaluation. Correctness alone is insufficient because different reasoning strategies may reach identical answers through radically different paths. One system may arrive at the correct diagnosis after a minimal set of carefully chosen questions. Another may require an extensive battery of expensive tests. Both are accurate, but the quality of reasoning differs. Evaluating diagnostic intelligence therefore requires measuring both outcomes and the resources consumed in achieving them. The resulting tradeoff resembles problems found throughout computer science, where computational efficiency matters alongside correctness.

In this setting, cost functions as a proxy for broader real-world constraints. It captures not only monetary expense but also invasiveness, patient burden, wait times, and resource utilization. A diagnostic strategy that minimizes uncertainty while maximizing information per unit cost becomes desirable. The challenge is therefore not unlike active learning, adaptive experimentation, or sequential decision theory, where each observation has a price and the goal is to acquire only the evidence necessary to make a confident decision.  

A particularly compelling aspect of the work is the treatment of missing information. In real clinical practice, many questions are asked that were never documented in a case report. Simply refusing to answer these questions would inadvertently reveal information about the structure of the dataset itself. To avoid such leakage, the framework generates plausible, case-consistent responses even when the original source material contains no corresponding observation. This design choice transforms a collection of static medical narratives into a realistic interactive world. From the perspective of benchmark construction, this represents a significant methodological contribution because it reduces opportunities for exploiting dataset artifacts.

The resulting experiments offer an intriguing picture of modern AI reasoning. Language models operating in their ordinary form achieve impressive diagnostic performance, but their behavior often reveals inefficient information gathering. Stronger models tend to order more tests because they maintain broader differentials and wish to rule out additional possibilities. Weaker models sometimes appear more efficient, but only because they fail to consider alternatives that would require further investigation. The apparent savings are therefore often illusory, resulting from incomplete exploration rather than superior strategy.

The orchestrated reasoning framework alters this dynamic. By explicitly tracking hypotheses, seeking disconfirming evidence, and reasoning about test value, it improves both accuracy and efficiency simultaneously. This outcome is important because it challenges the common assumption that performance improvements necessarily require greater expenditure of resources. Better reasoning can move the entire efficiency frontier outward. In effect, a more disciplined decision process extracts more value from the same underlying intelligence.  

Another noteworthy finding is the apparent generality of the approach. The orchestration strategy improves performance across a wide variety of underlying language models. This suggests that many of the benefits arise not from specific knowledge encoded in one model family but from structural properties of reasoning itself. Hypothesis maintenance, adversarial critique, cost-aware planning, and explicit uncertainty management appear to be broadly useful cognitive tools. The architecture functions as reusable reasoning infrastructure rather than a collection of model-specific optimizations.  

More broadly, the work invites reconsideration of how intelligence should be evaluated. Traditional comparisons often pit a single AI system against a single human expert. Yet many real-world tasks are solved not by isolated individuals but by teams. Hospitals rely on consultations, referrals, specialists, multidisciplinary reviews, and collaborative decision making. If artificial systems increasingly resemble coordinated groups of specialists rather than individual practitioners, then the notion of a one-to-one human comparison may become less meaningful. Intelligence may be better understood as an organizational property emerging from communication among specialized reasoning components.

The implications extend far beyond medicine. Any domain characterized by sequential evidence gathering, costly observations, and evolving uncertainty may benefit from similar approaches. Scientific discovery, cybersecurity, engineering diagnosis, legal investigation, intelligence analysis, and complex business decision-making all require determining what information should be acquired next rather than simply interpreting information already available. In each case, the central problem is one of adaptive inquiry.

At the same time, important limitations remain. Difficult educational cases differ from everyday practice. Rare diseases and challenging diagnostic puzzles provide valuable stress tests for reasoning systems, but they do not necessarily reflect real-world prevalence. Success on unusual cases does not automatically imply success in routine settings. Likewise, cost estimates capture only a subset of practical concerns. Human judgment incorporates ethical considerations, patient preferences, uncertainty about data quality, and contextual knowledge that cannot always be expressed through a diagnostic benchmark.

Nevertheless, the work points toward a broader shift in artificial intelligence research. For years, progress was measured primarily through static prediction tasks. Increasingly, the focus is moving toward interactive reasoning, where systems must decide what information to obtain, how to interpret it, and when to act. Intelligence is revealed not only by answers but by questions. A diagnostician who knows exactly which question to ask is demonstrating a form of expertise that cannot be captured by multiple-choice tests.

The deeper lesson is that reasoning is fundamentally sequential. Knowledge emerges through a dialogue with the environment, not from a single inference performed in isolation. Artificial systems that can manage this dialogue effectively, balancing curiosity, skepticism, efficiency, and confidence, represent a different class of capability than systems optimized solely for prediction. In that sense, the most significant contribution of this work is not a new medical benchmark or a new diagnostic architecture. It is the reframing of intelligence itself as the disciplined acquisition of information under uncertainty, a perspective that may prove increasingly important as AI systems move from answering questions to deciding which questions deserve to be asked.

#Codingexercise: Codingexercise-09-15-2026.docx 

 

Sample Application of vision model and global tiling:

import torch 

from transformers import AutoProcessor, AutoModel 

import requests 

from PIL import Image 

import io 

import numpy as np 

from sklearn.neighbors import NearestNeighbors 

# ------------------------------------------------------------ 

# 1. Load Prithvi EO 2.0 model + processor 

# ------------------------------------------------------------ 

model_name = "ibm-nasa-geospatial/Prithvi-EO-2.0-300M" 

 

processor = AutoProcessor.from_pretrained(model_name) 

model = AutoModel.from_pretrained(model_name) 

model.eval() 

 

# ------------------------------------------------------------ 

# 2. Load drone image, say from SAS URL 

# ------------------------------------------------------------ 

url = "https://sadronevideo.blob.core.windows.net/input/interesting/what-location.jpg?sp=r&st=2026-09-13T01:22:02Z&se=2026-09-13T09:37:02Z&spr=https&sv=2026-02-06&sr=b&sig=9Ab0REdBAyuLT5lsOizuRLd8ijPtqle8XtOvw%2FjjDKQ%3D" 

 

response = requests.get(url) 

image = Image.open(io.BytesIO(response.content)).convert("RGB") 

 

# ------------------------------------------------------------ 

# 3. Preprocess + embed using Prithvi EO 2.0 

# ------------------------------------------------------------ 

inputs = processor(images=image, return_tensors="pt") 

 

with torch.no_grad(): 

    outputs = model(**inputs) 

    # Prithvi returns last_hidden_state; we pool it to get a single vector 

    embedding = outputs.last_hidden_state.mean(dim=1).squeeze().cpu().numpy() 

 

print("Embedding shape:", embedding.shape) 

 

# ------------------------------------------------------------ 

# 4. Build a tiny reference corpus  

# Each entry: (embedding_vector, (lat, lon)) 

# ------------------------------------------------------------ 

 

# Example reference embeddings  

reference_embeddings = np.random.rand(5, embedding.shape[0]) reference_locations = [ 

    (37.769939, -122.387722), # San Francisco 

    (47.608494, -122.339175), # Seattle 

    (40.706347, -74.010397), # New York 

    (25.758758, -80.191192), # Miami 

    (42.371839, -71.117986), # Cambridge  

 

] 

REFERENCE_DIR = "./reference_tiles" 

 

tile_files = [ 

    ("Cambridge.jpg", (42.371839, -71.117986)), 

    ("Miami.jpg", (25.758758, -80.191192)), 

    ("NewYork.jpg", (40.706347, -74.010397)), 

    ("SanFrancisco.jpg", (37.769939, -122.387722)), 

    ("Seattle.jpg", (47.608494, -122.339175)), 

] 

 

image_paths = [os.path.join(REFERENCE_DIR, f[0]) for f in tile_files] 

gps_coords = [f[1] for f in tile_files] 

embeddings = [] 

images = [] 

 

for path in image_paths: 

    img = Image.open(path).convert("RGB") 

    images.append(img) 

 

    inputs = processor(images=img, return_tensors="pt") 

 

    with torch.no_grad(): 

        outputs = model(**inputs) 

        emb = outputs.last_hidden_state.mean(dim=1).squeeze().cpu().numpy() 

 

    embeddings.append(emb) 

 

embeddings = np.array(embeddings, dtype=np.float32) 

images = np.array(images, dtype=object) 

gps_coords = np.array(gps_coords, dtype=np.float32) 

 

np.save("reference_embeddings.npy", embeddings) 

np.save("reference_images.npy", images) 

np.save("reference_gps.npy", gps_coords) 

 

reference_embeddings = np.load("earth_tile_embeddings.npy") 

reference_locations = np.load("earth_tile_locations.npy") 

 

 

# ------------------------------------------------------------ 

# 5. Fit nearest-neighbor search 

# ------------------------------------------------------------ 

nn = NearestNeighbors(n_neighbors=1, metric="cosine") 

nn.fit(reference_embeddings) 

 

dist, idx = nn.kneighbors([embedding]) 

best_index = idx[0][0] 

best_distance = dist[0][0] 

 

estimated_location = reference_locations[best_index] 

 

print("\nEstimated GPS coordinates:", estimated_location) 

print("Cosine distance:", best_distance)

## Result:

# Nearest match GPS: [ 42.371839 -71.117986 ]

# Cosine distance: 0.82509133

#Codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/IQD0APISyhYWQoXFkmUQ6TClAdhaHsjStnq4WjlmdjaNQlQ?e=ru6Rwy