Paying for Answers, Not Idle GPUs: A Cost-Efficient Architecture for Bursty Vision–Language Chat
Paying for Answers, Not Idle GPUs: A Cost-Efficient Architecture for Bursty Vision–Language Chat 1
Introduction 1
What the Two Observations Establish—and What They Do Not 1
The Fundamental Mismatch: A Month of Availability Is Not a Month of Inference 2
A Better Cost Model for VLM Conversations 3
The Transcript Reveals a Second Cost Center: Repeated Context 5
Analyze the Media Once, Then Reuse the Evidence 6
Reduce Visual Tokens Without Removing the Evidence Needed for the Answer 7
Use Different Computational Paths for Different Questions 8
Cache at the Right Level—and for the Right Duration 9
Absorb Fleet Bursts with Queues Instead of Permanent Peak Capacity 10
Distinguish Interactive Batching from Deferred Batch Processing 10
Azure, AWS, and Specialist GPU Platforms Are Deployment Choices, Not Economic Guarantees 11
Beyond Public Cloud: Edge and On-Premises Inference Without Giving Up Elasticity 12
Keep Conversation State Small, Durable, and Honest 13
Make Scalability Economically Safe 14
Measure the Cost of Useful Work 14
A Practical Cost-Minimizing Architecture 15
Conclusion: Make Cost Follow Work, Not the Calendar 16
References: 16
Introduction
The most effective way to reduce the monthly cost of vision–language-model chat is to stop treating an intermittently used model as a continuously occupied service. For queries that arrive in bursts—perhaps from several client fleets, followed by hours or days of inactivity—the system should preserve its data, conversation state, and ability to accept work without necessarily preserving a running GPU. It should also avoid repeatedly asking an expensive model to interpret visual evidence that the system has already processed.
The governing principle is utilization: pay for the visual analysis that produces useful answers, not for the elapsed time between questions.
The supplied evidence in the form of Azure and QwenCloud cost analysis, and accompanying chat transcript make this distinction tangible. They do not establish a controlled price comparison between equivalent models. They do, however, expose two important dimensions of VLM economics: the cost of keeping inference capacity available across a month, and the cost of repeatedly carrying a large visual context through a conversation. Optimizing only one leaves substantial savings unrealized.
What the Two Observations Establish—and What They Do Not
The Azure screenshot shows a significant cost for the Foundry resource found-vision-1, with October 2026 selected. The plotted charges rise approximately linearly through about October 21. That nearly straight cost curve is a useful diagnostic signal. If questions were submitted only occasionally while spending increased steadily, an ongoing capacity charge is a plausible explanation. But the screenshot does not identify the model, deployment type, billing meter, replica count, or token consumption. It therefore cannot prove that the entire charge came from idle GPUs. Regular token usage or another recurring charge could also contribute.
This distinction matters because Azure supports materially different economic models under the Foundry umbrella. Standard inference is billed per token. Provisioned throughput is billed per allocated provisioned throughput unit per hour, whether or not requests arrive. Managed compute for open and community models is billed hourly per accelerator. The appropriate first step is therefore to identify the actual deployment and meter, rather than infer the billing mechanism from the product name or graph alone.
The QwenCloud screenshot represents a different kind of observation. It shows a one-million-token free quota for the displayed qwen3.8-max model, approximately 833,000 tokens consumed, 16.7% remaining, and an expiration date of November 16, 2026. It also shows incomplete payment information and a “Free quota only” setting for the visible model row.
That is evidence of promotional quota consumption, not a measured commercial inference bill. The screenshot does not show what those tokens would have cost after the promotion, whether every conversation used the displayed model, or whether the quota consumption came entirely from the attached video conversation. A free allowance is valuable for experimentation, but it is not evidence that the underlying service has zero long-run cost.
The transcript provides a third, particularly useful measurement. Seven exchanges include explicit usage counters. Their reported input sizes increase from 27,817 to 31,639 tokens, while their outputs remain between 285 and 759 tokens. The conversation concerns the same video and includes railway-track counts, bridge identification, vehicle colors, building characteristics, location estimates, and potential hazards.[1]
These observations should therefore be read as an architectural case study rather than a vendor ranking:
The Azure view raises the question of whether the application paid for capacity between bursts.
The QwenCloud view illustrates how a token-metered service can support experimentation under a quota without exposing the user to a continuously allocated GPU bill.
The transcript reveals how conversational reuse of a video can generate a substantial input-token burden even when each follow-up question is short.
A defensible comparison would require matching the model capabilities, visual preprocessing, query set, quality requirements, and billing conditions. The supplied evidence does not yet support a claim that one provider is a particular multiple cheaper than another.
The Fundamental Mismatch: A Month of Availability Is Not a Month of Inference
A fleet-facing application may need to remain reachable continuously even though its expensive inference engine is needed only intermittently. Confusing these requirements is one of the most consequential design mistakes in AI infrastructure.
The service that accepts a question can be lightweight. It authenticates the client, records the request, checks whether an answer already exists, stores references to the relevant media, and either responds immediately or places the work in a queue. None of those responsibilities inherently requires a large model to remain loaded on a GPU.
The GPU becomes necessary only when the request reaches an inference stage that cannot be satisfied by stored results, structured queries, or a less expensive computational path.
For a bursty workload, the proposed architecture should separate three things: a continuously addressable control plane, durable application state, and elastic inference capacity. Keeping the first two available does not require keeping the third fully provisioned.
This is not an argument against dedicated capacity. Dedicated inference can be economically attractive when demand is sustained, latency requirements are strict, and the workload can keep the hardware productively occupied. Microsoft explicitly positions standard deployments for variable or unpredictable traffic and provisioned deployments for workloads needing guaranteed throughput and consistent latency. The mistake is purchasing the latter merely because requests may arrive at any time.
A client’s ability to submit a question at any time is not the same as a requirement to generate the first token immediately at any time.
A Better Cost Model for VLM Conversations
The monthly cost of a VLM application is better represented as a sum of distinct components:
C_month=C_inference+C_idle capacity+C_media processing+C_storage and retrieval+C_network+C_operations
For token-priced inference, a simplified request cost is
where the prices are expressed per million tokens. Cache-write charges, long-context pricing tiers, auxiliary tools, and other provider-specific meters must be added where applicable.
For GPU hosting, the comparable expression is
C_GPU=R_GPU (H_startup+H_processing+H_idle )+C_supporting infrastructure
These equations explain why a provider with a higher unit price can still produce a lower monthly bill. A token API may charge more for an individual unit of inference than a well-utilized self-hosted GPU, yet cost less overall because it does not make the application purchase the idle intervals. Conversely, at sustained high throughput, the economics can reverse.
The objective should not be the lowest advertised GPU-hour or token price. It should be the lowest cost per accepted answer at the required quality and latency.
That last qualification is essential. An inexpensive answer that prompts corrections, additional calls, or manual investigation may be more expensive in practice than a slightly costlier answer that is correct and grounded on the first attempt.
The Transcript Reveals a Second Cost Center: Repeated Context
The attached conversation illustrates an important problem beyond hosting. A user submits a video and then asks a succession of related questions. Even when a follow-up question is only a sentence long, the recorded input remains around 28,000–32,000 tokens.[1]
The increasing counters are consistent with the conversation carrying forward a large visual context and accumulating text. They do not, by themselves, reveal whether the provider reused internal representations or applied a billing discount. An input-token count is not the same thing as an invoice. Nevertheless, the application should not assume that keeping a video in the chat interface makes subsequent questions inexpensive.
A useful way to understand the opportunity is to separate the work into two phases.
The first phase establishes reusable evidence: which scenes exist, which objects appear, when they appear, and what reliable metadata accompanies them. The second phase answers a particular question from that evidence.
Without this separation, the system may repeatedly reconstruct the same understanding of the same media. With it, “How many blue cars are present?” can often become a query over a vehicle inventory, followed by a short explanation, rather than another full-video interpretation.
The transcript makes the case especially clearly because its follow-up questions span several computational categories. Counting unique cars requires identity handling across frames. Estimating physical distances requires geometry or calibration. Precise geolocation requires external anchors or reliable metadata. Describing a roof structure may require only a few relevant crops. Treating all of these as identical “send the entire video to the largest model” operations is convenient, but economically blunt.[1]
Analyze the Media Once, Then Reuse the Evidence
For repeated questions about the same video, a durable evidence layer can be more valuable than a persistent conversation transcript.
A proposed ingestion pipeline would identify representative scenes, extract candidate objects, preserve timestamps, and associate results with immutable media identifiers. Where useful, it would add object tracks, OCR results, geographic metadata, embeddings, and relationships between entities. The key is not to produce an exhaustive description of every frame. It is to retain the evidence most likely to support subsequent questions.
For a drone-video application, that evidence might include a railway corridor, a bridge, industrial buildings, parking areas, vehicle tracks, and the intervals during which each is visible. A later question can retrieve the relevant portion of the catalog and, when necessary, the corresponding image crops or short clips.
This architecture does not require replacing a VLM with a database. It assigns each component the work it is best positioned to perform. The database retrieves established facts and relationships; geometric tools compute measurements from calibrated inputs; the VLM interprets ambiguous visual evidence and explains the result.
There is, however, an important economic limit. Preprocessing has a cost. Building a rich scene graph for a video that receives only one simple question may be wasteful. A selective strategy is preferable: create inexpensive metadata first, then enrich the catalog when repeated queries or high-value tasks justify the expense.
The break-even condition is straightforward:
where Q is the number of questions that can reuse the indexed evidence.
This formulation also argues against indexing every possible attribute in advance. If clients rarely ask about roof materials, it may be cheaper to infer that attribute on demand and retain the result than to classify every roof during ingestion.
The evidence store should distinguish observations from model-generated hypotheses. A visually inferred location should not silently acquire the same status as GPS metadata. Likewise, an approximate vehicle count should not become a permanent ground-truth label merely because it was generated during an earlier turn.
Reduce Visual Tokens Without Removing the Evidence Needed for the Answer
Visual input deserves particular attention because it can dominate a VLM request even when the text prompt is small. Alibaba Cloud’s documentation exposes controls such as frame-sampling frequency, maximum frame count, and image pixel limits. It explicitly recommends lower sampling frequency for relatively static scenes and higher frequency when fast motion requires it. Increasing image detail can increase visual-token consumption. The precise controls and token rules depend on the selected model.
This means that media preparation should be driven by the question.
A question about the dominant land use along a railway corridor may need a sparse set of representative views. A question about the color of small vehicles may need high-resolution crops of parking areas. A question about the exact moment an object enters the frame may need denser temporal sampling around the candidate boundary, but not across the entire clip.
A useful design is coarse-to-fine analysis. First, inspect inexpensive representations to locate relevant scenes. Then retrieve detail only where the task demands it. For example, the system might identify the interval containing a pedestrian bridge using sparse frames, then refine the entry and exit timestamps using a denser sample around that interval.
This differs from indiscriminate downsampling. Lowering resolution everywhere can erase small vehicles, narrow track branches, or roof features. Reducing frame rate everywhere can miss brief events. The savings must be evaluated against task-level quality, not simply the reduction in bytes or tokens.
The transcript also illustrates why geometric shortcuts require caution. It contains a chain of estimates involving travel distance, frame footprint, non-overlapping frames, and horizontal ground coverage. Those estimates depend on assumptions that the transcript does not independently verify. They should not become the basis for aggressive frame elimination without checking camera geometry and actual scene overlap.[1]
For aerial imagery, “non-overlapping frames” is not even a universal optimization target. Overlap may be valuable for object tracking, cross-frame identity, reconstruction, and temporal continuity. The appropriate goal is to remove redundant processing while preserving the evidence required for the task.
Use Different Computational Paths for Different Questions
A large general-purpose VLM should not be the default engine for every query.
A proposed fleet-facing router should first ask whether the answer already exists, whether it can be computed from structured evidence, whether a specialist model can establish the relevant facts, and only then whether a larger VLM is necessary.
For example, an already verified building coordinate should come from metadata. A count of tracked vehicles inside a parking-area polygon should come from the object catalog. A physical distance should come from calibrated geometry when available. An unfamiliar roof feature or a complex scene-level question may justify VLM interpretation.
This strategy has a foundation in the model-cascading literature. FrugalGPT studies prompt adaptation, approximation, and cascades that route different questions through different model combinations. Its reported savings arise from its own experimental workloads and should not be transplanted directly to aerial-video chat. The transferable lesson is the architecture: expensive inference should be conditional on need rather than automatic.[arxiv]
For VLM applications, the cascade should include non-language-model tools, not merely a succession of increasingly large chat models.
Escalation must also be governed by observed reliability, not by a model’s self-reported confidence alone. A useful acceptance policy might require identifiable supporting frames, consistent object tracks, or agreement with external metadata. If those conditions fail, the system can request better evidence, invoke a stronger model, or explicitly decline to make a precise claim.
This matters for cost because confidently wrong answers can create long corrective conversations. The attachment includes location estimates followed by corrected coordinates. It also includes approximate vehicle counts and color groupings rather than validated inventories. Those are reasons to evaluate grounding and error recovery, not just response fluency.[1]
Cache at the Right Level—and for the Right Duration
Caching is not a single mechanism.
An exact-answer cache avoids inference when an authorized client repeats the same question about the same unchanged evidence. A structured-evidence cache retains detections, tracks, measurements, and scene descriptions. A provider prompt cache may reuse a repeated prefix at a discounted token rate. A warm inference worker retains model weights and possibly runtime state, but continues to consume paid capacity.
These mechanisms should not be conflated. A media URL avoids uploading the same file again, but it does not guarantee that visual inference is free. Stored embeddings do not guarantee that a later VLM can answer a question without visual tokens. A prompt cache may expire long before the next burst.
Amazon Bedrock, for example, supports prompt caching only for supported models, bills cached reads at a model-specific cache-read rate, and may charge separately for cache writes. Its documentation also states that prompt caching is not supported with the batch inference API. These savings therefore cannot simply be assumed to stack.[2]
For conversations separated by days, durable application-level evidence is particularly important. Provider caches should be treated as opportunistic accelerators, not as the sole mechanism for preserving a month-long analytical context.
Cache identity must include more than the question text. A safe design should account for the media version, model and preprocessing version where relevant, geographic or temporal scope, and authorization boundary. Two clients can ask the same words about different flights, and two apparently similar scenes can contain different hazards.
The greatest reusable asset is often not the generated paragraph. It is the verified intermediate evidence from which several different answers can be assembled.
Absorb Fleet Bursts with Queues Instead of Permanent Peak Capacity
Several fleets submitting questions can create a sharp aggregate burst even when each fleet is lightly active. The answer is not necessarily a GPU allocation large enough to serve every request immediately.
A proposed architecture would accept requests through an authenticated endpoint, place work into durable queues, and scale inference workers according to backlog and service-level objectives. Interactive questions would receive priority over deferrable analysis. Client-level admission control would prevent one fleet from exhausting the shared capacity.
The service-level objective should be explicit. “Answer most active chat questions within tens of seconds” permits a different design from “always begin generation within one second.” Likewise, an overnight fleet summary can use a different execution path from an operator asking about an unfolding event.
AWS provides a concrete option for the asynchronous path. SageMaker Asynchronous Inference queues requests and supports scaling the instance count to zero. Its autoscaling documentation states that requests received while there are zero instances can wait for processing after the endpoint scales up. The service supports large payloads and long-running requests, making it relevant to media analysis that does not require immediate conversational latency.[2]
Cold starts remain a real trade-off. Model loading, container initialization, and capacity acquisition must be included in the latency budget. Zero minimum replicas are appropriate only when that delay is acceptable or another path can handle urgent work.
The resulting design can remain continuously available to clients without being continuously GPU-resident.
Distinguish Interactive Batching from Deferred Batch Processing
Batching improves economics in two different ways.
At the application level, several related questions about the same clip can be combined into one evidence-oriented analysis. The transcript already includes turns that ask multiple questions together, but subsequent turns revisit related aspects of the same scene. A deliberate first-pass inventory could reduce later repetition, provided the system does not generate an unnecessarily exhaustive report.[1]
At the serving level, a self-hosted runtime can schedule multiple requests efficiently on the GPU. The PagedAttention paper underlying vLLM addresses KV-cache fragmentation and sharing to support more efficient serving. Its reported throughput improvements concern the evaluated language-model workloads; they are not a guaranteed improvement for a particular multimodal model. Nevertheless, it establishes why memory management and request scheduling affect the cost of shared inference infrastructure.[4]
Deferred batch services are different again. They trade immediate answers for lower-priced asynchronous processing. Amazon Bedrock advertises batch inference for selected models at 50% below the corresponding on-demand inference price. Whether that is useful depends on model support, modality support, regional availability, and the application’s turnaround requirements.[2]
For fleet workloads, deferred batch processing is naturally suited to routine post-flight summaries, catalog enrichment, and retrospective reporting. It is not a substitute for every live chat request.
Azure, AWS, and Specialist GPU Platforms Are Deployment Choices, Not Economic Guarantees
Moving from Azure to AWS does not, by itself, solve idle-capacity cost. A continuously allocated GPU remains a continuously allocated GPU regardless of the provider.
Azure may already offer the appropriate economic model if a suitable VLM is available through standard token-priced inference. Provisioned throughput or dedicated managed compute becomes more compelling when demand is sustained, capacity guarantees matter, or customization requires a dedicated deployment. The supplied bill should therefore prompt a deployment-mode review before a migration decision.
On AWS, Bedrock offers a managed inference path with model-dependent pricing and features, while SageMaker supports custom hosting and asynchronous inference with scale-to-zero capabilities. Model capability must be checked explicitly: an inexpensive text model is not a substitute for a video-capable VLM, and a managed provider may not expose the exact open-weight model or visual preprocessing controls required by the application.[2]
Specialist GPU platforms provide another option. Runpod documents serverless workers that scale to zero and charge while workers run. Its documentation also makes clear that the idle timeout after the last job is billable. That is an important qualification: “serverless” does not mean the bill includes only token-generation time.
Such platforms may offer an attractive execution layer for open-weight VLMs, but the evaluation must include startup time, capacity availability, storage, model compatibility, data handling, and the operational work needed to produce a reliable multi-tenant service.
Changing providers should follow a measured comparison of equivalent execution paths. Otherwise, the application risks moving the same inefficiency to a different invoice.
Beyond Public Cloud: Edge and On-Premises Inference Without Giving Up Elasticity
AWS is itself a public-cloud alternative to Azure. A genuinely non-public-cloud strategy would place some inference on fleet gateways, operator workstations, or privately operated GPU infrastructure.
The economic attraction is straightforward: if suitable hardware already exists and has spare capacity, local processing can reduce external inference and media-transfer requirements. But the hardware is not free merely because it is owned. A realistic comparison includes depreciation or lease cost, power, maintenance, software operations, security, and capacity needed for failures and bursts.
A purely local design can also be a poor fit for several distributed fleets. A GPU at one site may be idle while another site is overloaded. Provisioning every site for its own worst-case burst can reproduce the utilization problem at a larger organizational scale.
A more flexible proposal is a hybrid system. Local infrastructure performs inexpensive filtering, frame selection, privacy-sensitive preprocessing, and frequently used compact-model inference. Shared infrastructure handles demanding queries and overflow. Durable evidence and conversation state remain independent of whichever worker executes the request.
In that design, scalability comes from the ability to route work among execution pools rather than from permanently owning peak capacity everywhere. It also allows a gradual transition: local inference can expand when measurements justify it, without making the service dependent on a single privately owned GPU.
The choice should follow workload evidence. A fleet with steady local demand and strict data-handling requirements may justify an on-premises pool. A small fleet producing occasional questions may be better served by managed token inference. A geographically distributed deployment may benefit from both.
Keep Conversation State Small, Durable, and Honest
A month-long conversation does not need to become a month-long prompt.
The application should preserve a compact, structured record of the media involved, the user’s intent, verified findings, unresolved questions, corrections, and relevant object or scene identifiers. Each new request should receive only the state and evidence necessary to answer it.
This directly addresses the pattern in the attachment, where follow-up input counters remain large while the questions themselves are short.[1]
A summary is useful, but it should not erase uncertainty. If an earlier model guessed a location, a conversation compressor should retain “unverified location hypothesis,” not convert it into a settled fact. If the user corrected a coordinate, the correction should supersede the earlier estimate while preserving provenance.
For many applications, the best prompt is therefore not the complete transcript. It is a compact task description plus retrieved evidence and a carefully maintained state record.
Output control matters too. Operational questions often need a count, a time interval, a coordinate with provenance, or a short evidence-backed explanation. Requiring a long narrative for every answer increases generation work and can obscure uncertainty. Structured outputs can support both efficient downstream handling and a concise user-facing response.
Make Scalability Economically Safe
Elastic infrastructure can scale spending as readily as it scales throughput. Cost control must therefore be part of request admission and execution.
A proposed production service should enforce per-tenant limits, maximum media duration, visual-token budgets, bounded output lengths, and explicit policies for expensive model escalation. It should also distinguish an accepted asynchronous job from a failed submission so clients do not repeatedly resubmit work that is already queued.
Retries deserve special attention. A timeout does not always mean inference stopped. The system should use durable request identifiers, idempotency where available, and recorded execution status to avoid duplicate processing. Agentic analysis should likewise have bounded tool calls and bounded escalation attempts rather than an open-ended loop.
These are architectural safeguards, not merely billing alerts. A warning after the budget has been exceeded cannot undo an expensive burst.
For shared fleets, cost attribution should follow each request from client submission through retrieval, preprocessing, inference, and retry. Otherwise, an inexpensive-looking model can conceal repeated decoding, excessive media transfer, or duplicate work elsewhere in the pipeline.
Measure the Cost of Useful Work
The strongest next experiment is not another screenshot of total spending. It is a workload replay with quality and latency measured alongside cost.
The supplied conversation is a useful starting point because it contains varied tasks: object counts, attributes, temporal localization, scene relationships, geolocation, and approximate measurements. But it is not yet a ground-truth benchmark. Several answers are explicitly approximate, and some location claims are corrected later in the document.[1]
A credible evaluation would replay representative queries against several execution strategies: direct token-priced VLM inference, retrieval-assisted inference, a smaller-model cascade, scale-to-zero custom hosting, and an always-on baseline where justified.
The replay should preserve the timing pattern. Uniformly distributing requests across a test run would conceal the very burst-and-idle behavior that drives the monthly economics. The evaluation should also include simultaneous requests from multiple clients, long inactive periods, cold starts, and repeated questions about previously analyzed media.
The central measure should be
C_"accepted answer" =(C_"inference" +C_"idle" +C_"preprocessing" +C_"retrieval" +C_"network" +C_"retries" )/N_"answers meeting quality requirements" .
Quality requirements must be task-specific. An approximate scene description and an operational hazard assessment should not use the same acceptance standard. A location answer should be evaluated against a geographic reference; a unique-car count should be evaluated against annotated identities rather than frame-level detections.
This methodology prevents a false economy in which a cheaper model appears successful because its errors are not being measured.
A Practical Cost-Minimizing Architecture
For the workload described—intermittent VLM conversations over a month, potentially initiated by several fleets—the most promising starting point is a stateless client-facing service backed by durable media storage, a reusable evidence catalog, an answer cache, and an inference router.
The router should prefer an existing authorized answer when one is valid. It should then prefer structured evidence or a specialist computation. If fresh visual interpretation is necessary, it should select the smallest visual input and least expensive model that meet the task’s quality requirements. A larger model or higher-resolution pass should be an escalation, not the default.
Interactive work can use token-priced managed inference or a deliberately sized warm pool. Deferrable work can enter an asynchronous queue and use scale-to-zero workers or supported batch services. A private or edge pool can participate where it provides measurable value, with overflow handled elsewhere.
The architecture should preserve state in storage rather than in a running GPU, preserve evidence rather than continually replay raw media, and preserve quality through task-specific verification rather than repeated ungrounded conversation.
Conclusion: Make Cost Follow Work, Not the Calendar
The supplied observations do not prove that Azure is intrinsically expensive or that QwenCloud is intrinsically free. They reveal a more important distinction: a month-long application can incur cost because capacity remains allocated across the calendar, because visual context is repeatedly processed, or because both occur together.
The first problem calls for the right deployment model: token-priced inference, scale-to-zero execution, or a measured hybrid instead of unnecessary permanent capacity. The second calls for better application design: selective visual input, durable evidence, retrieval, compact conversation state, and task-aware routing.
The most economical VLM service is therefore not necessarily the one with the smallest model, the cheapest GPU, or the largest free quota. It is the one that performs the least repeated work necessary to produce a reliable answer—and can still accept the next fleet’s request after days of inactivity without paying for a large model to wait.
References:
Transcript of VLM Chat: https://1drv.ms/w/c/d609fb70e39b65c8/IQAdwO_SHLCcS6WjQ9uxDZ-3ASaepHGd_oPX3-CINjlRYnc?e=6XTTtT
Analysis of costs between QwenCloud and Azure for VLM chat:
QwenCloud Screenshot:
Azure Screenshot:
Paged Attention: https://arxiv.org/pdf/2309.06180
