Cost calculation for drone video analysis:

The previous few articles have described agentic retrieval for insights from aerial drone images. Each agent uses an LLM and incurs tokens for query response. The KnowledgeAgent associated with the AI Search index with vector fields uses “gpt-4o-mini" model that involves automatic query decomposition and planning which yields higher number of subqueries than regular ‘gpt-4o’ model. Response for each subquery or agent execution incurs tokens. At this point, it would be helpful to calculate the cost complexity of an end-to-end drone video analysis.

The following table shows a breakdown of the typical cost involved in the end-to-end workflow from the user input of the drone video to the response on the chat interface associated with the video. It is assumed that the video, frames and associated artifacts are cleaned up at the end of the user session and that storage does not represent a significant factor in the cost calculations. The rest of the break-up pertains to the processing->analytics->feedback control loop and they are already optimized to handle the minimum workload needed to move on to the next stage.

Activity	Cost projection (USD)
Video Indexing ( first pass )	$0.09 per minute for typically 8 minute duration = $0.72  audio excluded at $0.024 per minute   Up to 40 hours of free indexing for trial accounts
Video Indexing (second pass) reduces to about a minute duration	Twice the cost as above
Extracting frames from the indexed video for a minimal set of images to populate Drone World	Base Cost for Azure FunctionApp is typically $135.27 per month on a P1V3 tier but Elastic tier. Even assuming $0.40 per million executions and free grant of 250,000 executions, the cost per end-to-end run is ~$0.92
Vectorizing and analyzing each image	Assuming dense captions, 3 transactions per image for analysis, a base rate of $1.50 per 1000 transactions and at least 30 images per video to generate embeddings for, this comes to about $0.10
Uploading and saving vectors in azure ai search index	A 1536-dimension vector is ~6KB per image resulting in 6MB vector data for 1000 images and an additional 2 MB for json insights. A single resource can host upto 200 indexes and assuming one index per user, the cost is about $75 per month. The cost of running semantic ranker is about $1 per 1000 queries. So the net cost for say 30 images without any vectorization of individual objects within the image is about $0.37
Agentic retrieval with knowledgeagent and connected agents for azure ai search and function calling for a search spanning 30 image vectors and associated analysis	With the use of azure ai search and function apps already incurred as above, the cost here is entirely from models and deployments alone. Both the gpt-4o-mini and the text-embedding-ada-002 deployments and their respective usage by each agent to say about 6 runs per query correspond to 128K tokens with a rate of $0.10 per 1M token coming to about a net of $0.07 per user query.
Preserving chat history and its usage in a tracking agent for the user based on the user session	This is not optional as tracking conversations is considered helpful to any interactive analysis and can be considered to be at most double the cost above.

In short, the end-to-end consumption causes a usage-based cost of about $ 0.98 per video.

#codingexercise: CodingExercise-07-15-2025.docx

 This is another toolset to use with an AI agent for agentic retrieval on aerial drone images using Azure AI search: 

#!/usr/bin/python
# azure-ai-agents==1.0.0
# azure-ai-projects==1.0.0b11
# azure-ai-vision-imageanalysis==1.0.0
# azure-common==1.1.28
# azure-core==1.34.0
# azure-identity==1.22.0
# azure-search-documents==11.6.0b12
# azure-storage-blob==12.25.1
# azure_ai_services==0.1.0
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from azure.ai.agents import AgentsClient
from azure.core.credentials import AzureKeyCredential
from azure.ai.projects import AIProjectClient
from typing import Any, Callable, Set, Dict, List, Optional
import os, time, sys
from azure.ai.agents import AgentsClient
from azure.ai.agents.models import (
    FunctionTool,
    ListSortOrder,
    RequiredFunctionToolCall,
    SubmitToolOutputsAction,
    ToolOutput,
)
from user_functions import fetch_weather, user_functions
sys.path.insert(0, os.path.abspath("."))
load_dotenv(override=True)
project_endpoint = os.environ["AZURE_PROJECT_ENDPOINT"]
project_api_key = os.environ["AZURE_PROJECT_API_KEY"]
agent_model = os.getenv("AZURE_AGENT_MODEL", "gpt-4o-mini")
agent_name = os.getenv("AZURE_FN_AGENT_NAME", "fn-agent-in-a-team")
api_version = "2025-05-01-Preview"
agent_max_output_tokens=10000
object_uri = os.getenv("AZURE_RED_CAR_2_SAS_URL").strip('"')
scene_uri = os.getenv("AZURE_QUERY_SAS_URI").strip('"')
from azure.ai.projects import AIProjectClient
project_client = AIProjectClient(endpoint=project_endpoint, credential=DefaultAzureCredential())
agents_client = AgentsClient(
    endpoint=project_endpoint,
    credential=DefaultAzureCredential(),
)

def agentic_retrieval(pattern_uri: Optional[str] = None, content_uri: Optional[str] = None) -> str:
    import dbscan
    if not pattern_uri:
        print(f"No pattern uri for object to be detected found.")
        pattern_uri = object_uri
    if not content_uri:
        print(f"No content uri for scene to detect objects found.")
        content_uri = scene_uri
    count = dbscan.count_multiple_matches(scene_uri, object_uri)
    return f"{count}"

image_user_functions: Set[Callable[..., Any]] = {
    agentic_retrieval
}

# Initialize function tool with user functions
functions = FunctionTool(functions=image_user_functions)
# instructions = "You are a helpful agent."
# query_text = "Hello, what is the weather in New York?"
instructions = "You are an assistant that answers the question how many objects were found in an image when both are given by their image URI. You evaluate a function to do this by passing their uri to the function and respond with the count."
query_text = f"How many objects given by its image URI {object_uri} are found in the image given by its image URI {scene_uri}?"
with agents_client:
    # Create an agent and run user's request with function calls
    # agent = agents_client.get_agent(agent_id="asst_qyMFcz1BnU0BS0QUmhxAAyFk")
    # """
    agent = agents_client.create_agent(
        model=agent_model,
        name=agent_name,
        instructions=instructions,
        tools=functions.definitions,
        tool_resources=functions.resources,
        top_p=1
    )
    # """
    print(f"Created agent, ID: {agent.id}")

    thread = agents_client.threads.create()
    print(f"Created thread, ID: {thread.id}")

    message = agents_client.messages.create(
        thread_id=thread.id,
        role="user",
        content=query_text,
    )
    print(f"Created message, ID: {message.id}")

    run = agents_client.runs.create(thread_id=thread.id, agent_id=agent.id)
    print(f"Created run, ID: {run.id}")

    while run.status in ["queued", "in_progress", "requires_action"]:
        time.sleep(1)
        run = agents_client.runs.get(thread_id=thread.id, run_id=run.id)

        if run.status == "requires_action" and isinstance(run.required_action, SubmitToolOutputsAction):
            tool_calls = run.required_action.submit_tool_outputs.tool_calls
            if not tool_calls:
                print("No tool calls provided - cancelling run")
                agents_client.runs.cancel(thread_id=thread.id, run_id=run.id)
                break

            tool_outputs = []
            for tool_call in tool_calls:
                if isinstance(tool_call, RequiredFunctionToolCall):
                    print("Is an instance of RequiredFunctionToolCall")
                    try:
                        print(f"Executing tool call: {tool_call}")
                        output = functions.execute(tool_call)
                        print(output)
                        tool_outputs.append(
                            ToolOutput(
                                tool_call_id=tool_call.id,
                                output=output,
                            )
                        )
                    except Exception as e:
                        print(f"Error executing tool_call {tool_call.id}: {e}")
                else:
                    print(f"{tool_call} skipped.")

            print(f"Tool outputs: {tool_outputs}")
            if tool_outputs:
                agents_client.runs.submit_tool_outputs(thread_id=thread.id, run_id=run.id, tool_outputs=tool_outputs)
            else:
                print(f"No tool output.")
        else:
            print(f"Waiting: {run}")

        print(f"Current run status: {run.status}")

    print(f"Run completed with status: {run.status} and details {run}")

    # Delete the agent when done
    agents_client.delete_agent(agent.id)
    print("Deleted agent")

    # Fetch and log all messages
    messages = agents_client.messages.list(thread_id=thread.id, order=ListSortOrder.ASCENDING)
    for msg in messages:
        if msg.text_messages:
            last_text = msg.text_messages[-1]
            print(f"{msg.role}: {last_text.text.value}")
           
# Output:
"""
Created agent, ID: asst_qyMFcz1BnU0BS0QUmhxAAyFk
Created thread, ID: thread_qlS6b0Lo2zxEfk5wvdbmipOg
Created message, ID: msg_Ye6koVrBF7O7RkrPGYmFokSK
Created run, ID: run_AOY65vGkbyeswPvvhURRQ7c3
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_h3UG84BhilrrMfsTRDfyNoPK', 'type': 'function', 'function': {'name': 'fetch_weather', 'arguments': '{"location":"New York"}'}}
{"weather": "Sunny, 25\u00b0C"}
Tool outputs: [{'tool_call_id': 'call_h3UG84BhilrrMfsTRDfyNoPK', 'output': '{"weather": "Sunny, 25\\u00b0C"}'}]
Current run status: RunStatus.COMPLETED
MessageRole.USER: Hello, what is the weather in New York?
MessageRole.AGENT: The weather in New York is sunny with a temperature of 25°C.\
"""

"""
Created agent, ID: asst_GUJsx765VDAhAvJ0oR0fTpyI
Created thread, ID: thread_xZ62v8T9LeIwQiHHr2IINi1G
Created message, ID: msg_UMUmOKxgNNpYn5SUuqqjw38F
Created run, ID: run_BXR4UlGDGCuA3dLtppTpgLoV
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_hfneTrfgMLgUA5Ue0zO043dI', 'type': 'function', 'function': {'name': 'agentic_retrieval', 'arguments': '{"pattern_uri": "<uri-1>", "content_uri": <uri-2>}'}}
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
Estimated object instances: 5
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
5
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_u719pByAgb6Gi2c9z87yVicY', 'type': 'function', 'function': {'name': 'agentic_retrieval', 'arguments': '{"pattern_uri": "<uri-1>", "content_uri": "<uri-2>"}'}}
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
5
Tool outputs: [{'tool_call_id': 'call_hfneTrfgMLgUA5Ue0zO043dI', 'output': '5'}, {'tool_call_id': 'call_u719pByAgb6Gi2c9z87yVicY', 'output': '5'}]
Current run status: RunStatus.COMPLETED
Run completed with status: RunStatus.COMPLETED and details {'id': 'run_BXR4UlGDGCuA3dLtppTpgLoV', 'object': 'thread.run', 'created_at': 1752476935, 'assistant_id': 'asst_GUJsx765VDAhAvJ0oR0fTpyI', 'thread_id': 'thread_xZ62v8T9LeIwQiHHr2IINi1G', 'status': 'completed', 'started_at': 1752476964, 'expires_at': None, 'cancelled_at': None, 'failed_at': None, 'completed_at': 1752476965, 'required_action': None, 'last_error': None, 'model': 'gpt-4o-mini', 'instructions': 'You are an assistant that answers the question how many objects were found in an image when both are given by their image URI. You evaluate a function to do this by passing their uri to the function and respond with the count.', 'tools': [{'type': 'function', 'function': {'name': 'agentic_retrieval', 'description': 'No description', 'parameters': {'type': 'object', 'properties': {'pattern_uri': {'type': ['string', 'null'], 'description': 'No description'}, 'content_uri': {'type': ['string', 'null'], 'description': 'No description'}}, 'required': []}, 'strict': False}}], 'tool_resources': {}, 'metadata': {}, 'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': None, 'max_prompt_tokens': None, 'truncation_strategy': {'type': 'auto', 'last_messages': None}, 'incomplete_details': None, 'usage': {'prompt_tokens': 1734, 'completion_tokens': 524, 'total_tokens': 2258, 'prompt_token_details': {'cached_tokens': 0}}, 'response_format': 'auto', 'tool_choice': 'auto', 'parallel_tool_calls': True}
MessageRole.USER: How many objects given by its image URI <uri-1> are found in the image given by its image URI <uri-2> ?
MessageRole.AGENT: The count of objects found in the images is 5 for both URIs provided.
"""

 

Reference: previous article: https://1drv.ms/w/c/d609fb70e39b65c8/EfwPxjajAJ5BslgH_78xcwoBb8tvfXF-fIRAZnVfHBIKVw?e=lhkzkY  

 

 In continuation with the previous posts, this describes how to create a file search ai agent for use with aerial drone images to query drone world:

#!/usr/bin/python

from dotenv import load_dotenv

load_dotenv(override=True)

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

from azure.ai.agents import AgentsClient

from azure.core.credentials import AzureKeyCredential

from azure.ai.projects import AIProjectClient

import os

project_endpoint = os.environ["AZURE_PROJECT_ENDPOINT"]

project_api_key = os.environ["AZURE_PROJECT_API_KEY"]

agent_model = os.getenv("AZURE_AGENT_MODEL", "gpt-4o-mini")

search_endpoint = os.environ["AZURE_SEARCH_SERVICE_ENDPOINT"]

api_version = os.getenv("AZURE_SEARCH_API_VERSION")

search_api_key = os.getenv("AZURE_SEARCH_ADMIN_KEY")

credential = AzureKeyCredential(search_api_key)

token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://search.azure.com/.default")

index_name = os.getenv("AZURE_SEARCH_INDEX_NAME", "index00")

azure_openai_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]

azure_openai_api_key = os.getenv("AZURE_OPENAI_API_KEY")

azure_openai_gpt_deployment = os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT", "gpt-4o-mini")

azure_openai_gpt_model = os.getenv("AZURE_OPENAI_GPT_MODEL", "gpt-4o-mini")

azure_openai_embedding_deployment = os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", "text-embedding-ada-002")

azure_openai_embedding_model = os.getenv("AZURE_OPENAI_EMBEDDING_MODEL", "text-embedding-ada-002")

api_version = "2025-05-01-Preview"

agent_max_output_tokens=10000

from azure.ai.agents.models import FileSearchTool

from azure.ai.projects import AIProjectClient

from azure.ai.projects.models import Index

from azure.ai.agents.models import (

    AgentThreadCreationOptions,

    ThreadMessageOptions,

    MessageTextContent,

    ListSortOrder

)

project_client = AIProjectClient(endpoint=project_endpoint, credential=DefaultAzureCredential())

for index_agent in index_client.list_agents():

    print(index_agent.name)

file_agent_name = "file-agent-in-a-team"

file_agent_instructions = "Search files and documents to find relevant information."

def create_connected_agent(name, instructions, tools):

    return project_client.agents.create_agent(

        model=azure_openai_gpt_model,

        # deployment=azure_openai_gpt_deployment,

        name=name,

        instructions=instructions,

        tools=tools.definitions,

        tool_resources=tools.resources,

        top_p=1

    )

def get_file_agent(name, instructions):

    file_search_agent = None

    for agent in project_client.agents.list_agents():

        print(f"{agent.name} matches {agent.name == 'file-agent-in-a-team'}")

        if agent.name == name:

            file_search_agent = agent

            break

    # file_search_agent = [ agent for agent in project_client.agents.list_agents() if agent.name == name][0] # get_agent(agent_id=name)

    if not file_search_agent:

        print("Creating A File Search Agent...")

        file_search_tool = FileSearchTool(

            vector_store_id=index_name, # This is your Azure AI Search index name

            vector_field="vector", # The field storing embeddings

            endpoint=search_endpoint,

            api_key=search_api_key)

        file_search_agent = create_connected_agent(

            name="file_search_agent",

            instructions="Search files and documents to find relevant information.",

            tools=[file_search_tool]

        )

    return file_search_agent

agent = get_file_agent(file_agent_name, file_agent_instructions)

def get_response(agent, instructions, query):

    messages = [

        {

            "role":"assistant",

            "content": instructions

        }

    ]

    run = project_client.agents.create_thread_and_process_run(agent_id = agent.id,

        thread = AgentThreadCreationOptions(messages = [

        ThreadMessageOptions(role="assistant", content=instructions),

        ThreadMessageOptions(role="user", content=query)]),)

    print(run)

    if run.status == "failed":

        print(f"Run error: {run.last_error}")

    # List all messages in the thread, in ascending order of creation

    messages = project_client.agents.messages.list(

        thread_id=run.thread_id,

        order=ListSortOrder.ASCENDING,

    )

    for msg in messages:

        last_part = msg.content[-1]

        if isinstance(last_part, MessageTextContent):

            print(f"{msg.role}: {last_part.text.value}")

    return last_part.text.value

if agent:

    query_text = "Are there dedicated bicycle crossings in green color at street intersections?"

    response = get_response(agent, file_agent_instructions, query_text)

    print(response)

else:

    print("No agent found.")

Output:

Sample Response:

{'id': 'run_qSjEC1Isk2NCC2DfLtUBKACn', 'object': 'thread.run', 'created_at': 1752370195, 'assistant_id': 'asst_ilwEdVRNApUDmqa2EB3sSBKp', 'thread_id': 'thread_gX5kKqSaPvtR5ISQSkTCZVdk', 'status': 'completed', 'started_at': 1752370195, 'expires_at': None, 'cancelled_at': None, 'failed_at': None, 'completed_at': 1752370200, 'required_action': None, 'last_error': None, 'model': 'gpt-4o-mini', 'instructions': 'You are an AI assistant that answers questions by searching files and documents to find relevant information. If you do not find a match for the query, respond with "I don't know", otherwise cite references as a string formated with the document id with ".json" suffix replaced with ".jpg" ', 'tools': [{'type': 'file_search'}], 'tool_resources': {}, 'metadata': {}, 'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': None, 'max_prompt_tokens': None, 'truncation_strategy': {'type': 'auto', 'last_messages': None}, 'incomplete_details': None, 'usage': {'prompt_tokens': 46448, 'completion_tokens': 66, 'total_tokens': 46514, 'prompt_token_details': {'cached_tokens': 0}}, 'response_format': 'auto', 'tool_choice': 'auto', 'parallel_tool_calls': True}

Agent List:

{'id': 'asst_ilwEdVRNApUDmqa2EB3sSBKp', 'object': 'assistant', 'created_at': 1752346264, 'name': 'file-agent-in-a-team', 'description': 'finds images given id and returns sas url', 'model': 'gpt-4o-mini', 'instructions': 'You are an AI assistant that answers questions by searching files and documents to find relevant information. If you do not find a match for the query, respond with "I don't know", otherwise cite references as a string formated with the document id with ".json" suffix replaced with ".jpg" ', 'tools': [{'type': 'file_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'file_search': {'vector_store_ids': ['vs_MNuwIdYIbJ6wVDdo2DFk01bT']}}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_f3IUTrON3hMpdyTJU51aCo7v', 'object': 'assistant', 'created_at': 1750537933, 'name': 'search-agent-in-a-team', 'description': None, 'model': 'gpt-4o-mini', 'instructions': ' \nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02. \nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer. \nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field. \n', 'tools': [{'type': 'azure_ai_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'azure_ai_search': {'indexes': [{'index_connection_id': '/subscriptions/656e67c6-f810-4ea6-8b89-636dd0b6774c/resourceGroups/rg-ctl-2/providers/Microsoft.CognitiveServices/accounts/found-vision-1/projects/droneimage/connections/srchvision01', 'index_name': 'index00', 'query_type': 'vector_semantic_hybrid', 'top_k': 5, 'filter': '', 'index_asset_id': ''}]}}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_lsH8uwS4hrg4v1lRpXm6sdtR', 'object': 'assistant', 'created_at': 1750523048, 'name': 'chat-agent-in-a-team', 'description': None, 'model': 'gpt-4o-mini', 'instructions': ' \nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02. \nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer. \nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field. \n', 'tools': [], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_JI9VWjdav3To7jjGUROejGkV', 'object': 'assistant', 'created_at': 1750291400, 'name': 'object-search-agent', 'description': None, 'model': 'gpt-4o-mini', 'instructions': '\nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02.\nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer.\nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field.\n', 'tools': [{'type': 'azure_ai_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'azure_ai_search': {'indexes': [{'index_connection_id': None, 'index_name': None, 'query_type': 'vector_semantic_hybrid', 'top_k': 5, 'filter': None, 'index_asset_id': 'index02/versions/1'}]}}, 'metadata': {}, 'response_format': 'auto'}

#codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/EYMCYvb9NRtOtcJwdXRDUi0BVzUEyGL-Rz2NKFaKj6KLgA?e=cbtmrO