Tuesday, July 15, 2025

 Cost calculation for drone video analysis:

The previous few articles have described agentic retrieval for insights from aerial drone images. Each agent uses an LLM and incurs tokens for query response. The KnowledgeAgent associated with the AI Search index with vector fields uses “gpt-4o-mini" model that involves automatic query decomposition and planning which yields higher number of subqueries than regular ‘gpt-4o’ model. Response for each subquery or agent execution incurs tokens. At this point, it would be helpful to calculate the cost complexity of an end-to-end drone video analysis.

The following table shows a breakdown of the typical cost involved in the end-to-end workflow from the user input of the drone video to the response on the chat interface associated with the video. It is assumed that the video, frames and associated artifacts are cleaned up at the end of the user session and that storage does not represent a significant factor in the cost calculations. The rest of the break-up pertains to the processing->analytics->feedback control loop and they are already optimized to handle the minimum workload needed to move on to the next stage.

Activity Cost projection (USD)

Video Indexing ( first pass ) • $0.09 per minute for typically 8 minute duration = $0.72

• audio excluded at $0.024 per minute

• Up to 40 hours of free indexing for trial accounts

Video Indexing (second pass) reduces to about a minute duration Twice the cost as above

Extracting frames from the indexed video for a minimal set of images to populate Drone World Base Cost for Azure FunctionApp is typically $135.27 per month on a P1V3 tier but Elastic tier. Even assuming $0.40 per million executions and free grant of 250,000 executions, the cost per end-to-end run is ~$0.92

Vectorizing and analyzing each image Assuming dense captions, 3 transactions per image for analysis, a base rate of $1.50 per 1000 transactions and at least 30 images per video to generate embeddings for, this comes to about $0.10

Uploading and saving vectors in azure ai search index A 1536-dimension vector is ~6KB per image resulting in 6MB vector data for 1000 images and an additional 2 MB for json insights. A single resource can host upto 200 indexes and assuming one index per user, the cost is about $75 per month. The cost of running semantic ranker is about $1 per 1000 queries. So the net cost for say 30 images without any vectorization of individual objects within the image is about $0.37

Agentic retrieval with knowledgeagent and connected agents for azure ai search and function calling for a search spanning 30 image vectors and associated analysis With the use of azure ai search and function apps already incurred as above, the cost here is entirely from models and deployments alone. Both the gpt-4o-mini and the text-embedding-ada-002 deployments and their respective usage by each agent to say about 6 runs per query correspond to 128K tokens with a rate of $0.10 per 1M token coming to about a net of $0.07 per user query.

Preserving chat history and its usage in a tracking agent for the user based on the user session This is not optional as tracking conversations is considered helpful to any interactive analysis and can be considered to be at most double the cost above.

In short, the end-to-end consumption causes a usage-based cost of about $ 0.98 per video.



Monday, July 14, 2025

 This is another toolset to use with an AI agent for agentic retrieval on aerial drone images using Azure AI search: 

#!/usr/bin/python
# azure-ai-agents==1.0.0
# azure-ai-projects==1.0.0b11
# azure-ai-vision-imageanalysis==1.0.0
# azure-common==1.1.28
# azure-core==1.34.0
# azure-identity==1.22.0
# azure-search-documents==11.6.0b12
# azure-storage-blob==12.25.1
# azure_ai_services==0.1.0
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from azure.ai.agents import AgentsClient
from azure.core.credentials import AzureKeyCredential
from azure.ai.projects import AIProjectClient
from typing import Any, Callable, Set, Dict, List, Optional
import os, time, sys
from azure.ai.agents import AgentsClient
from azure.ai.agents.models import (
    FunctionTool,
    ListSortOrder,
    RequiredFunctionToolCall,
    SubmitToolOutputsAction,
    ToolOutput,
)
from user_functions import fetch_weather, user_functions
sys.path.insert(0, os.path.abspath("."))
load_dotenv(override=True)
project_endpoint = os.environ["AZURE_PROJECT_ENDPOINT"]
project_api_key = os.environ["AZURE_PROJECT_API_KEY"]
agent_model = os.getenv("AZURE_AGENT_MODEL", "gpt-4o-mini")
agent_name = os.getenv("AZURE_FN_AGENT_NAME", "fn-agent-in-a-team")
api_version = "2025-05-01-Preview"
agent_max_output_tokens=10000
object_uri = os.getenv("AZURE_RED_CAR_2_SAS_URL").strip('"')
scene_uri = os.getenv("AZURE_QUERY_SAS_URI").strip('"')
from azure.ai.projects import AIProjectClient
project_client = AIProjectClient(endpoint=project_endpoint, credential=DefaultAzureCredential())
agents_client = AgentsClient(
    endpoint=project_endpoint,
    credential=DefaultAzureCredential(),
)

def agentic_retrieval(pattern_uri: Optional[str] = None, content_uri: Optional[str] = None) -> str:
    import dbscan
    if not pattern_uri:
        print(f"No pattern uri for object to be detected found.")
        pattern_uri = object_uri
    if not content_uri:
        print(f"No content uri for scene to detect objects found.")
        content_uri = scene_uri
    count = dbscan.count_multiple_matches(scene_uri, object_uri)
    return f"{count}"

image_user_functions: Set[Callable[..., Any]] = {
    agentic_retrieval
}

# Initialize function tool with user functions
functions = FunctionTool(functions=image_user_functions)
# instructions = "You are a helpful agent."
# query_text = "Hello, what is the weather in New York?"
instructions = "You are an assistant that answers the question how many objects were found in an image when both are given by their image URI. You evaluate a function to do this by passing their uri to the function and respond with the count."
query_text = f"How many objects given by its image URI {object_uri} are found in the image given by its image URI {scene_uri}?"
with agents_client:
    # Create an agent and run user's request with function calls
    # agent = agents_client.get_agent(agent_id="asst_qyMFcz1BnU0BS0QUmhxAAyFk")
    # """
    agent = agents_client.create_agent(
        model=agent_model,
        name=agent_name,
        instructions=instructions,
        tools=functions.definitions,
        tool_resources=functions.resources,
        top_p=1
    )
    # """
    print(f"Created agent, ID: {agent.id}")

    thread = agents_client.threads.create()
    print(f"Created thread, ID: {thread.id}")

    message = agents_client.messages.create(
        thread_id=thread.id,
        role="user",
        content=query_text,
    )
    print(f"Created message, ID: {message.id}")

    run = agents_client.runs.create(thread_id=thread.id, agent_id=agent.id)
    print(f"Created run, ID: {run.id}")

    while run.status in ["queued", "in_progress", "requires_action"]:
        time.sleep(1)
        run = agents_client.runs.get(thread_id=thread.id, run_id=run.id)

        if run.status == "requires_action" and isinstance(run.required_action, SubmitToolOutputsAction):
            tool_calls = run.required_action.submit_tool_outputs.tool_calls
            if not tool_calls:
                print("No tool calls provided - cancelling run")
                agents_client.runs.cancel(thread_id=thread.id, run_id=run.id)
                break

            tool_outputs = []
            for tool_call in tool_calls:
                if isinstance(tool_call, RequiredFunctionToolCall):
                    print("Is an instance of RequiredFunctionToolCall")
                    try:
                        print(f"Executing tool call: {tool_call}")
                        output = functions.execute(tool_call)
                        print(output)
                        tool_outputs.append(
                            ToolOutput(
                                tool_call_id=tool_call.id,
                                output=output,
                            )
                        )
                    except Exception as e:
                        print(f"Error executing tool_call {tool_call.id}: {e}")
                else:
                    print(f"{tool_call} skipped.")

            print(f"Tool outputs: {tool_outputs}")
            if tool_outputs:
                agents_client.runs.submit_tool_outputs(thread_id=thread.id, run_id=run.id, tool_outputs=tool_outputs)
            else:
                print(f"No tool output.")
        else:
            print(f"Waiting: {run}")

        print(f"Current run status: {run.status}")

    print(f"Run completed with status: {run.status} and details {run}")

    # Delete the agent when done
    agents_client.delete_agent(agent.id)
    print("Deleted agent")

    # Fetch and log all messages
    messages = agents_client.messages.list(thread_id=thread.id, order=ListSortOrder.ASCENDING)
    for msg in messages:
        if msg.text_messages:
            last_text = msg.text_messages[-1]
            print(f"{msg.role}: {last_text.text.value}")
           
# Output:
"""
Created agent, ID: asst_qyMFcz1BnU0BS0QUmhxAAyFk
Created thread, ID: thread_qlS6b0Lo2zxEfk5wvdbmipOg
Created message, ID: msg_Ye6koVrBF7O7RkrPGYmFokSK
Created run, ID: run_AOY65vGkbyeswPvvhURRQ7c3
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_h3UG84BhilrrMfsTRDfyNoPK', 'type': 'function', 'function': {'name': 'fetch_weather', 'arguments': '{"location":"New York"}'}}
{"weather": "Sunny, 25\u00b0C"}
Tool outputs: [{'tool_call_id': 'call_h3UG84BhilrrMfsTRDfyNoPK', 'output': '{"weather": "Sunny, 25\\u00b0C"}'}]
Current run status: RunStatus.COMPLETED
MessageRole.USER: Hello, what is the weather in New York?
MessageRole.AGENT: The weather in New York is sunny with a temperature of 25°C.\
"""

"""
Created agent, ID: asst_GUJsx765VDAhAvJ0oR0fTpyI
Created thread, ID: thread_xZ62v8T9LeIwQiHHr2IINi1G
Created message, ID: msg_UMUmOKxgNNpYn5SUuqqjw38F
Created run, ID: run_BXR4UlGDGCuA3dLtppTpgLoV
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Current run status: RunStatus.IN_PROGRESS
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_hfneTrfgMLgUA5Ue0zO043dI', 'type': 'function', 'function': {'name': 'agentic_retrieval', 'arguments': '{"pattern_uri": "<uri-1>", "content_uri": <uri-2>}'}}
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
Estimated object instances: 5
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
5
Is an instance of RequiredFunctionToolCall
Executing tool call: {'id': 'call_u719pByAgb6Gi2c9z87yVicY', 'type': 'function', 'function': {'name': 'agentic_retrieval', 'arguments': '{"pattern_uri": "<uri-1>", "content_uri": "<uri-2>"}'}}
len of labels=24 and labels=[ 1 -1 -1  1 -1  1  0 -1 -1 -1  1 -1 -1 -1 -1 -1 -1 -1 -1 -1  0 -1  1 -1]
5
Tool outputs: [{'tool_call_id': 'call_hfneTrfgMLgUA5Ue0zO043dI', 'output': '5'}, {'tool_call_id': 'call_u719pByAgb6Gi2c9z87yVicY', 'output': '5'}]
Current run status: RunStatus.COMPLETED
Run completed with status: RunStatus.COMPLETED and details {'id': 'run_BXR4UlGDGCuA3dLtppTpgLoV', 'object': 'thread.run', 'created_at': 1752476935, 'assistant_id': 'asst_GUJsx765VDAhAvJ0oR0fTpyI', 'thread_id': 'thread_xZ62v8T9LeIwQiHHr2IINi1G', 'status': 'completed', 'started_at': 1752476964, 'expires_at': None, 'cancelled_at': None, 'failed_at': None, 'completed_at': 1752476965, 'required_action': None, 'last_error': None, 'model': 'gpt-4o-mini', 'instructions': 'You are an assistant that answers the question how many objects were found in an image when both are given by their image URI. You evaluate a function to do this by passing their uri to the function and respond with the count.', 'tools': [{'type': 'function', 'function': {'name': 'agentic_retrieval', 'description': 'No description', 'parameters': {'type': 'object', 'properties': {'pattern_uri': {'type': ['string', 'null'], 'description': 'No description'}, 'content_uri': {'type': ['string', 'null'], 'description': 'No description'}}, 'required': []}, 'strict': False}}], 'tool_resources': {}, 'metadata': {}, 'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': None, 'max_prompt_tokens': None, 'truncation_strategy': {'type': 'auto', 'last_messages': None}, 'incomplete_details': None, 'usage': {'prompt_tokens': 1734, 'completion_tokens': 524, 'total_tokens': 2258, 'prompt_token_details': {'cached_tokens': 0}}, 'response_format': 'auto', 'tool_choice': 'auto', 'parallel_tool_calls': True}
MessageRole.USER: How many objects given by its image URI <uri-1> are found in the image given by its image URI <uri-2> ?
MessageRole.AGENT: The count of objects found in the images is 5 for both URIs provided.
"""

 

 

Sunday, July 13, 2025

 In continuation with the previous posts, this describes how to create a file search ai agent for use with aerial drone images to query drone world:

#!/usr/bin/python

from dotenv import load_dotenv

load_dotenv(override=True)

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

from azure.ai.agents import AgentsClient

from azure.core.credentials import AzureKeyCredential

from azure.ai.projects import AIProjectClient

import os

project_endpoint = os.environ["AZURE_PROJECT_ENDPOINT"]

project_api_key = os.environ["AZURE_PROJECT_API_KEY"]

agent_model = os.getenv("AZURE_AGENT_MODEL", "gpt-4o-mini")

search_endpoint = os.environ["AZURE_SEARCH_SERVICE_ENDPOINT"]

api_version = os.getenv("AZURE_SEARCH_API_VERSION")

search_api_key = os.getenv("AZURE_SEARCH_ADMIN_KEY")

credential = AzureKeyCredential(search_api_key)

token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://search.azure.com/.default")

index_name = os.getenv("AZURE_SEARCH_INDEX_NAME", "index00")

azure_openai_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]

azure_openai_api_key = os.getenv("AZURE_OPENAI_API_KEY")

azure_openai_gpt_deployment = os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT", "gpt-4o-mini")

azure_openai_gpt_model = os.getenv("AZURE_OPENAI_GPT_MODEL", "gpt-4o-mini")

azure_openai_embedding_deployment = os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", "text-embedding-ada-002")

azure_openai_embedding_model = os.getenv("AZURE_OPENAI_EMBEDDING_MODEL", "text-embedding-ada-002")

api_version = "2025-05-01-Preview"

agent_max_output_tokens=10000

from azure.ai.agents.models import FileSearchTool

from azure.ai.projects import AIProjectClient

from azure.ai.projects.models import Index

from azure.ai.agents.models import (

    AgentThreadCreationOptions,

    ThreadMessageOptions,

    MessageTextContent,

    ListSortOrder

)

project_client = AIProjectClient(endpoint=project_endpoint, credential=DefaultAzureCredential())

for index_agent in index_client.list_agents():

    print(index_agent.name)

file_agent_name = "file-agent-in-a-team"

file_agent_instructions = "Search files and documents to find relevant information."

def create_connected_agent(name, instructions, tools):

    return project_client.agents.create_agent(

        model=azure_openai_gpt_model,

        # deployment=azure_openai_gpt_deployment,

        name=name,

        instructions=instructions,

        tools=tools.definitions,

        tool_resources=tools.resources,

        top_p=1

    )

def get_file_agent(name, instructions):

    file_search_agent = None

    for agent in project_client.agents.list_agents():

        print(f"{agent.name} matches {agent.name == 'file-agent-in-a-team'}")

        if agent.name == name:

            file_search_agent = agent

            break

    # file_search_agent = [ agent for agent in project_client.agents.list_agents() if agent.name == name][0] # get_agent(agent_id=name)

    if not file_search_agent:

        print("Creating A File Search Agent...")

        file_search_tool = FileSearchTool(

            vector_store_id=index_name, # This is your Azure AI Search index name

            vector_field="vector", # The field storing embeddings

            endpoint=search_endpoint,

            api_key=search_api_key)

        file_search_agent = create_connected_agent(

            name="file_search_agent",

            instructions="Search files and documents to find relevant information.",

            tools=[file_search_tool]

        )

    return file_search_agent

agent = get_file_agent(file_agent_name, file_agent_instructions)

def get_response(agent, instructions, query):

    messages = [

        {

            "role":"assistant",

            "content": instructions

        }

    ]

    run = project_client.agents.create_thread_and_process_run(agent_id = agent.id,

        thread = AgentThreadCreationOptions(messages = [

        ThreadMessageOptions(role="assistant", content=instructions),

        ThreadMessageOptions(role="user", content=query)]),)

    print(run)

    if run.status == "failed":

        print(f"Run error: {run.last_error}")

    # List all messages in the thread, in ascending order of creation

    messages = project_client.agents.messages.list(

        thread_id=run.thread_id,

        order=ListSortOrder.ASCENDING,

    )

    for msg in messages:

        last_part = msg.content[-1]

        if isinstance(last_part, MessageTextContent):

            print(f"{msg.role}: {last_part.text.value}")

    return last_part.text.value

if agent:

    query_text = "Are there dedicated bicycle crossings in green color at street intersections?"

    response = get_response(agent, file_agent_instructions, query_text)

    print(response)

else:

    print("No agent found.")

Output:

Sample Response:

{'id': 'run_qSjEC1Isk2NCC2DfLtUBKACn', 'object': 'thread.run', 'created_at': 1752370195, 'assistant_id': 'asst_ilwEdVRNApUDmqa2EB3sSBKp', 'thread_id': 'thread_gX5kKqSaPvtR5ISQSkTCZVdk', 'status': 'completed', 'started_at': 1752370195, 'expires_at': None, 'cancelled_at': None, 'failed_at': None, 'completed_at': 1752370200, 'required_action': None, 'last_error': None, 'model': 'gpt-4o-mini', 'instructions': 'You are an AI assistant that answers questions by searching files and documents to find relevant information. If you do not find a match for the query, respond with "I don't know", otherwise cite references as a string formated with the document id with ".json" suffix replaced with ".jpg" ', 'tools': [{'type': 'file_search'}], 'tool_resources': {}, 'metadata': {}, 'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': None, 'max_prompt_tokens': None, 'truncation_strategy': {'type': 'auto', 'last_messages': None}, 'incomplete_details': None, 'usage': {'prompt_tokens': 46448, 'completion_tokens': 66, 'total_tokens': 46514, 'prompt_token_details': {'cached_tokens': 0}}, 'response_format': 'auto', 'tool_choice': 'auto', 'parallel_tool_calls': True}

Agent List:

{'id': 'asst_ilwEdVRNApUDmqa2EB3sSBKp', 'object': 'assistant', 'created_at': 1752346264, 'name': 'file-agent-in-a-team', 'description': 'finds images given id and returns sas url', 'model': 'gpt-4o-mini', 'instructions': 'You are an AI assistant that answers questions by searching files and documents to find relevant information. If you do not find a match for the query, respond with "I don't know", otherwise cite references as a string formated with the document id with ".json" suffix replaced with ".jpg" ', 'tools': [{'type': 'file_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'file_search': {'vector_store_ids': ['vs_MNuwIdYIbJ6wVDdo2DFk01bT']}}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_f3IUTrON3hMpdyTJU51aCo7v', 'object': 'assistant', 'created_at': 1750537933, 'name': 'search-agent-in-a-team', 'description': None, 'model': 'gpt-4o-mini', 'instructions': ' \nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02. \nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer. \nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field. \n', 'tools': [{'type': 'azure_ai_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'azure_ai_search': {'indexes': [{'index_connection_id': '/subscriptions/656e67c6-f810-4ea6-8b89-636dd0b6774c/resourceGroups/rg-ctl-2/providers/Microsoft.CognitiveServices/accounts/found-vision-1/projects/droneimage/connections/srchvision01', 'index_name': 'index00', 'query_type': 'vector_semantic_hybrid', 'top_k': 5, 'filter': '', 'index_asset_id': ''}]}}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_lsH8uwS4hrg4v1lRpXm6sdtR', 'object': 'assistant', 'created_at': 1750523048, 'name': 'chat-agent-in-a-team', 'description': None, 'model': 'gpt-4o-mini', 'instructions': ' \nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02. \nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer. \nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field. \n', 'tools': [], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {}, 'metadata': {}, 'response_format': 'auto'} {'id': 'asst_JI9VWjdav3To7jjGUROejGkV', 'object': 'assistant', 'created_at': 1750291400, 'name': 'object-search-agent', 'description': None, 'model': 'gpt-4o-mini', 'instructions': '\nYou are an AI assistant that answers questions about the stored and indexed drone images and objects in search index index02.\nThe data source is an Azure AI Search resource where the schema has JSON description field, a vector field and an id field and this id field must be cited in your answer.\nIf you do not find a match for the query, respond with "I don't know", otherwise cite references with the value of the id field.\n', 'tools': [{'type': 'azure_ai_search'}], 'top_p': 1.0, 'temperature': 1.0, 'tool_resources': {'azure_ai_search': {'indexes': [{'index_connection_id': None, 'index_name': None, 'query_type': 'vector_semantic_hybrid', 'top_k': 5, 'filter': None, 'index_asset_id': 'index02/versions/1'}]}}, 'metadata': {}, 'response_format': 'auto'}

#codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/EYMCYvb9NRtOtcJwdXRDUi0BVzUEyGL-Rz2NKFaKj6KLgA?e=cbtmrO 

Saturday, July 12, 2025

 In Strategic: The Skill to Set Direction, Create Advantage, and Achieve Executive Excellence, Rich Horwath delivers a compelling guide for leaders aiming to forge clear, differentiated strategies and drive lasting success. At its core, the book emphasizes that strategy is not just a plan—it’s a deliberate set of choices about where to play and how to win, rooted in clarity and trade-offs. Horwath argues that imitation, particularly competing on price or replicating a rival’s approach, is a weak substitute for true strategic thinking. Instead, he champions distinctiveness—identifying and nurturing what sets your organization apart.

He Introduces the GOST Framework—goals, objectives, strategy, and tactics—clarifying how high-level aspirations translate into specific, actionable plans. A goal is broad, an objective is precise, strategy is conceptual, and tactics are tangible. His Rule of Touch offers a practical litmus test: if you can physically touch it, it’s a tactic, not strategy.

Horwath critiques the widespread reliance on annual resource reallocation, highlighting that continuous and agile reassessment is key to performance. Through research with major organizations, he shows that timely resource shifts fuel growth and innovation. Concentration, not diversification, is where greatness often lies. Drawing on examples like Apple, Google, and Amazon, he reveals how focusing on core strengths rather than spreading thin leads to market leadership.

The book also explores strategic pitfalls—indecisiveness, risk aversion, and failure to innovate—arguing that the worst mistake isn’t making the wrong choice, but making none at all. Emotional intelligence emerges as another pillar of leadership: self-awareness and relationship management boost communication, resilience, and overall effectiveness.

Horwath pays close attention to organizational culture, suggesting that purpose-driven declarations (mission, vision, values) should steer behaviors and decisions. Toxic influences—such as persistent negativity or blame—must be rooted out to preserve strategic integrity. Effective culture, much like strategy, doesn’t happen by accident; it must be designed, reinforced, and protected.

On planning, he urges leaders to balance short-term tactics with long-term vision. Despite widespread reliance on one-year plans, imagining your company years down the road enhances adaptability and foresight. Talent management plays a key role here: hiring based on past behavior, not just experience, ensures stronger team performance.

Finally, Horwath encourages an open-minded, question-driven approach to innovation. Strategic options shouldn’t be treated as either/or propositions; instead, elements from different paths can be fused. He champions collaborative environments while warning against unproductive meetings and the myth of multitasking, advocating for structured, focused sessions and monotasking for clarity and impact.

Through vivid examples, thoughtful frameworks, and sharp insights, Horwath guides readers to build businesses that are strategically sound, culturally vibrant, and constantly evolving. His message is clear: strategic excellence isn’t reserved for the few—it’s a skill that can be cultivated, executed, and mastered.

https://1drv.ms/w/c/d609fb70e39b65c8/Echlm-Nw-wkggNZmIwEAAAAB49-Zfs6KQbo0KRqhy5BQig?e=IMM1MD 


Friday, July 11, 2025

 In “A Founder’s Guide to GTM Strategy: Getting Your First 100 Customers,” article written by Ryan Craggs in May 2025, he offers a practical, nuanced roadmap for early-stage startups aiming to gain traction. The guide begins by identifying a common pitfall: scaling too early without validating that a real market need exists. Drawing on insights from founders like Jarod Estacio and Mercury’s Head of Community Mallory Contois, it emphasizes deep customer understanding as the cornerstone of success.

Rather than relying on superficial feedback from friends or assumptions, founders are urged to engage in rigorous, structured customer discovery. Estacio, for instance, spoke with 500–1,000 potential users before fully committing to Grid’s direction, highlighting the power of iterative validation. This process includes using lean startup principles like problem discovery interviews, smoke tests via simple landing pages, and frameworks inspired by Marc Andreessen to assess problem-solution fit, market fit, business model fit, and product-market fit.

Once validation is underway, the guide stresses the importance of founder-led go-to-market execution. Many founders rush to hire a head of sales prematurely, but Contois and GTM expert Cailen D’Sa argue that early sales conversations yield critical insights that can’t be delegated. Founders need to understand objections, refine their pitch, and deeply learn what resonates before scaling the function. When it’s time to hire, roles should be clearly scoped — whether the hire is tasked with dialing prospects or optimizing systems.

Craggs then outlines four major growth channels: sales-led, marketing-led, product-led, and partnership-led. The advice is to test each aggressively but intentionally, aligning them with the ideal customer profile (ICP). That ICP isn't just about age or job title — it’s about understanding behaviors, pain points, and decision-making contexts. As Estacio points out, founders often underestimate this work and rely too much on investor networks or startup accelerators.

For execution, founders are encouraged to use lightweight but powerful tools like Apollo for outbound engagement, Gong for call analysis, and Clearbit for data enrichment. These tools allow agile experimentation without the overhead of full enterprise systems.

On metrics, Craggs emphasizes that what you measure should evolve. In the beginning, daily active users might be the North Star, but over time, monthly retention, conversion rates, and channel-specific qualified leads become more telling. Estacio notes that maturity means shifting goals — but always remaining focused on one key metric at a time.

Ultimately, the guide argues that GTM isn’t one-size-fits-all. Founders who succeed combine grit, resilience, and clarity of purpose with disciplined iteration. The takeaway isn’t just to know your customer — it’s to deeply validate, engage hands-on, and adapt fast. As Contois puts it, successful founders remain nimble and data-driven while aligning their execution with larger market forces. For startups seeking those first 100 customers, this playbook offers not just direction, but insight rooted in lived experience.


Thursday, July 10, 2025

 This is a code sample to leverage embedding models directly for making natural language queries against image vectors from aerial drone images:

#! /usr/bin/python

import json

import sys

import os

import requests

from azure.core.credentials import AzureKeyCredential

from azure.identity import DefaultAzureCredential

from azure.search.documents import SearchClient

from azure.search.documents.indexes import SearchIndexClient

from azure.search.documents.models import VectorizedQuery

sys.path.insert(0, os.path.abspath(".."))

from visionprocessor.vectorizer import vectorize_image

search_endpoint = os.getenv("AZURE_SEARCH_SERVICE_ENDPOINT")

index_name = os.getenv("AZURE_SEARCH_INDEX_NAME")

api_version = os.getenv("AZURE_SEARCH_API_VERSION")

search_api_key = os.getenv("AZURE_SEARCH_ADMIN_KEY")

vision_api_key = os.getenv("AZURE_AI_VISION_API_KEY")

vision_api_version = os.getenv("AZURE_AI_VISION_API_VERSION")

vision_region = os.getenv("AZURE_AI_VISION_REGION")

vision_endpoint = os.getenv("AZURE_AI_VISION_ENDPOINT")

azure_openai_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]

azure_openai_api_key = os.getenv("AZURE_OPENAI_API_KEY")

azure_openai_gpt_deployment = os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT", "gpt-4o-mini")

azure_openai_gpt_model = os.getenv("AZURE_OPENAI_GPT_MODEL", "gpt-4o-mini")

azure_openai_embedding_api = os.getenv("AZURE_OPENAI_EMBEDDING_API")

azure_openai_embedding_api_key = os.getenv("AZURE_OPENAI_EMBEDDING_API_KEY")

azure_openai_embedding_deployment = os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", "text-embedding-ada-002")

azure_openai_embedding_model = os.getenv("AZURE_OPENAI_EMBEDDING_MODEL", "text-embedding-ada-002")

credential = AzureKeyCredential(search_api_key)

search_client = SearchClient(endpoint=search_endpoint, index_name=index_name, credential=credential)

OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT") # e.g. https://your-openai-resource.openai.azure.com

OPENAI_KEY = os.getenv("AZURE_OPENAI_KEY")

OPENAI_DEPLOYMENT = "text-embedding-ada-002"

SEARCH_ENDPOINT = os.getenv("AZURE_SEARCH_ENDPOINT") # e.g. https://your-search-resource.search.windows.net

SEARCH_KEY = os.getenv("AZURE_SEARCH_KEY")

SEARCH_INDEX = "drone-images-index"

VECTOR_FIELD = "imageVector"

# === STEP 1: Embed the query using text-embedding-ada-002 ===

def get_text_embedding(query):

    url = azure_openai_embedding_api

    print(azure_openai_embedding_api_key)

    headers = {

        "Content-Type": "application/json",

        "api-key": azure_openai_embedding_api_key

    }

    payload = {

        "input": query

    }

    response = requests.post(url, headers=headers, json=payload)

    response.raise_for_status()

    return response.json()["data"][0]["embedding"]

# === STEP 2: Search Azure AI Search with the query vector ===

def search_drone_images(query_text, top_k=5):

    embedding = get_text_embedding(query_text)

    print(f"len of embedding={len(embedding)}")

    client = SearchClient(endpoint=search_endpoint,

                          index_name=index_name,

                          credential=credential)

    vectorized_query = VectorizedQuery(vector=embedding, k_nearest_neighbors=top_k, fields="vector", exhaustive=True)

    results = client.search(

        search_text=None,

        vector_queries=[vectorized_query],

        top=top_k,

        semantic_configuration_name = "mysemantic",

        include_total_count=True,

    )

    print(f"\n Top {top_k} matches for: \"{query_text}\"")

    for i, result in enumerate(results):

        print(f"{i+1}. ID: {result['id']} | Score: {result['@search.score']:.4f} | IMG: {result['id']}.jpg")

# === EXAMPLE USAGE ===

if __name__ == "__main__":

    # query = "How many red cars are parked near the building with a circular roof structure?"

    query = "Do bicycles have dedicated green street crossings at intersections?"

    search_drone_images(query)

"""

Answer 1:

015614.jpg

015584.jpg

015587.jpg

015612.jpg

015581.jpg

Answer 2:

Top 5 matches for: "Do bicycles have dedicated green street crossings at intersections?"1. ID: 015614 | Score: 0.5055 | IMG: 015614.jpg

2. ID: 015584 | Score: 0.5053 | IMG: 015584.jpg

3. ID: 015595 | Score: 0.5050 | IMG: 015595.jpg

4. ID: 015602 | Score: 0.5048 | IMG: 015602.jpg

5. ID: 015612 | Score: 0.5048 | IMG: 015612.jpg

"""

And this can become its own agent in the Azure AI Foundry. The usual agents are: 1. Grounding With Bing Search, 2. File Search, 3. AI Search, 4. Function calling 5. Code Interpreter but specialized agents can automatically be invoked by tracking agent as connected agents. The operations include connections, datasets, deployments, evaluations, indexes. Inference, red teams and telemetry operations.


Wednesday, July 9, 2025

 The previous posts explained how to detect and count instances of objects in a scene with the help of Hdbscan clustering algorithm. This article explains how to delegate this logic to an agent so that it can be brought on to answer specific questions on “how many” from users.

#!/usr/bin/python

# azure-ai-agents==1.0.0

# azure-ai-projects==1.0.0b11

# azure-ai-vision-imageanalysis==1.0.0

# azure-common==1.1.28

# azure-core==1.34.0

# azure-identity==1.22.0

# azure-search-documents==11.6.0b12

# azure-storage-blob==12.25.1

# azure_ai_services==0.1.0

from dotenv import load_dotenv

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

from azure.ai.agents import AgentsClient

from azure.core.credentials import AzureKeyCredential

from azure.ai.projects import AIProjectClient

from azure.ai.agents.models import AzureAISearchTool, AzureAISearchQueryType, MessageRole, ListSortOrder

import os

load_dotenv(override=True)

project_endpoint = os.environ["AZURE_PROJECT_ENDPOINT"]

project_api_key = os.environ["AZURE_PROJECT_API_KEY"]

agent_model = os.getenv("AZURE_AGENT_MODEL", "gpt-4o-mini")

search_endpoint = os.environ["AZURE_SEARCH_SERVICE_ENDPOINT"]

api_version = os.getenv("AZURE_SEARCH_API_VERSION")

search_api_key = os.getenv("AZURE_SEARCH_ADMIN_KEY")

credential = AzureKeyCredential(search_api_key)

token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://search.azure.com/.default")

index_name = os.getenv("AZURE_SEARCH_INDEX_NAME", "index00")

azure_openai_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]

azure_openai_api_key = os.getenv("AZURE_OPENAI_API_KEY")

azure_openai_gpt_deployment = os.getenv("AZURE_OPENAI_GPT_DEPLOYMENT", "gpt-4o-mini")

azure_openai_gpt_model = os.getenv("AZURE_OPENAI_GPT_MODEL", "gpt-4o-mini")

azure_openai_embedding_deployment = os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", "text-embedding-ada-002")

azure_openai_embedding_model = os.getenv("AZURE_OPENAI_EMBEDDING_MODEL", "text-embedding-ada-002")

chat_agent_name = os.getenv("AZURE_CHAT_AGENT_NAME", "chat-agent-in-a-team")

search_agent_name = os.getenv("AZURE_SEARCH_AGENT_NAME", "sceneobject-agent-in-a-team")

search_connection_id = os.environ["AI_AZURE_AI_CONNECTION_ID"] # resource id of AI Search resource

api_version = "2025-05-01-Preview"

agent_max_output_tokens=10000

object_uri = os.getenv("AZURE_RED_CAR_2_SAS_URL").strip('"')

scene_uri = os.getenv("AZURE_QUERY_SAS_URI").strip('"')

from azure.search.documents.indexes.models import KnowledgeAgent, KnowledgeAgentAzureOpenAIModel, KnowledgeAgentTargetIndex, KnowledgeAgentRequestLimits, AzureOpenAIVectorizerParameters

from azure.search.documents.indexes import SearchIndexClient

from azure.ai.projects import AIProjectClient

project_client = AIProjectClient(endpoint=project_endpoint, credential=DefaultAzureCredential())

instructions = """

You are an AI assistant that answers questions specifically about how many objects are detected in an image when both the object and image are given as image urls.

Your response must be a count of the objects in the image or 0 if you can't find any. If you encounter errors or exceptions, you must respond with "I don't know".

"""

messages = [

    {

        "role":"system",

        "content": instructions

    }

]

search_tool = AzureAISearchTool(

    index_connection_id=search_connection_id,

    index_name=index_name,

    query_type=AzureAISearchQueryType.VECTOR_SEMANTIC_HYBRID,

    filter="", # Optional filter expression

    top_k=5 # Number of results to return

)

agent = None

for existing_agent in list(project_client.agents.list_agents()):

    if existing_agent.name == search_agent_name:

        print(existing_agent.id)

        agent = existing_agent

if agent == None:

    agent = project_client.agents.create_agent(

        model=azure_openai_gpt_model,

        # deployment=azure_openai_gpt_deployment,

        name=search_agent_name,

        instructions=instructions,

        tools=search_tool.definitions,

        tool_resources=search_tool.resources,

        top_p=1

    )

# agent = project_client.agents.get_agent("asst_lsH8uwS4hrg4v1lRpXm6sdtR")

print(f"AI agent '{search_agent_name}' created or retrieved successfully:{agent}")

from azure.ai.agents.models import FunctionTool, ToolSet, ListSortOrder

from azure.search.documents.agent import KnowledgeAgentRetrievalClient

from azure.search.documents.agent.models import KnowledgeAgentRetrievalRequest, KnowledgeAgentMessage, KnowledgeAgentMessageTextContent, KnowledgeAgentIndexParams

query_text = f"How many {object_uri} can be found in {image_uri}?"

messages.append({

    "role": "user",

    "content": query_text

    #"How many parking lots are empty when compared to all the parking lots?"

})

thread = project_client.agents.threads.create()

retrieval_results = {}

def agentic_retrieval(scene_uri, object_uri) -> str:

    import dbscan

    return count_multiple_matches(scene_uri, object_uri)

# https://learn.microsoft.com/en-us/azure/ai-services/agents/how-to/tools/function-calling

functions = FunctionTool({ agentic_retrieval })

toolset = ToolSet()

toolset.add(functions)

toolset.add(search_tool)

project_client.agents.enable_auto_function_calls(toolset)

from azure.ai.agents.models import AgentsNamedToolChoice, AgentsNamedToolChoiceType, FunctionName

message = project_client.agents.messages.create(

    thread_id=thread.id,

    role="user",

    content = query_text

    # "How many red cars can be found near a building with a roof that has a circular structure?"

    # content= "How many parking lots are empty when compared to all the parking lots?"

)

run = project_client.agents.runs.create_and_process(

    thread_id=thread.id,

    agent_id=agent.id,

    tool_choice=AgentsNamedToolChoice(type=AgentsNamedToolChoiceType.FUNCTION, function=FunctionName(name="agentic_retrieval")),

    toolset=toolset)

if run.status == "failed":

    raise RuntimeError(f"Run failed: {run.last_error}")

output = project_client.agents.messages.get_last_message_text_by_role(thread_id=thread.id, role="assistant").text.value

print("Agent response:", output.replace(".", "\n"))

import json

retrieval_result = retrieval_results.get(message.id)

if retrieval_result is None:

    raise RuntimeError(f"No retrieval results found for message {message.id}")

print("Retrieval activity")

print(json.dumps([activity.as_dict() for activity in retrieval_result.activity], indent=2))

print("Retrieval results")

print(json.dumps([reference.as_dict() for reference in retrieval_result.refere