This is an article on when to go local for hosting automation and apps before eventually moving it to the cloud. In the era of cloud-first development, this still holds value. We take the specific example of building copilots locally. The alternative paradigm to local data processing is federated learning and inferences which helps with privacy preservation, improved data diversity and decentralized data ownership but works best with mature machine learning models. 

As a recap, a Copilot is an AI companion that can communicate with a user over a prompt and a response. It can be used for various services such as Azure and Security, and it respects subscription filters. Copilots help users figure out workflows, queries, code and even the links to documentation. They can even obey commands such as changing the theme to light or dark mode. Copilots are well-integrated with many connectors and types of data sources supported. They implement different Natural Language Processing models and are available in various flagship products such as Microsoft 365 and GitHub. They can help create emails, code and collaboration artifacts faster and better.    

  

This article delves into the creation of a copilot to suggest IaC code relevant to a query. It follows the same precedence as a GitHub Copilot that helps developers write code in programming languages. It is powered by the OpenAI Codex model, which is a modified production version of the Generative Pre-trained Transformer-3 aka (GPT-3). The GPT-3 AI model created by OpenAI features 175 billion parameters for language processing. This is a collaboration effort between OpenAI, Microsoft and GitHub.    

  

A copilot can be developed with no code using Azure OpenAI studio. We just need to instantiate a studio, associate a model, add the data sources, and allow the model to train. The models differ in syntactic or semantic search.  The latter uses a concept called embedding that discovers the latent meaning behind the occurrences of tokens in the given data. So, it is more inclusive than the former.  A search for time will specifically search for that keyword with the GPT-3 but a search for clock will include the references to time with a model that leverages embeddings.  Either way, a search service is required to create an index over the dataset because it facilitates fast retrieval. A database such as Azure Cosmos DB can be used to assist with vector search.   

  

At present, all these resources are created in a cloud, but their functionality can also be recreated on a local Windows machine with the upcoming release of the Windows AI Studio. This helps to train the model on documents that are available only locally. Usually, the time to set up the resources is only a couple of minutes but the time to train the model on all the data is the bulk of the duration after which the model can start making responses to the queries posed by the user. The time for the model to respond once it is trained is usually in the order of a couple of seconds.  A cloud storage account has the luxury to retain documents indefinitely and with no limit to size but the training of a model on the corresponding data accrues cost and increases with the size of the data ingested to form an index.