An earlier article1 described the creation and usage of a Knowledge Base for LLMs. One of the ideas emphasized behind is the end-to-end service expectations from the system and not just the provisioning of a vector database. In this regard, it is important to call out that semantic similarity and embeddings just does not cut it to capture the nuances of a query. In vector databases, each data point (document, image, or any object) is often stored along with metadata – structured information that provides additional context. For example, metadata could include attributes like timestamp, author, location, category, etc. During a vector search, filters can be applied on this metadata to narrow down the results, ensuring only relevant items are retrieved. This is particularly helpful when the dataset is large and diverse. This technique is sometimes referred to as “metadata filtering”

Some examples of where this makes a difference include:

1. Product recommendations: This case involves an e-commerce vector search where product embeddings are used to find similar items. If a customer searches for “lightweight hiking shoes,” the vector embeddings find semantically similar products. Adding a metadata filter like gender: female or brand: Columbia ensures the results align with specific requirements.

2. Content Moderation or compliance: Imagine a company using vector search to identify similar documents across various teams. By filtering metadata like department: legal or classification: confidential, only the relevant documents are retrieved. This prevents retrieving semantically similar but irrelevant documents from unrelated teams or departments.

3. Geospatial Search: A travel app uses vector embeddings to recommend destinations based on a user’s travel history and preferences. Using metadata filters for location: within 100 miles ensures the recommendations are regionally relevant.

4. Media Libraries: In a vector search for images, combining embeddings with metadata like resolution: >=1080p or author: John Doe helps surface high-quality or specific submissions.

And some examples where it doesn’t:

1. Homogeneous Datasets: If the dataset lacks meaningful metadata (e.g., all records have the same category or timestamp), filtering doesn’t add value because the metadata doesn’t differentiate between records.

2. Highly Unstructured Queries: For a generic query like “artificial intelligence” in a research database, metadata filtering might not help much if the user is looking for broad, cross-disciplinary results. Overly restrictive filters could exclude valuable documents.

3. When Metadata is Sparse or Inaccurate: If the metadata is inconsistently applied or missing in many records, relying on filters can lead to incomplete or skewed results.

Another technique that improves query responses is “contextual embeddings”.This improves retrieval accuracy, cutting failures with re-ranking. It involves both a well-known Retrieval Augmented Generation technique with semantic search using embeddings and lexical search using sparse retrievers like BM25. The entire knowledge base is split into chunks. Both the TF-IDF encodings as well as semantic embeddings are generated. Parallel searches using both lexical and semantic searches are run. The results are then combined and ranked. The most relevant chunks are located, and the response is generated with enhanced context. This enhancement over multimodal embeddings and GraphRAG2 is inspired by Anthropic and a Microsoft Community blog.

#Codingexercise

https://1drv.ms/w/c/d609fb70e39b65c8/EdJ3VDeiX2hGgAjzKHaFVoYBTCOvDz2W8EjTCUg08hyWkQ?e=BDjivM

Use of RAG in creating KB

 Gen AI created a new set of applications that require a different data architecture than traditional systems which includes structured and unstructured data. Applications like chatbot can perform satisfactorily only with information from diverse data sources. A chatbot requires an LLM model to respond with information from a knowledge base, typically a vector database. The underlying principle in a chatbot is Retrieval Augmented Generation. The LLM could be newer GPT3.5 or GPT4 to reduce hallucinations, maintain up-to-date information, and leverage domain-specific knowledge.

As with all LLMs, it is important to ensure AI safety and security1 to include a diverse set of data and to leverage the proper separation of the read-write and read-only accesses needed between the model and the judge. Use of a feedback loop to emit the gradings as telemetry and its inclusion into the feedback loop for the model when deciding on the formation shape and size, albeit optional, can ensure the parameters of remaining under the constraints imposed is always met.

Evaluating the quality of chatbot responses must take into account both the knowledge base and the model involved. LLM-as-a-judge evaluates the quality of a chatbot as an external entity. Although, it suffers from limitations such as it may not be at par with human grading, it might require several auto-evaluation samples, it may have different responsiveness to different chatbot prompts and slight variations in the prompt or problem can drastically affect its performance, it can still agree on human grading on over 80% of judgements. This is achieved by using a 1-5 grading scale, using GPT-3.5 to save costs and when there is one grading example per score and using GPT-4 when there are no examples to understand grading rules.

 The use of Large Language Model (LLM) for building a knowledge base (KB) seems to be a tribal art but in fact, it is applicable here as in the vast collection of domain specific text across many industries. A knowledge graph captures relationships between entities so bot the nodes and the edges are important to discover and there is no estimate of precision and recall to begin with. We take a specific example one application of LLM to build a KB with IText2KB. This is a zero-shot method for constructing incremental, topic-independent knowledge graphs from unstructured data using large-language models, without the need for extensive post-processing which is one of the main challenges for constructing knowledge graphs. Other challenges generally include the unstructured data type which might result in lossy processing and require advanced NLP techniques for meaningful insights, few-shot learning and cross-domain knowledge extraction. NLP techniques, in turn, face limitations, including reliance on pre-defined entities, and extensive human annotation.

This approach consists of four modules: Document distiller, Incremental Entities Extractor, Incremental Relations Extractor, and Neo4J graph integrator. The Document Distiller uses LLMs specifically GPT-4 to rewrite documents into semantic blocks, guided by a flexible schema to enhance graph construction. The Incremental Entities Extractor iteratively builds a global entity set by matching local entities from documents with previously extracted global entities. The Incremental Relations Extractor utilizes global document entities to extract both stated and implied relations, with variations based on the context provided. The approach is adaptable to various use cases, as the schema can be customized based on user preferences. The final module integrates the extracted entities and relations into a Neo4j database to visualize the knowledge graph. This forms a zero-shot technique because there are no predefined examples or ontologies.

The effectiveness of this technique which has broad applicability, can best be described by some metrics such as schema consistency scores across documents where a high score reflects high performance, information consistency metric where the higher consistency is desirable, triplet extraction precision which is more for local context-specific entites than for global entites and affects the richness of the graph, the false discovery rate which should be as low as possible for a successful entity/resolution process and estimation of cosine similarity for merging entites and relationships and to remove duplicates. This method outperforms on all these metrics. The results from experiments with documents such as CVs, scientific articles and websites have also emphasized effective data refinement and impact of document chunk size on KG construction.

Every use case listed in the scenarios targeted by the UAV swarm cloud automation maps to a set of direct and current commercial players and companies who could benefit and many across industries are catching up to the emerging AI trends and deploying applications with LLMs and RAGs. So, explaining the benefits of cloud-based pipelines to continuously analyze drone imagery to build a knowledge base of landscape would not be lost on many of them. But the ideal partner for this venture would be someone who engages deeply from the start so that field tests are not only practical but routine. The value of the software will be better articulated through the voice of the customer rather than the founder and such a partnership will likely be a win-win for both from the get-go.  This article explains not only the selection but the method of engagement with such a partner. 

Drone imagery is popular today in many defense-industry related applications by virtue of remote-operated drones. The use of UAV swarm is however better applied to surveying, remote sensing, disaster preparedness and responses such as wildfires, and those that make use of LiDAR data. Power line and windmill monitoring companies are especially suited for making use of a fleet of drones. Besides, there are over ten LiDAR companies that are public in US and many more across Europe and Asia that make use of a fleet of drones, photogrammetry and LiDAR data. Those that are using simultaneous localization and mapping (SLAM), structure-from-motion (SfM), and semantic segmentation with CNNs are possibly building their own knowledge bases, so it would not hurt to show them one that is built in the cloud in incremental, observable and near real-time.  

The right way to build this software is also iterative with stakeholder input. We leverage the agile sprint-based approach to build this software. Keeping a community edition or open source, opens engagement with the partner while drawing developer audience from source code and community platforms, including marketplaces for public clouds and journals, newsletters, podcasts, and social media platforms where we can find contacts and leads. A specific milestone could be pegged as presenting a PoC at AI Infra summit.  

Aside from technical aspect, a winning business plan could target a market that’s both well-defined and large, so this can be an advantage when fundraising or getting the attention of an investor. Polishing the business plan and addressing weakness prevents the investors from having to micromanage – an unhealthy situation. VCs also make it known through social media and other marketing avenues that they are funding startups but in casting a net, it is important to establish a shared reality of success. Consistently proving that the founding idea is going to work will have a snowball effect. While an experienced founder may bank on VC contacts, the first-time founder can dodge an obstacle course of promising leads that go nowhere and may have to rely on angel investors. Pitch-deck and follow-up calls must be rehearsed and never emailed or done offline. Controlling the narrative, reframing the questions, and answering them on our terms are in our hands. Creating a contract might be necessary but it cannot be relied upon. Developing a sales funnel during external engagements is important. From the start, an open-source GitHub for curating ideas and implementations should be made available. 

 Emerging trends:

Constructing an incremental “knowledge base” of a landscape from drone imagery merges ideas from simultaneous localization and mapping (SLAM), structure-from-motion (SfM), and semantic segmentation. Incremental SLAM and 3D reconstruction is suggested in the ORB-SLAM2 paper by Mur-Atal and Tardos in 2017 where a 3D Map is built by estimating camera poses and reconstructing scene geometry from monocular, stereo, or RGB-D inputs. Such SLAM framework can also be extended by fusing in semantic cues to enrich the resulting map with object and scene labels The idea of including semantic information for 3D reconstruction is demonstrated by SemanticFusion written by McCormick et al for ICCV 2017 where they use a Convolutional Neural Network aka CNN for semantic segmentation as their system fuses semantic labels into a surfel-based 3D map, thereby transforming a purely geometric reconstruction into a semantically rich representation of a scene. SemanticFusion helps to label parts of the scene – turning a raw point cloud or mesh into a knowledge base where objects, surfaces and even relationships can be recognized and queries. SfM, on the other hand, helps to stitch multi-view data into a consistent 3D-model where the techniques are particularly relevant for drone applications. Incremental SfM pipelines can populate information about a 3D space based on the data that arrives in the pipeline, but the drones can “walk the grid” around an area of interest to make sure sufficient data is captured to buid the 3D-model from 0 to 100% and the progress can even be tracked. Semantic layer is not added to SfM processing, but semantic segmentation or object detection can be layered on independently overly the purely geometric data. Layering-on additional modules for say, object detection, region classification, or even reasoning over scene changes helps to start with basic geometric layouts and add optinally to build comprehensive knowledge base. Algorithms that crunch these sensor data whether they are images or LiDAR data must operate in real-time and not on batch periodic analysis. They can, however, be dedicated to specific domains such as urban monitoring, agricultural surveying, or environmental monitoring for additional context-specific knowledge.

 Count the number of array slices in an array of random integers such that the difference between the min and the max values in the slice is <= k

public static int getCountOfSlicesWithMinMaxDiffGEk(int[] A, int k){

        int N = A.length;

        int[] maxQ = new int[N+1];

        int[] posmaxQ = new int[N+1];

        int[] minQ = new int[N+1];

        int[] posminQ = new int[N+1];

        int firstMax = 0, lastMax = -1;

        int firstMin = 0, lastMin = -1;

        int j = 0, result = 0;

        for (int i = 0; i < N; i++) {

            while (j < N) {

                while(lastMax >= firstMax && maxQ[lastMax] <= A[j]) {

                    lastMax -= 1;

                }

                lastMax += 1;

                maxQ[lastMax] = A[j];

                posmaxQ[lastMax] = j;

                while (lastMin >= firstMin && minQ[lastMin] >= A[j]) {

                    lastMin -= 1;

                }

                lastMin += 1;

                minQ[lastMin] = A[j];

                posminQ[lastMin] = j;

                if (maxQ[firstMax] - minQ[firstMin] <= k) {

                    j += 1;

                } else {

                    break;

                }

            }

            System.out.println("result="+result + " i=" + i + " j="+ j);

            result += (j-i);

            if (result >= Integer.MAX_VALUE) {

                result = Integer.MAX_VALUE;

            }

            if (posminQ[firstMin] == i) {

                firstMin += 1;

            }

            if (posmaxQ[firstMax] == i) {

                firstMax += 1;

            }

        }

        return result;

    }

A: 3,5,7,6,3 K=2

result=0 i=0 j=2

result=2 i=1 j=4

result=5 i=2 j=4

result=7 i=3 j=4

result=8 i=4 j=5

9

#Paper95: https://1drv.ms/w/c/d609fb70e39b65c8/EbV65i2MZFtLj1P7JUDLiCMBIyiNuEfc1O1kx47__m_bgg?e=E5lNAk

 This is a summary of a book titled “Be the Unicorn” written by William Vanderbloemen and published by Harper Collins in 2023. It talks about 12 data driven habits that separate best leaders from the rest to the point that they stand out as unicorns. These soft skills are presented with intriguing, teachable case studies and include:  

1. Move quickly to seize opportunities 

2. To get ahead, be authentic 

3. Build the agility to welcome new opportunities 

4. Become a problem solver 

5. Learn to anticipate the future 

6. Be over-prepared 

7. Develop self-awareness 

8. Stay curious 

9. Establish a network of connections 

10. Learn to be likable. 

11. Focus to maximize productivity 

12. Live a purpose-driven life. 

The author is founder of an executive search firm and his list of 12 soft skills comes out of his experience in working with others. Unicorns are distinctive, admired, and respected individuals who stand out as the spark that lights up a room. To become a unicorn, one must master these talents. 

Moving quickly to seize opportunities is crucial, as it allows one to stand out from peers and become irreplaceable. Responding to challenges and opportunities with alacrity is essential. Being authentic is crucial in the search engine era, as it helps build trusting relationships with others. 

Building agility is essential to welcome new opportunities. Ursula Burns, a chemical engineering graduate, exemplifies the value of agility. She was adaptable, agile, and willing to accept change for what it was. Burns's formula for success exemplifies the value of agility. 

When individuals stretch their mindset and horizons with these soft skills, they will become natural with their newfound abilities. 

Problem-solving is essential for success in any business. Companies will pay for and retain solutions to problems, as they are everywhere. Kevin Plank, a college football player, started Under Armour to solve athletes' sweating problem by researching synthetic fabrics. To position yourself advantageously, focus on understanding the problem and developing a profitable plan. 

Analyze the future by understanding the environment and making predictions. Marc Benioff co-founded Salesforce, an online business and sales-management system that customers can use without purchasing hardware or software. Understanding the environment is crucial for making predictions and staying ahead of competitors. 

Being over-prepared is also essential for success. John Wooden, a UCLA basketball coach, over-prepared his players to prevent blisters, leading to 10 national championships. Over-prepared leaders are better equipped for crises and face constant shocks, while under-prepared leaders are never ready for a crisis. Over-preparation builds agility and is more likely to be successful in any situation. 

To achieve success in life, one must develop self-awareness, stay curious, establish a network of connections, and learn to be likable. Self-awareness involves being honest with oneself and understanding one's strengths and weaknesses. This leads to increased creativity, effectiveness in jobs, better relationships, leadership, and promotion. To increase self-awareness, practice humility, be patient, and trust others. 

Curiousness is a key factor in success, as it drives creativity and knowledge. To increase curiosity, ask questions and listen carefully to answers. Networking with people in your field helps you learn about job openings and become a favored applicant for promotions. 

Likeability is a key characteristic of successful job applicants, as it trumps competency. To achieve likability, talk less and listen more, building empathy and showing people that you put others first. By doing so, you will be drawn to them and contribute to their success. 

Richard Branson, a successful entrepreneur, exemplifies the importance of focus and organization in maximizing productivity. Technology allows us to work anytime, anywhere, and with anyone, but it can make it difficult to focus on tasks. To maximize productivity, measure your own and your company's productivity by noting important goals achieved during specific work periods. Live a purpose-driven life, like Reshma Saujani, who founded Girls Who Code, by identifying your "why" and finding a workplace where you can address it meaningfully every day. As Canadian Prime Minister Justin Trudeau said, "Change is moving faster than ever, but will never move this slowly again." Nurturing leadership skills, such as the "The Lady and the Unicorn" tapestries, can lead to a strong, unique, and victorious life.