ETA at waypoints using time-series algorithms:

Problem statement: Given the NURBS method for trajectory generation for UAV swarms as described in previous article, the UAV trajectory was independent of in-flight parameters and both the position and velocity profile of planned trajectory could be obtained using the global locations and expected time of arrival at the waypoints. While a single drone can adhere to the planned trajectory, the internal dynamics of the UAV swarm and their effect on the ETA are harder to quantify. A closer tracking of the ETA at waypoints and trajectory deviations is needed for the UAV swarm.

   

Solution:

Consider a closed loop trajectory of UAV swarm. The effects of UAV swarm dynamics are easier to observe along waypoints in the loop because the NURBS trajectory assumes a constant velocity profile. Uncertainty in external variables such as unmodeled wind-field or uncertainty from internal friction between the drone units, can lead to different arrival times. Uncertainty affecting cruise velocity can be modeled using Gaussian independent random variables with covariance but a time-series algorithm does not need any attributes other than the historical collection of ETAs at the waypoints to be able to predict the next ETA. It only looks at scalar value regardless of the type or factors playing into the arrival time of the swarm while weights can be used to normalize the irregularity of distances between waypoints on the trajectory between start to finish. The historical data is utilized to predict an estimation on the arrival time as if the arrival were a scatter plot along the timeline. Unlike other data mining algorithms that involve additional attributes of the event, this approach uses a single auto-regressive method on the continuous data to make a short-term prediction. The regression is automatically trained as the data accrues so there is no need to parameterize or quantify uncertainties. 

Central to the step of fitting the linear regression, is the notion of covariance stationarity which suggests: 

·        The mean is not dependent on t 

·        The standard deviation is not dependent on t 

·        The covariance (Yt, Yt-j) exists and is finite and does not depend on t 

·        This last factor is called jth order autocovariance 

·        The jth order autocorrelation is described as autocovariance divided by the square of standard deviation 

 

The autocovariance measures the direction of the linear dependence between Yt and Yt-j. 
while the autocorrelation measures both the direction and the strength of the linear dependence between the Yt and Yt-j. 
An autoregressive process is defined as one in which the time dependence in the process decays to zero as the random variables in the process get farther and farther apart. It has the following properties: 
E(Yt) = mean 
Var(Yt) = sigma squared 
Cov(Yt, Yt-1) = sigma squared . phi 
Cor(Yt, Yt-1) = phi 

To fit the linear regression for a restricted data set, we determine the values of the random variable from the length p transformations of the time series data set. 

For a given time-series data set, a corresponding nine data sets for length p transformations are created. The p varies from zero to eight for the nine data sets. Each of these transformed datasets is centered and standardized before modeling; that is for each variable we subtract the mean value and divide by the standard deviation. Then we divided the data set into a training set used as input to the learning method and a holdout set to evaluate the model. The holdout set contains the cases corresponding to the last five observations in the sequence. 

 Waypoint Selection

A previous article1 introduced waypoints and trajectory smoothing for UAV swarms. This section focuses on waypoint selection.

The fight path management we propose is about the example of flying a fleet of drones around skyscrapers. The sample space can be considered a grid that must be navigated from one end to another and all intermediary spaces can be thought of as waypoints to occupy on the way to the other end and allowing the fleet to organize themselves around these intermediary points. By treating sub grids within grids as potential candidates to select from, a path can be forged with a sequence of sub grids to forge to the other end and the fleet organizes itself around each sub grid. The sub grids are pre-determined, invariant and uniform in size in each epoch.

Searching for the optimum intermediary point for the flight of the drones translates to the selection of waypoints by way of centroids of the sub grids. Each viable waypoint acts as a vector for various features such as potential gain towards eventual destination, safety, signal strength and wind effects. All information about adjacencies of sub grids as viable paths is known beforehand. Treating sub grids as nodes in a graph, and using depth first traversal for topological sort, it is possible to discover paths between start to finish. The approach outlined here uses a gradient descent method to determine the local optima given the waypoints as vectors. A quadratic form representing the waypoints as vectors is assumed to denote their initial matrix.

The solution to the quadratic form representing the embeddings is found by arriving at the minima represented by Ax = b using conjugate gradient method.     

We are given input matrix A, b, a starting value x, a number of iterations i-max and an error tolerance epsilon < 1     

This method proceeds this way:      

set I to 0      

set residual to b - Ax      

set search-direction to residual.     

And delta-new to the dot-product of residual-transposed.residual.     

Initialize delta-0 to delta-new     

while I < I-max and delta > epsilon^2 delta-0 do:      

    q = dot-product(A, search-direction)     

    alpha = delta-new / (search-direction-transposed. q)      

    x = x + alpha.search-direction     

    If I is divisible by 50      

        r = b - Ax      

    else      

        r = r - alpha.q      

    delta-old = delta-new     

    delta-new = dot-product(residual-transposed,residual)     

     Beta = delta-new/delta-old     

     Search-direction = residual + Beta. Search-direction     

     I = I + 1      

The Jacobi iteration gives eigen values and eigen vectors.

 Monitoring, Telemetry and Observability are important aspects of infrastructure. The public cloud becomes the gold standard in demonstrating both active and passive monitoring. With a vast landscape of platforms, products, services, solutions, frameworks and dynamic clouds, modern IT infrastructure has enormous complexity to overcome to set up monitoring. Yet, they are seldom explained. In this article, we list five such challenges.

The first is the most obvious by nature of a diverse landscape and this is complexity. Contemporary environments for many teams and organizations are dynamic, complex, ephemeral and distributed. Tools for monitoring must keep up with these. To set up monitoring for a big picture that spans hybrid stacks and environments, one must grapple with disconnected data, alerts and reports and engage in continuously updating tagging schemas to maintain context. So, the solution to addressing complexity, unified observability and security with automated contextualization is a key solution. A comprehensive solution can indeed monitor containers, hosting frameworks like Kubernetes, and cloud resources. Topology and dependency mapping enable this flexible and streamlined observability.

The second challenge is the sprawl of tools and technologies for monitoring that are often also disconnected. Do-it-yourself and open-source solutions for monitoring were partly to blame for this. Leveraging built-in solutions from the cloud eases the overall efficiency and effort involved. This challenge has often resulted in a patchwork view, blind spots and duplicated efforts and redundant monitoring. This implies that a solution would comprise of a single, integrated full-stack platform that reduces licensing costs, increases visibility to support compliance, and empowers proactive issue remediation and robust security.

The third challenge is the sheer size of MELT (Metrics, Logs and Traces) data. With the ever-increasing volume, variety and velocity of data generated, IT Teams are tasked with finding ways to ingest, store, analyze and interpret the information often grappling with numerous and disconnected ways to do each. This results in critical issue being buried under a ton of data or overlooked due to unavailability or inadequate context which results in delayed decision making and potential for errors whose cost and impact to business are both huge and indeterministic. The right modern monitoring tool acts as a single source of truth, enriching data with context and not shying away from using AI to reason vast volumes of data. It would also have sufficient processing to emit only quality alerts and reduce triage efforts.

The fourth challenge is troubleshooting and time to resolution because teams suffering from glitches and outages do not have the luxury to root cause incidents as they must struggle to restore operations and business. As users struggle with frustrations, poor experiences, insufficient information, and the risks of not meeting Service Level Agreements, there is decreased productivity, low team morale and difficulty in retaining the most valuable employees in addition to fines that can be incurred from missed SLAs. A true monitoring solution will come with programmability features that can make triaging and resolving easier. AI can also be used to find patterns and anomalies so that there can be some proactive measures on approaching thresholds rather than being reactive after incidents.

The fifth challenge is the areas of the technological landscape that either do not participate in monitoring or do so insufficiently. In fact, data breaches and hacks that can result from incomplete monitoring have devastating financial consequences, fines and legal fees besides damaged market reputation that erodes stakeholders’ and customers’ trust. A single-entry point for comprehensive monitoring across entire infrastructure is a favored solution to meet this challenge. By visualizing the dependencies and relationships among application components and providing real-time, end-to-end observability with no manual configuration, gaps, or blind spots, a monitoring solution renders a complete picture.

Reference: Previous articles.

#Codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/Echlm-Nw-wkggNYXMwEAAAABrVDdrKy8p5xOR2KWZOh3Yw?e=hNUMeP

 This is a summary of the book titled “The Equality Machine: harnessing digital technology for a brighter, more inclusive future” written by Orly Lobel and published by Public Affairs in 2022. The author proposes “An Equality Machine” in his drive to use the common grounds of humanity to bridge two disparate and often at opposite ends of the spectrum of people impacted by technology: 1. those who fear new technologies due to their potential to exacerbate existing inequities and 2. those who envision a technological utopia without anticipating risks. The goal of this proposal is to create a better future in which humanity uses “technology for good”. It’s common knowledge that advances in technology such as artificial intelligence and chatbots are recognized both for their potential to empower as well as their drawbacks in meeting equity and fairness. Careful auditing can help algorithms from displaying the same bias as humans do. Making the data more transparent helps to value the labor involved. Feminizing agents and chatbots can normalize existing inequities. New technologies also help to discover gaps in representation and protect people from crime and disease. With their interactions to these technologies, humans are cognizant of their shift in interactions with others and with bots. Makers of chatbots and new technological inventions can explore assumptions that disrupt stereotypes.

The rise of intelligent machines has prompted a need for upholding values of equity and fairness. Technological change has been polarized, with insiders focusing on disruption and embracing new technologies, while outsiders, such as people of color, women, and those from rural areas, worry about exclusion and inequities. To improve machine fairness, humanity must strike a balance between naive optimism and fearful pessimism. Machine learning algorithms can often ascertain identity markers from other data, but this does not address the root causes of inequities. To prevent algorithmic models from reflecting human biases, organizations must be proactive about auditing the output of their AI models as well as their data inputs. Human resources can run hypothetical job candidates through their AI models to test for biases and choose more inclusive data sets. AI decision-making can offer advantages, such as easier dissecting and correcting machine bias than flawed human decision-making. Additionally, predictive algorithmic models can help companies screen a larger pool of applicants for more nuanced qualities, such as high performance and long-term retention. It would be prudent to strike a balance between machine screening and human review.

Technology can help stakeholders work towards a future of financial equity by enabling access to vast amounts of data, identifying and correcting disparities, and reducing biases. Research shows that algorithms created to reduce bias in the fintech industry were 40% less discriminatory than humans. Research also shows that companies are more likely to penalize women for initiating salary negotiations even though men might be praised for assertiveness. AI and societal shifts towards greater data transparency are empowering workers with a better understanding of their labor market value. Some governments have passed legislation banning employers from asking prospective employees to disclose their past salaries. New digital resources, such as Payscale, are bringing greater transparency to the salary negotiation process. Feminizing AI assistants and chatbots can normalize existing inequities, but companies must reflect on the preference to depict subservient robots as female. This reinforces gender as a binary construct and promotes outmoded views of women's roles in society.

Researchers are using new technologies to detect patterns in representation gaps and address systemic inequities. Natural language processing (NLP) methods are being used to analyze large amounts of information, revealing unequal power dynamics and opportunities. AI can be used to assess whether people with different identity markers are getting equitable representation in media forms. Machine learning and AI analytics can help detect gaps in representation and biases in various media industries and inspire more empowering narratives. Technology can also help protect people from harmful influences by enabling organizations to share data and develop data hubs. AI and health data can also help stakeholders accelerate drug discovery and collaborate to prevent the global spread of viruses. However, democratizing AI use in medical research contexts is crucial to ensure improved health outcomes for everyone, not just the rich.

Algorithms and embodied robots are transforming human connection and social bonds. Algorithmic biases can exacerbate existing class, racial, and social divides, while the growing prevalence of robots with sexual capacities is transforming intimacy and emotional connection. Some argue that framing robots solely as AI-empowered sex dolls is oversimplification, while others worry about the potential for violence against women.

Roboticists can challenge stereotypes by creating robots that challenge assumptions. Embodied robots can support humans in various functions, such as care labor, reception work, and space exploration. However, some critics worry about privacy risks, consent, and misuse of data.

Robots can surprise those they interact with by disrupting expectations. NASA uses feminine-looking robots like Valkyrie to support in-space exploration, while masculine-looking robots like Tank act as "roboceptionists." These robots demonstrate the choice roboticists face when designing robots that cater to existing biases or inspire imaginative new possibilities.

 This is a summary of the book titled “Future Ready: Your organization’s guide to rethinking climate, resilience and sustainability” written by Alastair MacGregor and Tom Lewis and published by Wiley in 2023. This book is about integrating sustainability and resilience into infrastructure and building projects. They introduce climate science for practitioners and decision makers who grapple with evolving environmental challenges. They recognize the target audience of the book are already aware they need to make changes. Those in the construction industry can support sustainability by adopting innovative technologies and sustainable materials. Those in transportation must prioritize decarbonization. Infrastructure designers must meet urbanization challenges. The authors recommend that they are best served by putting systems for measuring, say emissions, setting targets and implementing strategies to meet those targets. Achieving net zero will involve integrating technologies and understanding carbon markets. Leaders can generate support for these activities. Nature-based solutions provide a holistic and cost-effective approach to addressing challenges of sustainability and resilience.

Climate change has highlighted the need for infrastructure designers and urban planners to incorporate sustainability and resilience in their decision-making. The aftermath of Superstorm Sandy exposed vulnerabilities in current systems, prompting policymakers, urban planners, and industry professionals to shift their approaches to risk management and sustainability. The Future Ready framework, developed by WSP, consists of four lenses for considering decisions: climate, society, technology, and resources. The construction industry can support sustainability by adopting innovative technologies and sustainable materials. Buildings contribute nearly 50% of global greenhouse gas emissions and consume significant natural resources. Decision-makers must rethink building practices, focusing on sustainability, resilience, and long-term value. Innovative projects integrating modern technologies and sustainable materials, such as LEED and BREEAM, can provide potential paths forward.

Transportation innovations have advanced global trade and daily life, but they also contribute to 27% of US GHG emissions, exacerbating climate change and increasing infrastructure risks. To build a resilient, sustainable, and equitable future, decision-makers must prioritize decarbonization. The adoption of electric vehicles and hydrogen fuel cell vehicles will play a crucial role in reducing emissions, but adoption rates and charging infrastructure currently fall short of climate targets. Infrastructure designers are embracing innovations to meet climate and urbanization challenges, such as clean water infrastructure, green infrastructure, electrification, green hydrogen, and high-voltage direct current networks. Leaders in urban areas can enhance resilience with technological innovations, community-based solutions, and sustainable development. Cities contribute significantly to global GHG emissions and face high climate risks due to their dense populations, so they must adopt comprehensive, integrated approaches to climate resilience that address both mitigation and adaptation. Community-based solutions are essential in resilience efforts, as demonstrated by Staten Island's Living Breakwaters project.

Organizations must establish systems for measuring emissions, set targets, and implement strategies to achieve net-zero emissions. This involves accurately measuring GHG emissions, setting science-based targets, and implementing comprehensive strategies. Scientists set a deadline of 2050 to achieve net-zero emissions, but sooner it is better. Tools like wedge diagrams, gap analyses, backcasting, and life cycle assessment can help manage pathways to net zero. Integrating technologies and understanding carbon markets is crucial for achieving net-zero goals. Energy efficiency measures, industry-specific technologies, and carbon removal offsets can help reduce emissions. Carbon markets use compliance-based and voluntary schemes to encourage reductions of emissions and investments in renewables. Assessment systems and standardized reporting frameworks are necessary to manage climate-related risks. Scenario analysis tools like the American Society of Civil Engineers' Future World Vision can help decision-makers visualize potential climate risks and develop adaptive strategies. Innovation must continue to reach global net-zero targets.

Leaders can support climate-related initiatives by achieving early successes, educating stakeholders, and fostering engagement. They should focus on measurable strategies and address issues such as employee retention and regulatory uncertainties. External engagement is crucial for public sector organizations, involving diverse stakeholders. Nature-based solutions (NbS) offer a cost-effective, adaptive, and sustainable approach to environmental challenges, enhancing infrastructure resilience and community well-being. Examples include the restoration of oyster reefs in Florida, which serve as natural barriers against storm waves, protect infrastructure, and improve water quality. Community engagement is essential for tailoring solutions to local needs and addressing climate challenges.

#Codingexercise: https://1drv.ms/w/c/d609fb70e39b65c8/EVdE3xARiidMgiSKPiX554EBVSwvZ0dDxvVyIzK5X2k7gA?e=c7iNMK

 

This is a summary of the book titled “The Equality Machine: harnessing digital technology for a brighter, more inclusive future” written by Orly Lobel and published by Public Affairs in 2022. The author proposes “An Equality Machine” in his drive to use the common grounds of humanity to bridge two disparate and often at opposite ends of the spectrum of people impacted by technology: 1. those who fear new technologies due to their potential to exacerbate existing inequities and 2. those who envision a technological utopia without anticipating risks. The goal of this proposal is to create a better future in which humanity uses “technology for good”. It’s common knowledge that advances in technology such as artificial intelligence and chatbots are recognized both for their potential to empower as well as their drawbacks in meeting equity and fairness. Careful auditing can help algorithms from displaying the same bias as humans do. Making the data more transparent helps to value the labor involved. Feminizing agents and chatbots can normalize existing inequities. New technologies also help to discover gaps in representation and protect people from crime and disease. With their interactions to these technologies, humans are cognizant of their shift in interactions with others and with bots. Makers of chatbots and new technological inventions can explore assumptions that disrupt stereotypes.

The rise of intelligent machines has prompted a need for upholding values of equity and fairness. Technological change has been polarized, with insiders focusing on disruption and embracing new technologies, while outsiders, such as people of color, women, and those from rural areas, worry about exclusion and inequities. To improve machine fairness, humanity must strike a balance between naive optimism and fearful pessimism. Machine learning algorithms can often ascertain identity markers from other data, but this does not address the root causes of inequities. To prevent algorithmic models from reflecting human biases, organizations must be proactive about auditing the output of their AI models as well as their data inputs. Human resources can run hypothetical job candidates through their AI models to test for biases and choose more inclusive data sets. AI decision-making can offer advantages, such as easier dissecting and correcting machine bias than flawed human decision-making. Additionally, predictive algorithmic models can help companies screen a larger pool of applicants for more nuanced qualities, such as high performance and long-term retention. It would be prudent to strike a balance between machine screening and human review.

Technology can help stakeholders work towards a future of financial equity by enabling access to vast amounts of data, identifying and correcting disparities, and reducing biases. Research shows that algorithms created to reduce bias in the fintech industry were 40% less discriminatory than humans. Research also shows that companies are more likely to penalize women for initiating salary negotiations even though men might be praised for assertiveness. AI and societal shifts towards greater data transparency are empowering workers with a better understanding of their labor market value. Some governments have passed legislation banning employers from asking prospective employees to disclose their past salaries. New digital resources, such as Payscale, are bringing greater transparency to the salary negotiation process. Feminizing AI assistants and chatbots can normalize existing inequities, but companies must reflect on the preference to depict subservient robots as female. This reinforces gender as a binary construct and promotes outmoded views of women's roles in society.

Researchers are using new technologies to detect patterns in representation gaps and address systemic inequities. Natural language processing (NLP) methods are being used to analyze large amounts of information, revealing unequal power dynamics and opportunities. AI can be used to assess whether people with different identity markers are getting equitable representation in media forms. Machine learning and AI analytics can help detect gaps in representation and biases in various media industries and inspire more empowering narratives. Technology can also help protect people from harmful influences by enabling organizations to share data and develop data hubs. AI and health data can also help stakeholders accelerate drug discovery and collaborate to prevent the global spread of viruses. However, democratizing AI use in medical research contexts is crucial to ensure improved health outcomes for everyone, not just the rich.

Algorithms and embodied robots are transforming human connection and social bonds. Algorithmic biases can exacerbate existing class, racial, and social divides, while the growing prevalence of robots with sexual capacities is transforming intimacy and emotional connection. Some argue that framing robots solely as AI-empowered sex dolls is oversimplification, while others worry about the potential for violence against women.

Roboticists can challenge stereotypes by creating robots that challenge assumptions. Embodied robots can support humans in various functions, such as care labor, reception work, and space exploration. However, some critics worry about privacy risks, consent, and misuse of data.

Robots can surprise those they interact with by disrupting expectations. NASA uses feminine-looking robots like Valkyrie to support in-space exploration, while masculine-looking robots like Tank act as "roboceptionists." These robots demonstrate the choice roboticists face when designing robots that cater to existing biases or inspire imaginative new possibilities.

#codingexercise: CodingExercise-01-07-2025.docx

 With year over year advances in Large language models are becoming popular faster than ever, this is a summary of all the advances made last year including the learnings. First, the GPT-4 barrier was completely broken. Among seventy other competitors who overtook GPT-4 including Google’s Gemini and Anthropics’ Claude3, LLMs have successfully tackled long inputs where entire books can be thrown at the LLM and it would answer questions with high precision and recall and even solved coding challenges with trainable samples that users provide. Second, these models can now run on laptops even though GPT-4 earlier used to require datacenters with one or more $40,000 plus GPU. On a laptop, 64 GB of RAM is sufficient to train these models. Third, these competitions reduced the prices of LLMs significantly and provided more value by virtue of increased efficiency. Earlier the price was $30 per million tokens on OpenAI for GPT-4 while GPT-4o is now $2.50 per million tokens. These price drops are driven by two factors: increased competition and increased efficiency. The price drops tie to how much energy is being used for running prompts and the efficiency thing is really important for those concerned with the LLM’s environmental impact. Fourth, LLMs are becoming multi-modal. In computer vision, being able to vectorize images as well as text and use them in queries with vector search has proved tremendously successful in many use cases. Now, audio and video are starting to emerge. In Nov 2023, GPT-4 was the leader with multimodal vision and that is now caught up by many other models. For example, Amazon’s Nova has garnered support for both image and video and is available on their public cloud. This brings us to our fifth advancement, voice and live camera mode are science fiction come to life. Just a year back, audio, and live video mode were just an illusion. OpenAI’s Whisper speech-to-text model and a text-to-speech model enabled conversations with ChatGPT’s mobile apps but the actual model just saw text. The GPT-4o in 2024 is a true multi-model model that could accept audio input and generate incredibly realistic sounding speech without the separation. Sixth, prompt driven app generation is now mainstream and virtually permeated every industry sector. We knew they were good at writing code and entire static websites could be created with a single prompt, but Claude Artifacts took it up a notch by writing an on-demand interactive application that lets you use it directly inside the Claude interface. Seventh, universal access to the best models lasted just a few short months. OpenAI made GPT-4o free to all users unlike earlier where users did not know the latest as they were allowed access to earlier versions. This has changed with a monthly subscription service where they can keep up with the latest. Eighth, “Agents” still haven’t really happened yet. The term itself was extremely frustrating in that it did not clarify the purpose as whether the agents did low-level work or could also do high-level composites. And to make it more ambiguous, the term autonomous was thrown into the mix. They were not supposed to make meaningful decisions themselves because that eventually hits a roadblock or leads to hallucinations. And just as if to prove that, prompt injection leveraged this ambiguity. Ninth, Evals really matter. As one expert put it, the boring but crucial secret behind good system prompt is test-driven development. You don’t write down a system prompt and then find ways to test it. You write down tests and then find a system prompt that passes them. Tenth, there is a rise of inference-scaling “reasoning” models. This is an extension of chain-of-thought prompting trick where if you get a model to talk out loud about a problem it’s solving, you often get a result which the model would not have achieved otherwise. The biggest innovation here is that it opens up a new way to scale the model as they take on harder problems by spending more compute on inference.

Reference: previous articles & simonwillison.net