Lessons learnt from build environment enhancements for building Sofware applications.

Introduction: Building software applications to produce the distributable is a routine for all applications. The logic in the build and automation systems is an investment for an organization and requires many man-hours for development. This article captures some of the lessons learned from different build environments.

Description: Build automations do not just invoke the compiler to build and link the code. It is an opportunity to introduce workflows that validate and vet the application, raise the software quality and introduce numerous gates for the application to be released.  Some of these are enumerated below:

1) The software application is dependent on several dependencies. These must be managed and updated so that the application remains defects and vulnerabilities free. Maintaining a source of truth for the application builds is an important aspect of the build system. Software the builds locally on developer workstations is fraught with environmental factors that are not often visible to the developers. Ensuring that the application always builds in a controlled environment even if the dependencies and modules are recreated each time, is one of the core responsibilities of the build.

2) Starting from a clean state to not pollute the environment with pre-existing binaries and builds is necessary for the build system. This is true for both the developer builds as well as the remote branch builds. These builds must make it easy for clean up so that the task of bringing in just the minimum for builds is done right each time.

3) Access permissions to the build must be tightly controlled. The introduction of tools and the versioning of the software cannot be modified without proper processes in place. It may be required for developer teams to introduce their own build logic.  A specific format, and file is required for this purpose. The introduction of the additional logic via the checkin of the file is like the introduction of code into the pipeline which requires no escalation of privilege.

4) The versioning of the builds must be proper. A lot of work has gone into the naming system, and this works for most versioning. It is called SemVer which is short for semantic versioning. Care must be taken to introduce the version stamps into each builds and binary so that the origin of a particular assembly can be unequivocally displayed.

5) the use of data protection, encryption and certification tools is as relevant to build artifacts as they are for securing customer data. There must be no tampering of artifacts or the cross contamination of dependencies without requiring these to be clean.

6) Several tools may be introduced into the build for static analysis and verification of the code as well as that of the binaries produced by the build. Recently these have included artifacts and images from container orchestration frameworks.

Lastly, the build automation must scale to requests from thousands of repositories and organizations. This requires a pool of resources to be always available.