Considerations in testing software deployed to Minikube 

Products that are hosted on Minikube are expected to work the same as if they were deployed on any other Kubernetes container orchestration framework. This remains the case when the minikube deployments are large.  When they are small, they are more prone to failures. Additional validations are required for small deployments since they are not at par with a fully deployed instance on any cluster. 

1. The Minikube hosting feature is just like any other hosting platform such as AWS. It adds another dimension to existing test cases that involve a change of host on which the product runs. 

2. The product may support both install and upgrade. The new test cases will usually target install but they can be expanded to include upgrade as well. 

3. The product upgrade paths may vary if it is a patch or major version upgrade. Both paths will usually be supported on Minikube. This remains true for both small and large deployments of Minikube. 

4. Minikube’s support is usually host facing as opposed to the external world. It requires a docker registry that is local or reachable from the pods to be able to pull the images. The use of local docker registry with or without tls is an important consideration. 

5. The small deployments of Minikube should target lower number of replicas and containers. The specification of cpu, memory and disk for the whole host does not necessarily lower the size of various clusters used in this feature. A minimal-dev-values file is provided as guidance for lowering the number of replicas and containers and their consumption. 

6. Access to the cluster should be tested from both User Interface as well as with Kubectl commands. If the product is used to create namespaces for the users, testing may include one or two projects on Minikube because this is the typical case. A large number of namespaces is neither required nor supported on small deployments due to resource constraints.  

7. Error messages on using the user interface will not be tweaked since it is the same for the product across deployments regardless of size or flavor. A few negative test cases targeting error messages could be helpful. 

8. Security configuration of the Minikube installation is lower priority since instance is already completely owned. Still it might be better to test with the pre-install script provided in the scripts directory. Options for Pravega security such as setting tls on pravega controller can also be exercised. 

9. Small Minikube deployments are expected to have frequent restarts due to low resources. They should not number into hundreds. A test case that allows the Minikube deployment to run for some time will be helpful in this case. 

10. If the product deployed on minikube hosts user code, then that code should be tweaked to utilize resources within reasonable limits for what is available to the product. 

These are some of the considerations that make deployment validations different on Minikube.