This is a continuation of an earlier article on executing Apache Spark code on Azure Machine Learning Workspace.

Additionally, there are some caveats to mention:

 

1. Container registry must be accessible to the workspace and the image build serverless compute. Allowing incoming ip addresses on the container registry and outbound traffic to the container registry on the subnet associated with all the compute in the workspace, are prerequisites.

2. If the workspace is setup with limited public plane connectivity, it will likely impact the reachability to the azure container registry. Allowing unlimited public access is not required but helps with troubleshooting. 

3. If the compute are all provisioned under a specific subnet, associating a NAT gateway with those compute allows for a fixed ip prefix or even address for all outbound traffic from them. This will be helpful to allow list on the dependencies of the Azure Machine Learning Workspace, whether they are container registry, storage account or keyvault.

4. If there are errors authenticating with the container registry and the image build jobs in the workspace fail with the authentication error complaining about incorrect json, the sync-keys command on the workspace will restore the authentication.

Previous articles: IaCResolutionsPart103.docx

#CodingExercise-04-11-2024 https://1drv.ms/w/s!Ashlm-Nw-wnWhOw8Qdf84RkOPsNMuQ?e=7R8m69