CNCF-Hosted Co-Located Events Europe 2023

Combining Argo Workflows and Kubernetes can be extremely powerful. When configured correctly, there is no limit to the parallelism that users can achieve when running workflows. However, one of the most common bottlenecks that people run into is that they can’t get the “ReadWriteMany” access mode to work properly when using volumes. This limits the parallelism for the workflow being run because only one pod will be able to access the volume at a time if “ReadWriteMany” is not used. Additionally, most cloud-managed Kubernetes clusters will need some additional configuration in addition to the default settings to get the storage required to play nicely with Persistent Volumes that you can spin up with Argo Workflows. In this talk, we encourage you to consider using a ReadWriteMany disk instead of defaulting to Argo Workflows artifacts. We go over how to set up a Kubernetes cluster with an NFS provisioner using Argo CD. We will then show how to make it play nicely with Argo Workflows, allowing users who submit workflows to dynamically provision volumes that can be accessed in parallel by any pods within a given workflow. Additionally, we will go over use cases in which users would benefit from this setup. Principally, we will discuss how to parallelize Docker builds in a monorepo context.