scheduling workflow
The task scheduling work is performed using dask as describe in i2_scheduling.html by using dasks.delayed objects and the use of a localCluster in Iota2.py. If I remember well, this choice has mainly be made considering the ability of dask to create LocalCluster, PBSCluster and kubernetes objects which was convenient for our purposes (monitor tasks on different architectures but not for it's scheduling API interface. Moreover, PBSCluster and kubernetes objects are not really use in iota2.
some libraries deserve attention for their scheduling API and replace our homemade scheduling API :
I haven't investigated these scheduling libraries much, but the luigi
API looks interesting.