Investigate Numba for custom features computation
We are opening up the possibility for users to easily insert numerical python code in the iota2 worflow. This code is aimed to operate at the pixel or local neighborhood level for the kind of operations that we would usually write in C++.
Moving to python at this low level (low wrt the scheduling or distributed computing levels) can incur in loss of efficiency wrt native and multi-threaded code. When looking for ways to make python code run faster one can mainly use Cython or Numba (see this post for a quick comparison).
For iota2 user provided code, Numba seems to be a better option since there is no separate compilation step and only a decorator needs to be added to the functions provided by the user.
An example where Numba yields a 3x speedup without changing the code:
@njit(uint8(uint8[:], uint8), nogil=True, cache=True)
def majority_voting(labels: np.ndarray, undecided: np.uint8 = 0,
max_labels: np.uint8 = 20) -> np.uint8:
""" Perform majority voting of a list of labels """
counts = np.zeros(max_labels)
for label in labels:
counts[label] = counts[label] + 1
sorted_counts = np.sort(counts)
if sorted_counts[-1] == sorted_counts[-2]:
return undecided
else:
return np.argmax(counts)
We could even imagine dynamically adding the decorator so that the user does not need to do it.
There are maybe other places in iota2 where we can leverage Numba.
The goal of this issue is just raising awareness about this technology and start a discussion. Feel free to comment.