Multiprocessing

Module for handling multiprocessing Workers take a Task and check which subtasks can be executed

If subtasks generate figures, matplotlib needs to use a compatible backend. Tested backends: ‘pdf’ The main process should therefore execute:

>>> import matplotlib
... matplotlib.use('pdf')

Todo:

• connect to workers on other servers
• distribute TaskBase depedencies over the workers

Example:

>>> import apetype as at
... import apetype.workers
... import time, numpy as np
... 
... class Task(at.TaskBase):
...     a: int = 5
... 
...     def ar(_, a) -> np.ndarray:
...         return np.array([a*i for i in range(10)])
... 
...     def aplus10(_, ar: at.tasks.InjectItems) -> list:
...         print('y', ar)
...         time.sleep(2)
...         return ar+10
... 
...     def amaal10(_, ar, aplus10) -> np.ndarray:
...         return ar*aplus10
... 
... task = Task(parse=True)
... manager = at.workers.Manager(task, max_workers_x_subtask=2)
... manager.start()

class apetype.workers.Manager(task, *args, **kwargs)

class apetype.workers.WorkerProcess(task, number)

run()

Method to be run in sub-process; can be overridden in sub-class