fork_map is like python's builtin map function, but uses os.fork to execute the mapped
function in child processes. Unlike the builtin multiprocessing map functions, fork_map requires only its outputs (not its inputs) to be pickleable.
from fork_map import fork_map
result = fork_map(lambda x: x*2, range(10))The example above isn't possible with multiprocessing's Pool.map, because lambdas aren't pickleable.
Because fork_map uses os.fork, it has the same limitations as os.fork, namely:
- It only works on operating systems where it's available (e.g., not Windows).
- It is not safe to use in multithreaded code, because deadlocks may occur. See the warnings on
os.forkor this discussion for more information.
I wrote an article explaining the drawbacks to using os.fork like this. Read it here: