Multi-processing map() for Python

I think that Python should use multi-processing and/or multi-threading to take advantage of as many opportunities for parallel execution as possible. To this end, I've written a drop-in replacement for map() that runs across as many processes as requested. It should be otherwise identical in every way the built-in version (and if it's not please let me know!).

I also wrote a version based on Parallel Python that is a lot simpler but not quite identical to the original. In particular, it returns a generator instead of a list of values so that program execution doesn't block until the results are fetched.

Drop me a line if you find this interesting or useful or just plain dumb.

Download: forkmap.py

blog comments powered by Disqus