IntroductionThis page outlines relevant python packages, patches/etc that pertain to concurrent and threaded (thread like) programming. Process Creation- Parallel Python
- Job dispatcher, can fork process pools for multiple cores/cpus or across machines in a cluster.
- Processing
- Package for using processes mimicking the threading module.
- Pycopia-Process
- Modules for running, interacting with, and managing processes. A process manager for spawning and managing multiple processess. Support for Python coprocess. Expect module for interacting with processes. Can connect with pipes or pty. Objects for status reporting and process information.
- Papyros
- Papyros is a small platform independent parallel processing Python package, providing a master-slave model. Clients can submit jobs to a master object which is monitored by one or more slave objects that do the real work and return the processed jobs back to the master.
- Pprocess
- The pprocess module provides elementary support for parallel programming in Python using a fork-based process creation model in conjunction with a channel-based communications model implemented using socketpair and poll.
- Note, there's a Tutorial
- ChattyParalel
- Offload tasks to child processes, using a fork based processing model. A variation on Paul Boddie's pprocess.
- pyMPI
- MPI bindings for Python! Brings distributed parallel programming to Python from small SMP to huge clusters.
Concurrency- Candygram
- Quote: Candygram is a Python implementation of Erlang concurrency primitives. Erlang is widely respected for its elegant built-in facilities for concurrent programming. This package attempts to emulate those facilities as closely as possible in Python. With Candygram, developers can send and receive messages between threads using semantics nearly identical to those in the Erlang language.
- This recipe on ASPN outlines how to use candygram to inter-communicate between threads.
- Python Object Sharing
- POSH "is an extension module to Python that attempts to address the problems associated with the GIL by enabling placement of Python objects in shared memory"
Distributed Objects- Pyro is a remote method invocation (rmi) implementation for python. It's the Python equivalent of .NET remoting or Java RMI, it's also a lot faster than xmlrpc.
|