Jug allows you to write code that is broken up into tasks and run different tasks on different processors.
It uses the filesystem to communicate between processes and works correctly over NFS, so you can coordinate processes on different machines.
Jug is a pure Python implementation and should work on any platform.
Python versions 3.9 and above are supported (will almost certainly work on earlier versions as well, but they are not part of the CI suite).
Documentation: https://jug.readthedocs.io/
Mailing List: https://groups.google.com/group/jug-users
"I've been using jug with great success to distribute the running of a reasonably large set of parameter combinations" - Andreas Longva
You can install Jug with pip:
pip install Jug
If you want to use jug shell, install IPython as well:
pip install Jug ipython
Or, if you use conda, you can install jug from conda-forge using the following commands:
conda config --add channels conda-forge
conda install jug
If you use Jug to generate results for a scientific publication, please cite
Coelho, L.P., (2017). Jug: Software for Parallel Reproducible Computation in Python. Journal of Open Research Software. 5(1), p.30.
Here is a one minute example. Save the following to a file called
primes.py (if you have installed jug, you can obtain a slightly longer
version of this example by running jug demo on the command line):
from jug import TaskGenerator
from time import sleep
@TaskGenerator
def is_prime(n):
sleep(1.)
for j in range(2,n-1):
if (n % j) == 0:
return False
return True
primes100 = [is_prime(n) for n in range(2,101)]
This is a brute-force way to find all the prime numbers up to 100. Of
course, this is only for didactic purposes, normally you would use a
better method. Similarly, the sleep function is so that it does not
run too fast. Still, it illustrates the basic functionality of Jug for
embarrassingly parallel problems.
Type jug status primes.py to get:
Task name Waiting Ready Finished Running
----------------------------------------------------------------------
primes.is_prime 0 99 0 0
......................................................................
Total: 0 99 0 0
This tells you that you have 99 tasks called primes.is_prime ready to
run. So run jug execute primes.py &. You can even run multiple
instances in the background (if you have multiple cores, for example).
After starting 4 instances and waiting a few seconds, you can check the
status again (with jug status primes.py):
Task name Waiting Ready Finished Running
----------------------------------------------------------------------
primes.is_prime 0 63 32 4
......................................................................
Total: 0 63 32 4
Now you have 32 tasks finished, 4 running, and 63 still ready.
Eventually, they will all finish and you can inspect the results with
jug shell primes.py. This requires ipython to be installed and will
give you an ipython shell. The primes100 variable is
available, but it is an ugly list of jug.Task objects. To
get the actual value, you call the value function:
In [1]: primes100 = value(primes100)
In [2]: primes100[:10]
Out[2]: [True, True, False, True, False, True, False, False, False, True]
Released 20 September 2026
Two changes alter task hashes, so cached results for the affected tasks will be recomputed the first time you run them with this version:
- The pickle protocol used for hashing is now pinned to protocol 4. This only affects users of Python 3.14 and 3.15 (where pickle's default is protocol 5); Python 3.13 and earlier already used protocol 4. From now on, task hashes are stable across Python versions.
- Tasklets built from
lambdafunctions are now hashed using the lambda's constants, referenced names, closure and default arguments (previously only its bytecode was used, so lambdas that differed only in those hashed identically).
- Better error message when loading results fails (patch by Justin R. Porter, GH #92)
- Configuration files are now read as UTF-8 regardless of locale
jug cleanupnow reports the number of removed objects and of removed locks separately
- Use
@propertyand@abstractmethodinstead of the deprecatedabstractproperty - Modernize Python idioms and update documentation
- Fix
write_task_outfor numpy arrays, which are now written in.npyformat (they were silently pickled instead) - Fall back to pickle for numpy arrays of object dtype, which cannot be saved in numpy's native format
file_store.cleanup()no longer deletes the temporary files of workers that are still running- Fix the
tputfallback inget_terminal_size - Fix wrong exception type in
dict_store.cleanup() - Fix saving to/loading from a file backend in
jug.backend.dict_storeon Python 3 (pickle files must be opened in binary mode) - Fix help text of
jug status --cache-file
Released 12 March 2026
- Special case saving
polarsDataFrames infile_storefor speed. - More flexible parsing of booleans in
jug.options. - Support project-local configuration files (
.jugrcorjugrc). Jug now walks up the directory tree from the current working directory (up to the git project root) looking for local configuration files. Seeconfigurationfor details. - Ship the Jug assistant skill in the Python package and add
jug install-skills --output DIRto install it into Codex or Claude Code skills directories. Seeai-assistantsfor usage details.
- Fix
_get_terminal_size_linuxfor Python 3.14, which changed howfcntl.ioctlhandles string arguments. Useos.get_terminal_size()instead (patch by justinrporter, GH #90). - Fix
jug.backend.dict_storefor Python 3. - Fix
describeinjug.taskfor Python 3.
Released 8 May 2025
- Adds support for lambda functions in
Tasklets - Adds
NoHashclass to disable hashing for some arguments. This is injug.unsafeas it can be used to "fool" Jug, but it can be useful when there are nuisance arguments that are not relevant for the task (e.g., number of threads) - jug.file_store: create files with better permissions
- Convert to
pyproject.tomlfor building
- Better error detection for permission problems
- Bugfix when using local imports and
jug pack
Drops support for versions of Python older than 3.7. Technically, it should still work, but they are too old to test in Github CI, so we will not support them.
Released 5 November 2023
- Update for Python 3.12
Released 25 June 2023
- jug shell: Add
get_filtered_tasks() - jug: Fix
jug --version(which had been broken in the refactoring to use subcommands) - jug shell: Fix message in jug shell when there are no dependencies (it would repeatedly print the message stating this will only be run once)
- jug pack: Make it much faster to invalidate elements
- file_store: ensure that the temporary directory exists
- Drops support for Python 3.4
For older version see ChangeLog file or the full
history.