Build a Thread PoolStep 5 of 5

Distributing Across Workers

12345

Step 5: Distributing Across Workers

A real pool spreads work across its num_workers so no single worker is overloaded. Model that assignment: as run() processes the queue, assign each job to the next worker round-robin and tally how many jobs each worker handled.

  • track per-worker job counts (initialize a list of num_workers zeros)
  • in run(), assign job number i to worker i % num_workers and increment that worker's count
  • worker_loads(): return the per-worker counts

With 5 jobs across 2 workers you'd get [3, 2]: the fair round-robin split.

Hints
Starter code
from collections import deque


class Future:
    def __init__(self):
        self._done = False
        self._result = None
        self._exc = None

    def set_result(self, value):
        self._result = value
        self._done = True

    def set_exception(self, exc):
        self._exc = exc
        self._done = True

    def done(self):
        return self._done

    def result(self):
        if not self._done:
            raise RuntimeError("future is not done")
        if self._exc is not None:
            raise self._exc
        return self._result


class ThreadPool:
    def __init__(self, num_workers):
        self.num_workers = num_workers
        self._queue = deque()
        self._worker_jobs = [0] * num_workers

    def submit(self, fn, *args, **kwargs):
        fut = Future()
        self._queue.append((fut, fn, args, kwargs))
        return fut

    def run(self):
        """Drain the queue, assigning each job to a worker round-robin."""
        i = 0
        while self._queue:
            fut, fn, args, kwargs = self._queue.popleft()
            # TODO: worker = i % self.num_workers; self._worker_jobs[worker] += 1; i += 1
            try:
                fut.set_result(fn(*args, **kwargs))
            except Exception as e:
                fut.set_exception(e)

    def worker_loads(self):
        """Return the number of jobs each worker handled."""
        # TODO: return list(self._worker_jobs)
        pass
Write it on a laptop

The editor and the test runner need a wide screen. Open this page on a desktop browser and your progress will be waiting.

PreviousAll challenges