Paging a Backend Lazily

Step 3: Paging a Backend Lazily

Real data does not live in one list; it lives behind a backend that serves fixed-size pages. PagedSource models that: it stores the full item list, exposes page(index) returning that page's slice (or an empty list past the end), and counts every fetch in self.fetches.

Now build Cursor, an iterator that walks the whole dataset across pages while the caller sees one flat stream of items. The trick is laziness: the Cursor keeps the current page in a buffer and only calls page() again when that buffer is used up. Implement next to refill the buffer on demand, fetch the next page, and raise StopIteration when an empty page comes back.

Hints
Starter code
class ListCursor:
    def __init__(self, items):
        self._items = list(items)
        self._index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._index >= len(self._items):
            raise StopIteration
        value = self._items[self._index]
        self._index += 1
        return value

    def take(self, n):
        result = []
        for _ in range(n):
            try:
                result.append(next(self))
            except StopIteration:
                break
        return result


class PagedSource:
    def __init__(self, items, page_size):
        self.items = list(items)
        self.page_size = page_size
        self.fetches = 0

    def page(self, index):
        self.fetches += 1
        start = index * self.page_size
        return self.items[start:start + self.page_size]

class Cursor:
    def __init__(self, source):
        self.source = source
        self._page_index = 0
        self._buffer = []
        self._pos = 0
        self._done = False

    def __iter__(self):
        return self

    def __next__(self):
        """Return the next item, fetching a new page only when the buffer runs out."""
        # TODO:
        #   while the current page buffer is exhausted, fetch the next page
        #   (self.source.page(self._page_index)); if that page is empty the
        #   cursor is done and you raise StopIteration. Otherwise store the page
        #   as self._buffer, reset self._pos, then return self._buffer[self._pos]
        #   and advance self._pos.
        pass
Write it on a laptop

The editor and the test runner need a wide screen. Open this page on a desktop browser and your progress will be waiting.