Database Daily: Scanning a Page at a Time

2 May 2023

Goal: Update the scanner class to be page-aware, using BufferPool to fetch data from disk during iteration

I made good progress this morning updating the Scanner class to rely on the BufferPool to do disk reads on page at a time. The scanner asks for a page, iterates through it, returns it, and then requests another.

This led to some light refactoring in the scanner class itself to recover from null reads more gracefully. Additionally, I mocked out a reference counting strategy that the BufferPool can use to evict buffers that are no longer in use.

I left myself a number of TODO’s in buffer_pool.rb - next time I’ll work through those so that we’ll have a (naive) eviction strategy in place and we can see a scanner over ratings.db iterate through all records with the BufferPool running out of memory.

Written by Dan Drust on 2 May 2023

Continue Reading: Database Daily: Buffer Pool

Browse more posts