PostgreSQL on Shallow Brook Software

Prepared Statement Caching: An Easy Win?

Sun, 15 Dec 2024 00:00:00 +0000

In PostgreSQL, prepared statements are a means of optimization for executing SQL queries. Normally, SQL queries go through multiple phases of processing before being executed: parsing, rewriting, and planning. If your program executes the same query multiple times (as applications often do), then rerunning all of these steps each time is a bit redundant. By “preparing” the statement ahead of time, you can parse and rewrite the query once and then reuse the result. This means that a prepared statement only needs to be planned (and not parsed or rewritten) before execution.

Simple Database Migrations in Go

Sun, 10 Nov 2024 00:00:00 +0000

Historically, database migrations were a facet of web development that’d cause me to reach for a third-party library. I eventually discovered, however, that handing migrations is actually quite simple if you don’t need many features. At a high level, the process is as follows: list out the migration files, check which ones are missing, then apply them in order (each within a transaction). This whole workflow can be implemented in as few as ~100 lines of readable code!

Migrating Numeric IDs to UUIDs

Sun, 03 Nov 2024 00:00:00 +0000

There are a few ways to represent primary keys within a database. The most common way (in my experience) to use auto-incrementing integers. This means that first row’s key will be 1, the second’s will be 2, and so on. When it comes to my preferred style of software development, however, this approach has a major downside: new domain objects must first consult the database to determine their ID.

While I’m far from a domain-driven design (DDD) purist, I like the idea of being able to represent the core domain objects and business logic of an application in complete isolation from the outside world. Having domain objects depend on the database for IDs makes this impossible (or at least very difficult). If only there existed a different kind of unique ID that could be generated by the domain objects themselves without requiring any coordination with an external system…

Two Go + PostgreSQL Timestamp Gotchas

Sun, 30 Jun 2024 00:00:00 +0000

This week I added some additional tests to my Bloggulus project. In the process of doing so, I discovered a couple places where timestamp values weren’t matching what was expected. For some background, I generate, process, and store all timestamps in the UTC time standard.

The two gotchas were:

Unexpected conversion of UTC timestamps to local time
Mismatched precision between Go and PostgreSQL timestamps

Unexpected Conversion Link to heading

I store timestamps in the database using PostgreSQL’s timestamptz data type. The problem I ran into was that even when I was inserting proper UTC timestamps (generated by Go’s time package), they were being returned from the database in my local time zone. What’s going on? Why are these UTC timestamps being converted to Central Standard Time when selected from the database?

Digimon Cards and Database Indexes

Sun, 23 Jun 2024 00:00:00 +0000

I’m a big fan of the Digimon Card Game (the one released in 2020). The games are quick, the mechanics are engaging, and the community is friendly. Plus, I was a big fan of the Digimon anime as a kid so I have a lot of nostalgia for the characters. When I first got into the game, I realized that only a few sets had been released so far. I set my eyes on building and maintaining a complete collection: four copies of each card. With enough spreadsheets and time, this would allow me build any deck that I wanted. This was honestly a breath of fresh air coming from Magic: The Gathering.

Reinforcing Indirect Joins

Sun, 16 Jun 2024 00:00:00 +0000

Recently, I ran into an issue where a very large PostgreSQL query (involving roughly 30 tables) was taking much longer to plan than to execute. At its worst, it was taking multiple seconds to plan and a few hundred milliseconds to execute. My problem wasn’t really about standard query optimization: I was already past that. This was a problem of query complexity and trying to get the PostgreSQL planner/optimizer to more quickly arrive at an ideal query.

Using Newer PostgreSQL Client Tools in GitHub Actions

Sun, 05 May 2024 00:00:00 +0000

Recently, while updating my pg2s3 utility, I noticed that the project’s docker-compose.yml file was pinning PostgreSQL to version 14. I couldn’t remember why I did that, so I went ahead and removed it (what could go wrong?). Unfortunately, this led to some automated tests failing in GitHub Actions!

The Problem Link to heading

Thankfully, the error was very clear:

pg_dump: error: server version: 16.2 (Debian 16.2-1.pgdg120+2); pg_dump version: 14.11 (Ubuntu 14.11-1.pgdg22.04+1)
pg_dump: error: aborting because of server version mismatch

Classic version mismatch. The PostgreSQL 16 server (running in a container) was not compatible with the PostgreSQL 14 client tools (installed on the GitHub Actions runner). For some context, pg2s3 uses pg_dump and pg_restore to quickly export and import data.

Testing with Transactions

Sun, 24 Mar 2024 00:00:00 +0000

Most web applications eventually end up with tests that need to interact with a database. Perhaps your business logic is tightly coupled to the database or maybe you are wanting to test a clearly-defined storage layer. Either way, a common problem arises: how do you clean up the data used during testing? What should you do with all those scattered rows?

Most of the time, developers will resort to a few common strategies: