@hackage / pqi-conformance

pqi-conformance

A reusable hspec toolkit that checks any pqi adapter against the battle-tested postgresql-libpq library as a reference.

Goal: full libpq fidelity

The purpose of this suite is to enforce byte-identical output to libpq for every protocol-derived value. This means error message strings, notice text, result status, field metadata, cell data, and all structured error fields must all match libpq's output exactly, not just in shape or presence.

The suite runs the same operation on the candidate adapter and on a direct postgresql-libpq reference connection (the fidelity reference, which delegates directly to the C libpq library), connected to the same throwaway PostgreSQL container. It then asserts that the protocol-derived observations are equal.

Structurally incomparable values

A small number of values are structurally incomparable across connections and are handled differently:

• backendPID — the OS process ID of the backend. Each connection gets a distinct backend, so the two PIDs will never match. The spec asserts > 0 independently for each adapter.
• socket — the file descriptor of the client socket. Also connection-specific. Covered only by its own presence check.
• Notify.bePid — the PID of the notifying backend. Since each adapter's connection has its own backend, cross-adapter comparison would always fail. Instead, each adapter asserts independently that notification.bePid == backendPID connection (a within-connection assertion that verifies the PID field is correctly populated).

These omissions are a structural constraint of the differential testing approach, not an intentional leniency in the suite. All other values — including error message text, notice text, and cancel error text — are compared in full.

Structure: one module per operation

The suite is organised as one spec module per API operation, under Pqi.Conformance.Operation.* — ...Operation.Exec, ...Operation.ExecParams, ...Operation.LoSeek, ...Operation.Fnumber, and so on, one for every public method of Pqi.IsConnection, Pqi.IsResult, and Pqi.IsCancel, plus the standalone Pqi.unescapeBytea. Each module holds the differential scenarios that exercise that one operation (its happy paths, its edge cases, and its error paths). Shared scenario fragments live in Pqi.Conformance.Scenario.

Usage

An adapter's own test suite is a one-liner — it hands specs a Proxy of its connection type and specs takes care of booting the container and running the whole battery (every operation, the coverage meta-test, and SCRAM):

module Main (main) where

import Data.Proxy (Proxy (..))
import Pqi.Conformance (specs)
import MyAdapter (MyConnection)
import Test.Hspec (hspec)

main :: IO ()
main = hspec (specs (Proxy @MyConnection))

MyConnection only needs a Pqi.IsConnection instance; the candidate and the postgresql-libpq reference are both driven through that interface.

Contributing

Contributions extending the suite are very welcome. The more thoroughly we cover the operations, the more confidence we can have in the adapters.