Fix test/install, add test-build, verify-results, docs

[jnasbyupgrade]

test-build: Useful error messages for SQL syntax errors

When CREATE EXTENSION fails due to a syntax error, PostgreSQL reports a
cryptic error with limited context. test-build runs your extension SQL directly
through pg_regress first, so errors show the exact file, line, and position —
cutting debugging time significantly.

Place SQL files in test/build/ to enable. Auto-detects based on file presence.

test/install: Run expensive setup once instead of per-test

Extensions that install dependencies or load fixtures in every test file pay
that cost once per test. test/install runs setup SQL once before the entire
suite, and all regular tests share the resulting database state. This can
dramatically speed up test suites.

Place SQL files in test/install/ to enable. Auto-detects based on file presence.

• Uses a pg_regress schedule file so install and test files run in a single
  invocation (database state persists across phases)
• Expected output lives alongside .sql files as test/install/*.out

verify-results: Prevent blessing incorrect test output

make results now refuses to run when tests are failing. Prevents accidentally
updating expected files with wrong output. Disable with
PGXNTOOL_ENABLE_VERIFY_RESULTS=no.

Other changes

• Add .asc as a recognized asciidoc extension
• Document all new features in README

Companion PR: Postgres-Extensions/pgxntool-test#7

[jnasbyupgrade]

Enhancement request: Override mechanism for install/build file lists

I'm planning to use the test/install support for pg_tle integration in my extensions monorepo. The auto-detection is great for the common case, but I need to control which files run and in what order. Two requests:

1. Documented override for test/install and test-build file lists

TEST_INSTALL_SQL_FILES is auto-populated via wildcard, and the schedule is generated directly from it. There's no clean way to:

• Control ordering: Wildcard gives filesystem order. I need pgtle-setup to run before extensions.
• Conditionally include files: I have a pgtle-setup.sql that should only run when PGTLE_MODE is set (Docker/RDS uses pg_tle; local PGXS doesn't).

Suggested approach: Expose an intermediate variable that the schedule generation uses, documented as the public API for overriding:

# Default: auto-detected from test/install/*.sql (unordered)
# Projects can override for ordering or conditional inclusion.
PGXNTOOL_INSTALL_TESTS ?= $(notdir $(basename $(TEST_INSTALL_SQL_FILES)))

$(PGXNTOOL_INSTALL_SCHEDULE):
	@echo "# Auto-generated - DO NOT EDIT" > $@
	@for f in $(PGXNTOOL_INSTALL_TESTS); do \
		echo "test: ../install/$$f" >> $@; \
	done

Then projects can do:

include pgxntool/base.mk

# Explicit ordering + conditional pg_tle setup
PGXNTOOL_INSTALL_TESTS = extensions
ifdef PGTLE_MODE
PGXNTOOL_INSTALL_TESTS := pgtle-setup $(PGXNTOOL_INSTALL_TESTS)
endif

Same pattern would be useful for test-build (PGXNTOOL_BUILD_TESTS). The auto-detect remains the default — this just gives projects an escape hatch with a stable API.

2. Document new make targets to help with conflicts

Projects that already hand-coded test-build or verify-results (like my archive extension) will hit recipe conflicts on upgrade. The PGXNTOOL_ENABLE_* variables handle this, but only if people know about them.

Suggestions:

• README should have a "New Targets" or "Upgrading" section listing all new target names so upgraders can check for conflicts
• Ideally upgrade.sh could grep the project Makefile for potential conflicts with newly-introduced targets (e.g., grep -w 'test-build\|verify-results' Makefile). This is tricky since you'd be upgrading across potentially many pgxntool versions, but even a best-effort warning would help. Not a blocker for this PR though.

[jnasbyupgrade]

.source files would also be checked in

[jnasbyupgrade]

Agreed. Will update the diagram to note that .source files in input/ are also checked in.

[jnasbyupgrade]

This is almost the same code as for test/build; propose ways (in a comment) it could be refactored. We certainly want the code to remain clear, but I'm hoping we can remove some of the duplication.

See verify-results as well; it's pretty similar as well.

[jnasbyupgrade]

The duplication is in the variable validation/normalization blocks (lines 98-115, 141-158, 175-188). All three follow the same pattern:

1. If variable is set: normalize to lowercase, validate yes/no, error on bad value
2. If not set: auto-detect (wildcard check or default)

Options for refactoring:

A. GNU Make call function — define the validation as a reusable template:

# $(1) = variable name, $(2) = file list or "yes" for default-enabled
define pgxntool_feature_detect
ifdef $(1)
  $(1)_NORM = $$(strip $$(shell echo "$$($(1))" | tr '[:upper:]' '[:lower:]'))
  ifneq ($$($(1)_NORM),yes)
    ifneq ($$($(1)_NORM),no)
      $$(error $(1) must be "yes" or "no", got "$$($(1))")
    endif
  endif
  override $(1) := $$($(1)_NORM)
else
  ifneq ($$(strip $(2)),)
    $(1) = yes
  else
    $(1) = no
  endif
endif
endef

$(eval $(call pgxntool_feature_detect,PGXNTOOL_ENABLE_TEST_BUILD,$(TEST_BUILD_FILES)))
$(eval $(call pgxntool_feature_detect,PGXNTOOL_ENABLE_TEST_INSTALL,$(TEST_INSTALL_SQL_FILES)))
$(eval $(call pgxntool_feature_detect,PGXNTOOL_ENABLE_VERIFY_RESULTS,yes))

Pros: Eliminates all duplication, easy to add new features. Cons: $(eval $(call ...)) is less readable for people unfamiliar with GNU Make metaprogramming. Debugging errors from inside $(eval) can be confusing.

B. Keep inline but simplify — extract just the normalize+validate into a function, keep the auto-detect inline since each feature's detection logic differs slightly:

# Returns normalized yes/no or triggers error
define pgxntool_validate_yesno
$(strip $(if $(filter yes no,$(shell echo "$(1)" | tr '[:upper:]' '[:lower:]')),$(shell echo "$(1)" | tr '[:upper:]' '[:lower:]'),$(error $(2) must be "yes" or "no", got "$(1)")))
endef

Pros: Simpler, auto-detect logic stays visible. Cons: Still some repetition in the ifdef/else structure.

C. Leave as-is — the blocks are short, self-contained, and easy to understand. Each feature's config section is a complete unit you can read top-to-bottom.

I'd lean toward A since we already have three features and may add more. The $(eval $(call)) pattern is well-established in GNU Make. But readability is a real tradeoff — worth considering whether pgxntool users are likely to need to understand this code.

[jnasbyupgrade]

Lets do B

[jnasbyupgrade]

Sounds good — will extract the normalize+validate into a function and keep the auto-detect logic inline.

[jnasbyupgrade]

can't these two lines be combined with 267?

[jnasbyupgrade]

Yes — can combine into TEST_DEPS += install installcheck on a single line. The conditional test-build += stays separate to preserve ordering (testdeps → test-build → install → installcheck).

[jnasbyupgrade]

Need a comment on why we even need to do this

[jnasbyupgrade]

pg_regress aborts with a confusing 'could not open file' error if the expected output file doesn't exist — it doesn't create one automatically. Without this, adding a new .sql file to test/build/ would fail before even running the SQL. Will add a comment explaining this.

[jnasbyupgrade]

s/;/ and/

[jnasbyupgrade]

Will fix.

[jnasbyupgrade]

Why??

[jnasbyupgrade]

The die() function was added as a convenience for combining error output + exit. If it's not used or not pulling its weight, happy to remove it.

[jnasbyupgrade]

No, I mean why are you wrapping $exit_code in ""?

[jnasbyupgrade]

Fair point — exit only takes a numeric argument and $exit_code is always set from $1. The quoting is unnecessary here. Will remove.

[jnasbyupgrade]

Should still be distclean (but I don't think we need a custom variable).

[jnasbyupgrade]

Agreed. Will use standard distclean target directly instead of the PGXNTOOL_distclean variable.

[jnasbyupgrade]

Should still be distclean.

[jnasbyupgrade]

Will fix.

[jnasbyupgrade]

Should still be distclean.

[jnasbyupgrade]

Will fix.

[jnasbyupgrade]

Why's this needed?

[jnasbyupgrade]

Null-glob guard. If no .sql files exist in the directory, the glob *.sql expands to the literal string "$BUILD_DIR/*.sql". The [ -f "$file" ] check catches that and skips. Without it, the cp below would fail with "No such file".

[jnasbyupgrade]

Please add a comment explaining that.

[jnasbyupgrade]

Will do.

[jnasbyupgrade]

Lets do B