feat(e2e): lazy app init + playwright sharding for integration tests by jacekradko · Pull Request #8019 · clerk/javascript

jacekradko · 2026-03-10T03:26:58Z

Summary

Refactors integration test app initialization to be lazy (on-demand) instead of eager (all upfront in global setup)
Each app now initializes only when a test first needs it, via test.beforeAll in testAgainstRunningApps
Uses file-based locking to prevent multiple Playwright workers from initializing the same app concurrently
Adds Playwright native sharding support via PLAYWRIGHT_SHARD env var
Splits nextjs integration tests (the CI critical path) into 3 shards per Next.js version

How it works

Before: Global setup starts ALL 13 next.appRouter.* apps in parallel → every test waits for all apps → 8 min total
After: Each shard only initializes the ~4-5 apps its tests actually need → 3 shards run in parallel → ~3-4 min critical path

Lazy init flow

test.beforeAll calls app.init()
init() checks if app is already running (state file + health check)
If not, acquires a file-based lock (atomic wx flag)
Double-checks state file after lock (another worker may have finished)
Performs full init (clerk setup → commit → install → build → serve)
Writes to state file so other workers can adopt

Sharding

Playwright config reads PLAYWRIGHT_SHARD env var (e.g., "1/3")
CI matrix splits nextjs tests into 3 shards per Next.js version (6 total entries)
Each shard only builds the apps its tests actually reference

Test plan

Verify integration tests pass without sharding (backward compatible — PLAYWRIGHT_SHARD unset returns undefined)
Verify sharded nextjs tests pass in CI
Confirm each shard only initializes the apps it needs (check logs for Starting full init vs Adopted from state file)
Verify non-nextjs integration tests are unaffected

Summary by CodeRabbit

Tests
- Enabled test sharding (shard labels included in runs, artifacts, and traces), expanded test matrix, added per-app lazy initialization in beforeAll with a 5-minute init timeout, and updated startup messaging to indicate lazy app init.
Chores
- Added file-based process locking to coordinate inter-worker init, added optional app tokens to persisted state, and exposed shard info to CI test steps.

changeset-bot · 2026-03-10T03:27:02Z

🦋 Changeset detected

Latest commit: 7170adb

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-03-10T03:27:02Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
clerk-js-sandbox	Ready	Preview, Comment	Mar 11, 2026 2:44pm

pkg-pr-new · 2026-03-10T03:28:45Z

Open in StackBlitz

@clerk/agent-toolkit

npm i https://pkg.pr.new/@clerk/agent-toolkit@8019

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@8019

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@8019

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@8019

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@8019

@clerk/dev-cli

npm i https://pkg.pr.new/@clerk/dev-cli@8019

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@8019

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@8019

@clerk/express

npm i https://pkg.pr.new/@clerk/express@8019

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@8019

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@8019

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@8019

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@8019

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@8019

@clerk/react

npm i https://pkg.pr.new/@clerk/react@8019

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@8019

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@8019

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@8019

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@8019

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@8019

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@8019

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@8019

commit: 7170adb

coderabbitai · 2026-03-10T03:31:26Z

📝 Walkthrough

Walkthrough

Adds per-shard test support by propagating PLAYWRIGHT_SHARD through CI matrices, Playwright config, and artifact names. Converts integration apps to lazy initialization with per-app test.beforeAll (300s timeout), server health checks, adopt-from-state-file logic, and state-file writes guarded by a new file-based acquireProcessLock (exported). Extends state file AppParams with optional clerkFapi and clerkTestingToken and adds a .changeset metadata file.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(e2e): lazy app init + playwright sharding for integration tests' directly summarizes the main changes: implementing lazy app initialization and Playwright sharding for integration tests, which are the core objectives described in the PR objectives.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

…rkers Workers that adopt a running app from the state file were missing CLERK_FAPI and CLERK_TESTING_TOKEN env vars because clerkSetup() only runs in the worker that acquires the lock. Store the tokens in the state file and set them in process.env when adopting.

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@integration/models/longRunningApplication.ts`:
- Around line 200-205: The destroy() method must be made safe when init() never
ran: check whether the instance fields pid and appDir are set (or truthy) before
calling awaitableTreekill(pid) or fs.rm(appDir, { recursive: true, force: true
}); if neither is present, simply return immediately; if only pid is present
kill the process, if only appDir is present remove the directory, and handle
errors the same way as current implementation. Update the destroy() function in
the LongRunningApplication class to guard uses of pid and appDir and avoid
passing undefined/null into awaitableTreekill or fs.rm.
- Around line 182-191: acquireProcessLock(id) only prevents concurrent
initializers for the same app but multiple different apps can concurrently
perform read-modify-write on the shared JSON via stateFile.addLongRunningApp, so
wrap all stateFile mutations (e.g., stateFile.addLongRunningApp calls in this
file and the similar calls around lines 220-223) with a process-global
state-file lock to serialize access; add or reuse a helper like
acquireStateFileLock/releaseStateFileLock (or a simple in-process mutex/queue or
OS-level file lock) and call it around the read-modify-write sequence so each
addLongRunningApp runs exclusively, then release the lock after the write
completes.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 741e2985-4098-437b-98d8-b29d485ce7de

📥 Commits

Reviewing files that changed from the base of the PR and between 50a5db7 and 72e025a.

📒 Files selected for processing (2)

integration/models/longRunningApplication.ts
integration/models/stateFile.ts

integration/models/longRunningApplication.ts

- destroy() now checks pid/appDir before calling awaitableTreekill/fs.rm, safely handling cases where init() never ran or failed - Wrap addLongRunningApp with a file lock to prevent concurrent read-modify-write races when multiple apps initialize in parallel

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@integration/models/longRunningApplication.ts`:
- Line 116: parsePublishableKey can return null, so avoid directly destructuring
its result; in the code around the call to parsePublishableKey(publishableKey)
(where you currently extract instanceType and frontendApi), first capture the
return value into a variable (e.g., parsedKey =
parsePublishableKey(publishableKey)), check if parsedKey is null, and handle
that case (throw a clear error or return early) before using
parsedKey.instanceType and parsedKey.frontendApi; update any callers/logic in
longRunningApplication.ts that rely on instanceType/frontendApi accordingly to
ensure a safe path when parsePublishableKey returns null.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: f8bc3ceb-3d83-40e0-b52b-5565a1213bc2

📥 Commits

Reviewing files that changed from the base of the PR and between 72e025a and c877949.

📒 Files selected for processing (1)

integration/models/longRunningApplication.ts

coderabbitai · 2026-03-11T13:48:27Z

integration/models/longRunningApplication.ts

+      const publishableKey = params.env.publicVariables.get('CLERK_PUBLISHABLE_KEY');
+      const secretKey = params.env.privateVariables.get('CLERK_SECRET_KEY');
+      const apiUrl = params.env.privateVariables.get('CLERK_API_URL');
+      const { instanceType, frontendApi: frontendApiUrl } = parsePublishableKey(publishableKey);


⚠️ Potential issue | 🔴 Critical

Handle null return from parsePublishableKey to prevent runtime crash.

parsePublishableKey returns PublishableKey | null. If the publishable key is missing or malformed, this line will throw TypeError: Cannot destructure property 'instanceType' of '...' as it is null.

🐛 Proposed fix

- const { instanceType, frontendApi: frontendApiUrl } = parsePublishableKey(publishableKey); + const parsed = parsePublishableKey(publishableKey); + if (!parsed) { + throw new Error(`Invalid or missing CLERK_PUBLISHABLE_KEY for ${name}`); + } + const { instanceType, frontendApi: frontendApiUrl } = parsed;

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed. In `@integration/models/longRunningApplication.ts` at line 116, parsePublishableKey can return null, so avoid directly destructuring its result; in the code around the call to parsePublishableKey(publishableKey) (where you currently extract instanceType and frontendApi), first capture the return value into a variable (e.g., parsedKey = parsePublishableKey(publishableKey)), check if parsedKey is null, and handle that case (throw a clear error or return early) before using parsedKey.instanceType and parsedKey.frontendApi; update any callers/logic in longRunningApplication.ts that rely on instanceType/frontendApi accordingly to ensure a safe path when parsePublishableKey returns null.

Use optional chaining on app?.teardown() and thisApp?.teardown() to prevent cascading TypeError when beforeAll fails before assigning app.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@integration/tests/handshake.test.ts`:
- Around line 1078-1080: The first "Client handshake `@generic`" test suite still
calls await app.teardown() unguarded in its afterAll; update that afterAll to
check for app before calling teardown (e.g., if (app) await app.teardown()) just
like the other suites, and ensure any associated jwksServer.close() callback is
preserved; locate the suite by the beforeAll that sets up app and the afterAll
that currently calls app.teardown() and add the same null/undefined guard used
elsewhere in this file.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 4bd67bef-63ec-4a58-84b8-d35dc775a20b

📥 Commits

Reviewing files that changed from the base of the PR and between c877949 and 7170adb.

📒 Files selected for processing (1)

integration/tests/handshake.test.ts

coderabbitai · 2026-03-11T14:46:27Z

integration/tests/handshake.test.ts

  test.afterAll('setup local Clerk API mock', async () => {
-    await app.teardown();
+    await app?.teardown();
    return new Promise<void>(resolve => jwksServer.close(() => resolve()));


⚠️ Potential issue | 🟠 Major

The teardown guard is still incomplete in this file.

Lines 1079 and 1463 fix two suites, but the first Client handshake @generic`` suite still does await app.teardown() on Line 70. If its `beforeAll` fails before assigning `app`, this file still throws during `afterAll`, so the PR objective is only partially met. Please apply the same guard there as well.

Also applies to: 1462-1464

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed. In `@integration/tests/handshake.test.ts` around lines 1078 - 1080, The first "Client handshake `@generic`" test suite still calls await app.teardown() unguarded in its afterAll; update that afterAll to check for app before calling teardown (e.g., if (app) await app.teardown()) just like the other suites, and ensure any associated jwksServer.close() callback is preserved; locate the suite by the beforeAll that sets up app and the afterAll that currently calls app.teardown() and add the same null/undefined guard used elsewhere in this file.

jacekradko added 7 commits March 9, 2026 21:39

feat(e2e): add file-based process lock utility

e0d8337

feat(e2e): make longRunningApplication.init() lazy and idempotent

70989c1

feat(e2e): remove eager app init from global setup

92c3c32

feat(e2e): trigger lazy app init from testAgainstRunningApps

c1496d7

feat(e2e): add PLAYWRIGHT_SHARD env var support to playwright config

4e7c45d

ci(repo): add playwright sharding for nextjs integration tests

cbbf693

fix: use correct format placeholder and remove redundant comment

50a5db7

github-actions bot added actions integration labels Mar 10, 2026

vercel bot deployed to Preview March 11, 2026 11:06 View deployment

coderabbitai bot reviewed Mar 11, 2026

View reviewed changes

integration/models/longRunningApplication.ts Outdated Show resolved Hide resolved

integration/models/longRunningApplication.ts Show resolved Hide resolved

vercel bot deployed to Preview March 11, 2026 13:45 View deployment

coderabbitai bot reviewed Mar 11, 2026

View reviewed changes

fix(e2e): guard handshake test teardown when app init fails

7170adb

Use optional chaining on app?.teardown() and thisApp?.teardown() to prevent cascading TypeError when beforeAll fails before assigning app.

vercel bot deployed to Preview March 11, 2026 14:44 View deployment

coderabbitai bot reviewed Mar 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(e2e): lazy app init + playwright sharding for integration tests#8019

feat(e2e): lazy app init + playwright sharding for integration tests#8019
jacekradko wants to merge 10 commits intomainfrom
jacek/lazy-app-init-sharding

jacekradko commented Mar 10, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

changeset-bot bot commented Mar 10, 2026 •

edited

Loading

Uh oh!

vercel bot commented Mar 10, 2026 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Mar 10, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Mar 10, 2026 •

edited

Loading

Walkthrough

❌ Failed checks (1 warning)

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Mar 11, 2026

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jacekradko commented Mar 10, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

Lazy init flow

Sharding

Test plan

Summary by CodeRabbit

Uh oh!

changeset-bot bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

vercel bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

❌ Failed checks (1 warning)

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jacekradko commented Mar 10, 2026 •

edited by coderabbitai bot

Loading

changeset-bot bot commented Mar 10, 2026 •

edited

Loading

vercel bot commented Mar 10, 2026 •

edited

Loading

pkg-pr-new bot commented Mar 10, 2026 •

edited

Loading

coderabbitai bot commented Mar 10, 2026 •

edited

Loading