Sync Model Library to Webflow CMS via Data API#21
Merged
Conversation
Design for modifying fetch-models.js to sync model data directly to Webflow CMS instead of writing models.json. Covers batch diff sync, logo upload via Assets API, and configurable site targeting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix MODULAR_CLOUD_ORG type (Var, not Secret) - Expand Asset API upload to full two-step flow with request shapes - Add batch size limits (100 items max) and chunking note - Clarify publish scope (created + updated only, not deleted) - Add collection slug discovery (models-categories, models) - Define slug handling for models and categories - Fix workflow config: use environment choice input instead of broken dynamic secret lookup - Add image diff strategy (skip logo in comparison) - Note undefined field handling - Add MIME_TO_EXT to removal list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7 tasks covering: schema migration, Webflow API client, transform/diff logic with tests, main script rewrite, workflow update, cleanup, and integration testing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix createItems payload: wrap in {items: [{fieldData}, ...]}
- Fix deleteItems payload: use {itemIds: [...]} not {items: [{id}]}
- Add chunk() utility for >100 item batching
- Add defensive asset URL property check (hostedUrl/url/assetUrl)
- Add missing test cases: name fallback, unknown modalities
- Document cron runs target test site by default
- Add transitional state note for Task 2
- Note about test:unit vs Playwright tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove git commit/push step (no more models.json writes) - Remove contents:write permission (not needed) - Add WEBFLOW_API_TOKEN and WEBFLOW_SITE_ID env vars - Use TEST_WEBFLOW_API_TOKEN secret and TEST_WEBFLOW_SITE_ID var - Rename job from fetch to sync
- Rename workflow to Sync Model Library - Add environment choice input (test/production) with TEST_ and PROD_ prefixed secrets/vars - Add dry run checkbox that shows changes without pushing - Scheduled runs default to test environment
- logo-image -> logo - categories -> modalities - models-categories -> models-category - description: remove <p> wrapping (PlainText, not RichText)
All create/update/delete operations now use /items/live endpoints which publish immediately, eliminating the separate publish step.
Sites that have never been published return 404 on /items/live. The client now probes with a GET request and caches the result per collection. If live isn't available, uses staged endpoints and publishes after create/update.
fetch-models.js: - main() reads as a story: fetch, discover, sync categories, build field data, diff, apply changes - Dry-run gate in one place (main), not scattered across helpers - Renamed: transformModel -> normalizeApiModel, toWebflowFields -> buildWebflowFields, SKIP_DIFF_FIELDS -> FIELDS_MANAGED_OUTSIDE_DIFF - resolveLogo split into isUrl/isBase64DataUri/buildLogoField/ uploadBase64Logo/parseBase64DataUri - Extracted: fetchModels, discoverCollections, fetchExistingItems, buildModelFieldData, applyChanges, logDryRunSummary, logSyncSummary webflow-api.js: - Renamed live/suffix -> useLiveEndpoint/liveSuffix - Documented asset URL fallback
…ow-models-sync # Conflicts: # data/models.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Notes
/items/liveendpoints are available (they 404 on sites that have never been published). Falls back to staged endpoints + publish, with graceful handling if publish also 404s.Changes
Scripts
scripts/fetch-models.jsscripts/webflow-api.jsTests
tests/fetch-models/transform.test.jstests/fetch-models/diff.test.jsWorkflow
.github/workflows/fetch-models.ymlRemoved
data/models.jsondata/images/flux2.svgdata/images/smollm-135m-instruct-fp32.png