74 lines
4.2 KiB
Markdown
74 lines
4.2 KiB
Markdown
# Catalog Studio repository export
|
|
|
|
Studio mutations automatically queue an export to
|
|
`https://gitlab.epicnabbo.nl/remco/Epicnabbo-Catalogus-Updated-Daily.git`, branch
|
|
`Beta-3`. The jobs worker processes pending exports every minute. Streaming
|
|
imports stay active until their stream completes. Server actions for catalog
|
|
pages, offers, deletion and maintenance are covered as well.
|
|
|
|
## Server setup
|
|
|
|
1. Create a **dedicated clean clone** of the repository on `Beta-3`, outside the
|
|
CMS directory. Configure non-interactive Git push authentication for the
|
|
worker OS account using its credential helper. Do not put tokens in URLs.
|
|
2. Set `CATALOG_GIT_CHECKOUT` to that absolute clone directory in the CMS and
|
|
worker environments. Set `CATALOG_GIT_STATE_DIR` to a persistent, writable
|
|
directory outside the clone, shared by both processes on the same host.
|
|
3. Run `pnpm jobs:worker` alongside the CMS under your process supervisor.
|
|
Both processes must have access to the configured asset directories and DB.
|
|
The worker command enables the React server condition for server-only modules.
|
|
4. Restart the CMS after setting the environment. In **Studio → Sync**, use
|
|
**Export now / retry** for the initial export. Subsequent mutations queue
|
|
automatically. Status shows pending/active operations and the last commit.
|
|
|
|
Leaving `CATALOG_GIT_CHECKOUT` empty disables export. No credentials are shipped.
|
|
This source change alone does not configure or deploy the production service.
|
|
|
|
## Exported content
|
|
|
|
| Source | Repository destination |
|
|
| --- | --- |
|
|
| Configured Nitro bundles, including furniture, figures, effects and pets | `Gamedata/bundled` |
|
|
| Configured furni icons | `Gamedata/icons` |
|
|
| Existing badge/catalog images | `Gamedata/c_images` |
|
|
| FurnitureData and supported public game-data JSON files | `Gamedata/config` |
|
|
| Existing localized FurnitureData files | `catalogue version 2 ( Final (Dev)/langs furnidata` |
|
|
| `items_base`, `catalog_pages`, `catalog_items` | `catalogue version 2 ( Final (Dev)/sqls` |
|
|
| `catalog_pages_bc`, `catalog_items_bc`, when present | Same SQL directory |
|
|
|
|
SQL is read in one consistent, read-only InnoDB transaction. Dumps contain table
|
|
definitions and deterministic upserts with hexadecimal UTF-8 string literals.
|
|
Import `items_base.sql`, then `catalog_pages.sql`, then `catalog_items.sql`.
|
|
Existing schemas are not migrated by these dumps. Rows absent from the source
|
|
are omitted; importing an upsert dump into another existing database does not
|
|
delete that database's extra rows. No user, session or credential tables are exported.
|
|
|
|
Only existing local assets are exported. Translation generation follows Studio's
|
|
existing setting; export does not generate missing languages or download assets.
|
|
Public JSON is explicitly allowlisted so translation caches and private runtime
|
|
files cannot enter the repository. Invalid JSON or concurrent Studio changes
|
|
prevent publication of that snapshot.
|
|
|
|
## Failure and concurrency behavior
|
|
|
|
- Pending events survive process restarts; events added during publication remain
|
|
pending for the next run. Partial imports are exported as their settled local
|
|
state, including successful items from a batch containing failures.
|
|
- A single filesystem lock serializes the worker. Dead local process markers are
|
|
recovered on the next run. For an unreadable marker or a marker from another
|
|
host, stop the CMS and worker before repairing the queue directory.
|
|
- A failed push retains the local commit and pending events for retry. Concurrent
|
|
upstream commits are rebased; a conflict aborts the rebase and leaves the event
|
|
pending. Resolve conflicts in the dedicated clone, then retry.
|
|
- The checkout must be clean before each run. Unrelated files are preserved and
|
|
no force push is used. Missing local files do not cause remote deletions.
|
|
- Status errors omit raw Git output to avoid exposing authentication material.
|
|
|
|
## Verification
|
|
|
|
Run the `catalog-git-*` service tests and `src/lib/catalog-export-api.test.ts` with
|
|
Vitest. The integration test creates a temporary bare remote and verifies push,
|
|
failed-push recovery, no-op export and preservation of unrelated files. SQL
|
|
snapshot tests mock the database; production DB import and production push need
|
|
verification in the deployed environment.
|