Skip to content

Add OpenProcessing example synchronization tooling - #1577

Open
Yisakor-Mirany wants to merge 3 commits into
processing:mainfrom
Yisakor-Mirany:issue-1445-openprocessing-sync
Open

Add OpenProcessing example synchronization tooling#1577
Yisakor-Mirany wants to merge 3 commits into
processing:mainfrom
Yisakor-Mirany:issue-1445-openprocessing-sync

Conversation

@Yisakor-Mirany

Copy link
Copy Markdown

Summary

This PR begins implementing #1445 by adding tooling to synchronize p5.js website examples with an OpenProcessing curation.

The current implementation loads the English examples from the p5.js website, prepares their code and attribution data, connects to the OpenProcessing API, and compares the website examples against a configured OpenProcessing curation.

What changed

  • Added an OpenProcessing API client.
  • Added a script for comparing p5.js examples with an OpenProcessing curation.
  • Added reusable example attribution utilities.
  • Added attribution comments to prepared sketch code.
  • Added support for OPENPROCESSING_TOKEN and OPENPROCESSING_CURATION_ID environment variables.
  • Added npm run sync:openprocessing.
  • Added tests for OpenProcessing configuration and example attribution.
  • Kept API credentials outside of the source code.

Current behavior

The sync command currently performs a safe dry run:

npm run sync:openprocessing

It:

  1. Loads the English p5.js examples.
  2. Reads their metadata and sketch code.
  3. Adds attribution information to the prepared sketch code.
  4. Fetches the configured OpenProcessing curation.
  5. Reports examples missing from the curation and sketches that no longer correspond to website examples.

No OpenProcessing sketches are modified during the dry run.

Testing

Full test suite:

Test Files  13 passed (13)
Tests       64 passed (64)

Project checks:

0 errors
0 warnings

I also tested the sync script against a separate OpenProcessing test curation.

Screenshots

OpenProcessing sync dry run

Screenshot 2026-08-13 at 20 10 36

Automated tests

Screenshot 2026-08-13 at 20 11 21

Configuration

The integration expects the following environment variables:

OPENPROCESSING_TOKEN
OPENPROCESSING_CURATION_ID

No API credentials are committed to the repository.

Remaining work / feedback requested

The current PR establishes the synchronization, comparison, API, and attribution foundation.

I would especially appreciate guidance on the intended OpenProcessing API endpoint/payload for creating or updating the actual sketches before adding them to the curation by visualID.

Once that flow is confirmed, the synchronization can be extended to create/update missing sketches and remove outdated entries from the curation.

Related to #1445

@Nwakaego-Ego

Copy link
Copy Markdown
Contributor

Hi @Yisakor-Mirany Thank you for your work so far on this PR. While testing "npm run sync:openprocessing" on Windows, I encountered the following error: ENOENT: no such file or directory, open ...\p5js-website-2.0\code.js
The paths returned from "findDescriptionFiles()" contain Windows "" separators, while "lastIndexOf("/")" searches for "/". This returns "-1" and causes "codePath" to point to "code.js" at the repository root. Would you be open to using "dirname(descriptionPath)" with "join()" here so Node can handle the path separator for each operating system? It may also be helpful to add a test covering a Windows-style path. Let me know what you think.

I also checked the attribution requirement from #1445. The new exampleAttribution.ts utility and the existing src/layouts/ExampleLayout.astro file each work out the authors, collective attribution year, and licence separately. This means the same attribution rules are still written in two places.

Since #1445 asks for these rules to be defined once, could they be moved into one shared function that is used by both the website layout and the OpenProcessing sketch comments? Thank you.

@Yisakor-Mirany

Copy link
Copy Markdown
Author

Hi @Nwakaego-Ego . Thank you for the detailed feedback! I’ve addressed both points. I updated the path handling to use dirname() and join() so it works correctly across operating systems and added test coverage for Windows-style paths. I also refactored the attribution logic so the shared rules are defined in one place and reused by both the website layout and OpenProcessing synchronization.

I pushed the updates to the PR, and all tests/checks are now passing. Please let me know if there’s anything else you’d like me to adjust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants