Commit History
-
Bump cadgenbench pin to 8ae1432 (revert 0.2.0 edit-baseline rework; back to 0.1.0)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>182f8fc
-
Bump cadgenbench pin to 2abaea8 (cadquery optional; fixes libTKIVtk/VTK clash)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>f4b0f2f -
Pin VTK stack (pyvista 0.48.4 / vtk-osmesa 9.3.1) for reproducible builds
Michael Rabinovich +1: CursorThe last rebuild floated pyvista (pulled transitively by cadgenbench) and the headless vtk-osmesa swap to latest, producing a mismatched pair that failed at import (libvtkWrappingPythonCore3.12.so not found) and broke the Space. Pin both to the known-good versions. Co-authored-by: Cursor <cursoragent@cursor.com>ec67f32
-
Drop mesh-file submissions; STEP-only candidates
Michael Rabinovich +1: CursorRemove the triangle-mesh submission path (and its OpenSCAD framing) from the submit gate, UI copy, and metrics page; candidates are STEP/BREP only. Pin the eval image's cadgenbench to a47c951. Co-authored-by: Cursor <cursoragent@cursor.com>04e4262
-
Bump cadgenbench Docker pin to e7a60e0
Michael Rabinovichb430299
-
Bump cadgenbench Docker pin to 1924090
Michael Rabinovich9c0fafb -
Bump cadgenbench Docker pin to f284fbd
Michael Rabinovich7fa5cee
-
Bump CADGENBENCH_SHA to 037eade; add GT edit-diff report backfill tool
Michael Rabinovich +1: CursorPins the leaderboard image to the cadgenbench commit that renders the GT answer-key edit-diff report column. Adds tools/backfill_report_gt_edit_diff.py, the one-off (idempotent) patcher that swaps existing reports' editing GT columns to the answer-key turntable without re-evaluating. Co-authored-by: Cursor <cursoragent@cursor.com>b885669 -
Bump CADGENBENCH_SHA to ee8b257 (deterministic aligner)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>884294a -
Bump pinned cadgenbench SHA to 0cf5913 (sample-named run_summary keys)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>77afed9 -
bump cadgenbench -> 68e9337 (clamp edit-diff zoom to full view)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>2e44157 -
bump cadgenbench -> 2b1c939 (surface-deviation edit-diff field + Surface Distance F1 rename)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>5b57471
-
leaderboard: axis gizmo on task render grids; bump cadgenbench
Michael Rabinovich +1: Cursortasks.py overlays the shared XYZ orientation gizmo on the editing task's starting-shape render grid (imported from cadgenbench.common.axes_gizmo). Bump CADGENBENCH_SHA to 193f2f5 so the Space's report generator picks up the gizmo + headline-pill changes and tasks.py can import the helper. Co-authored-by: Cursor <cursoragent@cursor.com>1a5a9a2 -
deploy: bump cadgenbench to b76a55a + ship metrics assets
Michael Rabinovich +1: CursorPins the Space to cadgenbench b76a55a (report title with submission name, click-to-zoom images, metric pill deep-links) and copies the new assets/ dir into the image so the Metrics page's mating-group WebP is served by the /metrics-assets route. Co-authored-by: Cursor <cursoragent@cursor.com>0c1ec25
-
deploy: bump cadgenbench to 72f6e82 (scale-relative edit-diff tol, viz polish)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>96dcaf6 -
deploy: bump cadgenbench to 850281d (unified viz colours)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>4bfd457
-
leaderboard: bump cadgenbench to d514d6a (report thumbnail grid)
Michael Rabinovich +1: CursorPins the cadgenbench install to the report-grid commit so newly generated hosted reports render the grouped thumbnail-grid summary view. Excludes the later f9b142b scoring change deliberately. Co-authored-by: Cursor <cursoragent@cursor.com>fd0c525 -
leaderboard: bump cadgenbench to 1010043 + legend backfill tool
Michael Rabinovich +1: CursorPin to the report generator with color-chip interface/edit-diff legends, and add tools/backfill_report_legends.py to retro-patch the corrected legends into already-published reports/<id>.html in place (no re-eval). Co-authored-by: Cursor <cursoragent@cursor.com>1a24d4a -
leaderboard: download-zip button in merge-path report + backfill tool
Michael Rabinovich +1: CursorPass the submission-zip URL to generate_html in the merge path so reassembled reports show the "Download submission ZIP" button, and add a one-off tools/backfill_report_download_button.py that injects the button into already-published reports/<id>.html in place (no re-eval). Bump CADGENBENCH_SHA to 0c7690e. Co-authored-by: Cursor <cursoragent@cursor.com>c48c18f -
leaderboard: serve GT report assets via proxy; link them in hosted report
Michael Rabinovich +1: CursorAdd a /gt/{fixture}/{relpath} proxy route that streams the private GT view PNGs + PDF (token-holding), and pass gt_base_url/input_base_url to generate_html in the merge path so the hosted report links GT/input/overlay (lazy) instead of base64-inlining them. Bump CADGENBENCH_SHA to 25943a0. Co-authored-by: Cursor <cursoragent@cursor.com>5140b0a -
leaderboard: serve renders from the public bucket, not the dataset proxy
Michael Rabinovich +1: CursorCandidate renders (turntables + edit-diff WebP) now live in the public HF Storage Bucket under renders/<id>/, uploaded once by the eval job. The gallery grid and the hosted report reference them by anonymous bucket URL, so the Space is out of the render read path and the binary renders stop bloating the submissions dataset / its commit queue. - leaderboard: HF_RENDER_BUCKET + render URL/path/prefix helpers. - app: render resolvers return public bucket URLs; the two render proxy routes + fetchers are retired. Only the private GT render keeps its token-held proxy. - submit: merge passes render_base_url to generate_html and commits report-only; dispatched job gets CADGENBENCH_RENDER_BUCKET/HF_ENDPOINT. - admin: delete also purges the bucket renders/<id>/ prefix (previously orphaned). - Dockerfile: bump cadgenbench pin to 3d49822 (render_base_url support). Co-authored-by: Cursor <cursoragent@cursor.com>d2161b1
-
submit: allow missing fixture outputs
Michael Rabinovichdc8ff2a -
pin cadgenbench to e446cca (benchmark-v1 aligner + sidecar meshes)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>30909aa
-
docker: cache-bust the cadgenbench @main install
Michael Rabinovich +1: CursorADD the GitHub commits endpoint for the tracked ref before the pip install so BuildKit invalidates the cached layer whenever main moves; otherwise @main would stay pinned to the first build's HEAD. No-op when CADGENBENCH_SHA is a fixed commit SHA. Co-authored-by: Cursor <cursoragent@cursor.com>3909559 -
docker: track cadgenbench @main instead of a pinned SHA
Michael Rabinovich +1: CursorPre-v1 the Space should always rebuild against the latest cadgenbench. Default CADGENBENCH_SHA to `main`; lock to a specific commit SHA at the v1 release for reproducible scores (post-gt-swap.md Stage F). Co-authored-by: Cursor <cursoragent@cursor.com>40dd166 -
docker: stop COPYing the removed results.jsonl
Michael RabinovichThe bundled results.jsonl was git rm'd along with the leaderboard's local fallback, but the Dockerfile still listed it in the final COPY, so the build failed with `"/results.jsonl": not found`. Drop it from the COPY; only the *.py sources are needed now.3e18d9a
-
Dockerfile: swap vtk for vtk-osmesa so off-screen rendering works headless
Michael Rabinovich +1: CursorThe stock pyvista wheel pulls the stock vtk wheel from PyPI, which is built with vtkXOpenGLRenderWindow. That render window class needs an X server even in off_screen mode, so it segfaults the eval worker on the Space's truly-headless container ("DISPLAY=" with no X11 socket). The parent ProcessPoolExecutor sees the segfault as BrokenProcessPool six minutes into the eval, which is exactly the failure mode the first VTK submission hit. vtk-osmesa is the same VTK build but compiled against Mesa OSMesa (pure CPU software rasteriser, statically linked, no display server required). It shares the `import vtk` namespace so pyvista picks it up transparently. This is the canonical PyVista headless-Docker fix (per the PyVista docs + the maintainers' own Docker images). The wheel lives on Kitware's index, not PyPI proper. Co-authored-by: Cursor <cursoragent@cursor.com>545c818 -
Dockerfile: bump CADGENBENCH_SHA to b22a53c (ProcessPool spawn fix)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>fa32cdd -
Dockerfile + submit: swap Chromium for VTK; bump eval workers 1 -> 4
Michael Rabinovich +1: CursorPairs with cadgenbench 3fd213d (in-process VTK / PyVista renderer). Dockerfile drops the playwright install + Chromium browser layer (~200 MB), drops PLAYWRIGHT_BROWSERS_PATH, and adds the Mesa GL runtime libs VTK needs for off-screen rendering on the Space's cpu-upgrade tier (libgl1 was already there; libegl1, libegl-mesa0, libglx-mesa0 are new). CADGENBENCH_SHA bumps to 3fd213d. EVAL_WORKER_COUNT goes from "1" back to "4": the VTK renderer is cheap enough that one ProcessPoolExecutor worker per fixture no longer oversubscribes the host the way the old Chromium subprocess did. Co-authored-by: Cursor <cursoragent@cursor.com>0626a6b
-
Dockerfile: bump CADGENBENCH_SHA to 056cfbb (invalid-candidate fast path)
Michael Rabinovichff674cc
-
Dockerfile: bump CADGENBENCH_SHA to 9bd02fb (lazy baseline import)
Michael RabinovichPicks up the cadgenbench fix that makes `cadgenbench --help` and `cadgenbench evaluate` work in the eval-only install (no `[baseline]` extra). Before this bump, the chunk-4 worker's `python -m cadgenbench.cli evaluate <run_dir>` subprocess died during argparse setup with `ModuleNotFoundError: No module named 'dotenv'`, flipping every submission to status=failed before any work happened. Bumping CADGENBENCH_SHA invalidates the cadgenbench layer in the Dockerfile cache (Chromium layer + earlier layers stay cached), so the rebuild is ~2-3 min not the ~5 min cold cycle.67fd44c -
submit: move handle_submit into its own module + add validation pipeline
Michael RabinovichStep 6 (E) chunk 2 of the async submit work. Lifts the UI-only stub out of app.py into submit.py and grows it into the real cheap-sync validation pipeline. Still no Hub writes and no eval kick-off; the handler returns "would queue here" placeholder text. Async write + worker thread land in later chunks. Validation gates, in order: 1. Form: zip attached, agree-checkbox ticked. 2. Zip safety: valid zip, no absolute / parent-traversing entry names, no symlinks (Python 3.12's extractall normalises unsafe paths silently; we reject so submitters see a clear error). 3. meta.json schema: required keys, types, agree_to_publish is literal true, notes within the 500-char cap. meta.json is the source of truth for submitter / submission identity; the other form fields are advisory for now and kept only so the gr.click signature stays stable. 4. Fixture set: top-level dirs in the zip equal the fixture set reported by cadgenbench.common.paths.data_inputs_di…0501689 -
Dockerfile: bump CADGENBENCH_SHA to ed31274 (parallel evaluate)
Michael RabinovichPicks up the eval orchestrator's switch to ProcessPoolExecutor across fixtures (cadgenbench commit ed31274). On the Space's cpu-upgrade tier (8 vCPU) this should give a near-linear speedup on real-GT-scale runs; locally on the 5-stub set we saw ~2.4x going from --workers 1 to --workers 4 (1m13s -> 31s). The default worker count is min(8, os.cpu_count()) so the Space gets the speedup without setting any env var. Verified locally: per-fixture cad_score outputs are byte-identical between sequential and parallel runs (eval is per-fixture independent + deterministic).91244c6 -
Dockerfile: copy all Python modules, not just app.py
Michael RabinovichPrevious COPY hardcoded app.py only, which broke the moment app.py imported sibling leaderboard.py: the new module never made it into the image and the runtime hit ModuleNotFoundError. Switching to *.py picks up every Python file in AI4Engineering/ (currently app.py + leaderboard.py; submit.py lands next), so future module additions don't need a Dockerfile bump.f26021f -
Dockerfile: bump CADGENBENCH_SHA to 1747493 (Hub-only fixtures)
Michael Rabinovichcadgenbench main now resolves fixtures exclusively from the Hub dataset repos: the wheel-bundled _data/ and the in-repo data/ folder are both gone (commit 1747493). The Space's Settings variables CADGENBENCH_DATA_REPO and CADGENBENCH_DATA_GT_REPO are already set (chunk 5), and HF_TOKEN is the existing Space secret, so this rebuild verifies the end-to-end Hub-only path inside the production runtime.d8207be -
Dockerfile: bump CADGENBENCH_SHA to 36ac6b4 (Hub-aware paths.py)
Michael Rabinovichcadgenbench commit 36ac6b4 adds the Hub-dataset resolution branch to common/paths.py (snapshot_download from $CADGENBENCH_DATA_REPO / $CADGENBENCH_DATA_GT_REPO when set, falling back to the existing local/bundled chain otherwise). The Space's Settings variables CADGENBENCH_DATA_REPO and CADGENBENCH_DATA_GT_REPO are set in lockstep with this bump (HuggingAI4Engineering/cadgenbench-data and HuggingAI4Engineering/cadgenbench-data-gt respectively), so the runtime now resolves fixtures from the Hub dataset repos instead of the wheel-bundled fallback. GT dataset is private; the existing HF_TOKEN Space secret provides read access (already set during Step 5 for the submissions dataset). No new secret required.aa1e886 -
scrub for the cadgenbench Public flip: drop NIST + remove build secret
Michael Rabinovich +1: CursorThree changes bundled together so the next Space rebuild lands in the post-NIST, post-secret end state in one go: - legacy/nist_*_comparison_3d.html (9 MB each) deleted; these were old NIST-stub comparison artifacts already labelled legacy, kept around for visual reference only. Regenerate from the real benchmark dataset post-launch if useful. - results.jsonl emptied. The three baseline seed rows it held were scheduled for pre-launch deletion anyway (per space-setup/migration.md Phase D), and the same rows were just rewritten on the Hub side of cadgenbench-submissions to drop the NIST per-fixture entries. The Space's leaderboard read path uses results.jsonl only as a Hub-fetch fallback; an empty file just produces an empty table on fallback, which is the right behaviour for a development Space. - Dockerfile: dropped the `RUN --mount=type=secret,id=GH_PAT` line and switched the cadgenbench install URL from the authenticated `git+https://michaelr27:$(cat /run/secre…a80542f -
Dockerfile: use actual GH username in classic-PAT URL, not x-access-token
Michael Rabinovich +1: CursorPrevious push tried 'https://x-access-token:TOKEN@github.com/...' (GitHub-Actions placeholder convention). GitHub rejected it: 'Invalid username or token. Password authentication is not supported for Git operations.' Classic PATs over HTTPS Basic Auth need a real GitHub username as the user half of the URL, even though the username is ignored once GitHub validates the PAT as the credential. Switching to 'https://michaelr27:TOKEN@github.com/...' which is the form GitHub's docs themselves recommend. Hardcoding 'michaelr27' is acceptable temporary debt: this whole URL goes away when we either swap GH_PAT for the read-only deploy key (URL becomes git+ssh://) or when huggingface/cadgenbench flips Public (URL drops auth entirely). Co-authored-by: Cursor <cursoragent@cursor.com>12c3a54 -
Dockerfile: use x-access-token:TOKEN URL form for classic PAT
Michael Rabinovich +1: CursorPrevious URL form 'https://$TOKEN@github.com/...' relied on GitHub's special-case where fine-grained PATs are accepted as a single-value username. Classic PATs don't get that special-casing: git parses 'TOKEN@github.com' as username=TOKEN with no password and then prompts for one, which fails in the non-interactive Docker build with 'No such device or address'. Switching to 'https://x-access-token:$TOKEN@github.com/...' (GitHub Actions convention) is the standard explicit-Basic-Auth form and works uniformly for classic + fine-grained PATs, so the URL stays the same when we eventually swap GH_PAT for a deploy key (the URL changes to ssh+git anyway in that case). Co-authored-by: Cursor <cursoragent@cursor.com>818fd28 -
Dockerfile: point cadgenbench install at huggingface/cadgenbench
Michael Rabinovich +1: CursorSwitch source from MichaelRabinovich/LeForge to huggingface/cadgenbench now that the latter has the full history (181 commits, see space-setup/migration.md Phase C). The pinned commit (d7e0468) exists on both remotes so this is URL-only; cadgenbench version bumps are a separate concern handled by ARG CADGENBENCH_SHA. The GH_PAT secret on the Space is being rotated in lockstep from a fine-grained PAT scoped to LeForge to a classic PAT with 'repo' scope. Classic is broader than ideal (gives the PAT read on any private repo the issuing user can see, not just huggingface/cadgenbench) but it's the only self-serve option: the huggingface GH org has fine-grained PATs disabled, and the cadgenbench space-setup user isn't an admin on the org repo so can't add a per-repo deploy key either. Tracked as follow-up debt to swap for a deploy key once admin permissions land (asked Leandro). Co-authored-by: Cursor <cursoragent@cursor.com>aa263ac
-
step 6 (A2): switch space to sdk:docker, install cadgenbench from private LeForge via GH_PAT build secret
Michael Rabinovich +1: CursorThe HF Spaces Gradio SDK can't substitute Space secrets into requirements.txt at build time (only at runtime), and embedding the literal token in requirements.txt is a no-go (would land in git history of the Space). Switching to sdk: docker collapses two open Step 6 sub-decisions into one small change: (A2) private GitHub dep install -- handled by `RUN --mount=type=secret` in the Dockerfile; the PAT (Space secret GH_PAT) is mounted at /run/secrets/GH_PAT for that one RUN and never lands in any layer. (D) Playwright/Chromium for STEP -> PNG rendering -- handled by `playwright install --with-deps chromium` in the same Dockerfile. Layer ordering optimises rebuild speed: OS deps + requirements.txt deps + Chromium come early (stable), cadgenbench from git comes mid (bumps on each commit), app.py + results.jsonl come last (changes most). Front-matter: sdk: gradio -> sdk: docker, drop sdk_version + app_file (Docker SDK uses Dockerfile CMD), add app_port: 7860. .…900c7c1