mirrors/Open-Generative-AI
2
0
Fork 0
mirror of https://github.com/Anil-matcha/Open-Generative-AI.git synced 2026-05-07 01:17:18 +00:00
Code Issues Projects Releases Packages Wiki Activity
118 commits 2 branches 11 tags 761 MiB
Commit graph

4 commits

Author SHA1 Message Date
Anuragp22
24e4bb40e5 feat: background worker, prompt template, section switcher, home hub 
Bundles three changes that were originally landed as stacked PRs into
peer branches and never reached main. Combined here as a single PR
targeting main.

worker/index.js (new)
---------------------
Long-lived Node process that:
- Polls Postgres every 2s for batches in status='running'.
- Claims queued jobs with SELECT ... FOR UPDATE SKIP LOCKED (single
  worker today, ready for N-worker scale tomorrow with no further
  code change).
- Re-uploads local-only trainer images (/api/uploads/...) to MuAPI's
  /upload_file once and caches the resulting CDN URL on the trainer
  row so future jobs reuse it.
- Submits POST /api/v1/<batch.model> with prompt, images_list,
  aspect_ratio, duration, quality.
- Polls /api/v1/predictions/:id/result, extracts videoUrl from
  outputs[0]/url/output.url/video_url on success.
- Backoff: min(10 * 3^retries, 300)s. After retries >= 3, status=
  'failed', waiting for manual Retry from the UI.
- Recovery on boot: re-queues anything left in 'submitting' from a
  crashed run; 'polling' jobs keep their muapiRequestId.

Configuration: MUAPI_API_KEY (required to do work), MUAPI_BASE_URL
(default https://api.muapi.ai), WORKER_TICK_MS (default 2000),
UPLOAD_DIR (default /data/uploads).

worker/package.json: { "type": "module" } so plain Node 20 runs the
ESM imports.

Dockerfile: copy /app/worker into the runner stage.

docker-compose.override.yml: replace placeholder with the real
'npx prisma generate && node worker/index.js'.

lib/promptTemplate.js (new) + worker integration
------------------------------------------------
renderPrompt({trainer, studio, job}) wraps each row's raw practice
description in a fixed narrative template that pins identity,
biomechanics, environment, lighting, clothing, expression, video
style, and a list of forbidden behaviours (no camera movement, no
cuts, no limb warping, no pose distortion, etc).

duration, aspect_ratio, and quality remain SEPARATE fields in the
MuAPI request body; they are never injected into the prompt text.

lib/csvParser.js: stops appending 'Start position: ... Camera: ...'
into the stored prompt. Just stores the raw practice description;
startPosition + cameraAngle remain available as their own columns
for the renderer to consume.

components/SectionSwitcher.jsx (new)
------------------------------------
Two-pill switcher (Studio · Batch) rendered in the top-left of every
shell. Active pill is yellow on black. Click writes the choice to
localStorage so the home hub can highlight 'Last used'.

StandaloneShell, BatchShell, BatchDetail headers all updated to host
the switcher next to the logo. The redundant '← Studio' link in
BatchShell's right-side actions is removed since the switcher
replaces it.

app/page.js + components/HomeHub.jsx
------------------------------------
/ used to redirect to /studio. Now serves a hub with two cards:
Batch (CSV-driven automation) and Studio (one-off generations).
The card the user last opened gets a yellow accent + 'Last used'
tag from localStorage.

Stub-package copy
-----------------
packages/ai-agent and packages/workflow-ui description strings and
runtime 'feature unavailable' notices: removed organization-specific
phrasing in favour of generic 'this fork' wording.
 2026-04-23 09:41:55 +05:30
Anuragp22
4cf0d503b8 feat(batch): batch detail page with progress, controls, retry, CSV export 
Slice 5 of the batch feature. Marketing team can now click into a
batch from the list, watch a live progress dashboard, drive it with
Start/Pause/Resume/Cancel, retry individual failed jobs, and pull a
results CSV when it's done.

Includes a stakeholder-demo helper that simulates worker progress
(slice 4 hasn't shipped yet) so non-technical viewers see a realistic
running state without needing real MuAPI generations.

Backend
-------
- POST /api/batches/[id]/start: draft|paused -> running, requeues
  any cancelled jobs.
- POST /api/batches/[id]/pause: running -> paused.
- POST /api/batches/[id]/resume: paused -> running.
- POST /api/batches/[id]/cancel: any -> cancelled, marks queued/draft
  jobs as cancelled (in-flight ones complete naturally).
- POST /api/batches/[id]/simulate: demo-only. Picks ~10% of queued
  jobs, marks ~90% of those done with a placeholder video URL and
  ~10% failed. Updates batch counters atomically.
- GET /api/batches/[id]/export: streams a CSV with row-index,
  practice, trainer, studio, status, video URL, error, retries.
  Properly quotes cells, slugifies the filename.
- POST /api/jobs/[id]/retry: resets a failed job to queued so the
  worker re-attempts it.

UI
--
- app/batch/[id]/page.js: leaf route mounting BatchDetailShell.
- components/batch/BatchDetailShell.jsx: same MuAPI-key gate as
  BatchShell so opening a deep link works in a fresh browser.
- components/batch/BatchDetail.jsx: full progress dashboard.
  - Top header: back link, batch name, live status pill, CSV
    download button.
  - Progress card: done/total numbers, breakdown line
    (failed / running / queued), animated yellow bar, "auto-refresh
    every 3s" indicator while the batch is running or paused.
  - Settings strip: model / duration / quality / aspect / concurrency.
  - Filter pills: All | Pending | Done | Failed.
  - Demo button: "⚡ Simulate progress (demo)".
  - Jobs table: row index, practice (with inline error message),
    trainer, studio, status pill, ▶ Play (opens video preview modal),
    Retry (only on failed rows).
  - 3s polling driven by the running/paused state.
- components/batch/BatchesTab.jsx: rows are now clickable, the
  wizard navigates straight to the new batch's detail page after
  saving so the demo flow is one continuous click-through.

Verification
------------
- Created a 5-row batch via the API.
- start -> status went to running.
- simulate -> 1 done with placeholder URL, 0 failed (10% of 5 = 0
  failed by design).
- GET detail returned the batch with jobs joined to trainer/studio.
- /batch/[id] page returned 200.
- CSV export returned text/csv with the right columns and rows.
 2026-04-23 08:48:42 +05:30
Anuragp22
694c7075e2 feat(batch): New Batch wizard, CSV parser, and batch list 
Slice 3 of the batch feature. Marketing team can now upload the
Rasika-style CSV, watch each row auto-map to the trainer/studio
they uploaded in slice 2, get a real MuAPI cost estimate, and
persist the batch as a draft.

Backend
-------
- POST /api/batches: creates a batch in status='draft' along with one
  job per CSV row (status='queued', not yet picked up since the
  worker arrives in slice 4).
- GET /api/batches: list view payload.
- GET /api/batches/[id]: detail with jobs joined to trainer/studio
  rows. Used by the slice-5 progress UI.
- POST /api/batches/[id]/estimate-cost: forwards the batch's model +
  payload to MuAPI's /api/v1/app/calculate_dynamic_cost, multiplies
  by row count, returns {perJob, total, currency}.

CSV
---
- lib/csvParser.js: PapaParse-based parser that validates required
  columns, normalises duration ("15 sec" -> 15) snapped to Seedance
  2.0's [5,10,15], maps quality ("1080P" -> "high"), and composes a
  prompt from description + start position + camera angle.

UI
--
- components/batch/NewBatchWizard.jsx: full-screen 3-step wizard.
  - Step 1 — name, CSV upload, model/duration/quality/aspect/
    concurrency settings.
  - Step 2 — review every row in a table. Auto-maps Character ->
    Trainer.csvLabel and Studio -> Studio.csvLabel, shows per-row
    override dropdowns and bulk-assign helpers, hard-blocks Next
    until every active row has both a trainer and studio.
  - Step 3 — saves the batch and triggers the cost estimate. On
    success, surfaces the batch id and points the user to the
    upcoming worker slice.
- components/batch/BatchesTab.jsx: replaces the placeholder with a
  real list view (status pill, progress counts, model, created-at)
  and the "+ New batch" button that opens the wizard.
- components/batch/BatchShell.jsx: pass apiKey to BatchesTab.

Verification
------------
- POST /api/batches with one row -> 201, batch + job persisted in
  Postgres (verified with psql).
- GET /api/batches lists the row.
- Cost estimate endpoint returns 401 without a key, calls MuAPI
  correctly with one (live test pending real credits).
 2026-04-23 08:01:15 +05:30
Anuragp22
1b12df90ef feat(batch): trainers + studios CRUD with /batch shell UI 
Slice 2 of the batch feature. Marketing team can now upload the six
trainer images and the studio image once and reuse them across every
future batch.

Backend
-------
- lib/muapiUpload.js: Node-side wrapper around POST
  /api/v1/upload_file with FormData + x-api-key. Mirrors
  packages/studio/src/muapi.js:131-178 but uses native fetch +
  FormData (no XHR).
- lib/batchAuth.js: shared getApiKey() — header > cookie > env fallback.
- lib/localUploadStore.js: writes a local backup of every uploaded
  asset to /data/uploads/{trainers,studios}/<id>.<ext> on the
  uploads_data volume, so we can re-upload to MuAPI if their CDN
  ever expires the URL.
- app/api/trainers/route.js + [id]/route.js: GET list, POST multipart
  (uploads to MuAPI then persists), DELETE (refuses if any active job
  references the row).
- app/api/studios/route.js + [id]/route.js: identical shape.

UI
--
- app/batch/page.js: leaf route mounting BatchShell.
- components/batch/BatchShell.jsx: 3-tab dark-theme shell
  (Batches / Trainers / Studios) with the same MuAPI key gate as
  StandaloneShell (reuses ApiKeyModal + the muapi_key cookie).
- components/batch/AssetLibrary.jsx + AddAssetModal.jsx: shared grid +
  upload modal driving both tabs from one component.
- components/batch/TrainersTab.jsx, StudiosTab.jsx: thin wrappers.
- components/batch/BatchesTab.jsx: placeholder for slice 3.

Stub packages (upstream submodules unavailable)
-----------------------------------------------
The ai-agent and workflow-ui submodules referenced in .gitmodules
return 404 (Anil-matcha/workflow-ui and jaiprasad04/ai-agent are
deleted/private). next build couldn't resolve their imports.

- Removed the dead .gitmodules entries.
- Added local stubs at packages/ai-agent and packages/workflow-ui
  that export no-op components rendering "feature unavailable" so
  the build succeeds. The /agents/* and Workflows tab show that
  notice; the studios our marketing team actually uses
  (Image / Video / Lip Sync / Cinema) keep working since they
  live in the studio package which we have.

Docker / dev workflow
---------------------
- Dockerfile: run prisma generate during the builder stage, copy
  prisma/ and lib/ into the runner stage so migrations and shared
  helpers are present at runtime.
- docker-compose.override.yml (new): dev-mode overrides — mounts
  source, runs as root to dodge node_modules permission issues with
  the prod nextjs user, runs prisma generate + migrate deploy +
  next dev. Worker container is a placeholder until slice 4 lands.
- The full stack (postgres + web + worker) now boots with
  `docker compose up -d` and serves /batch on http://localhost:3000.
 2026-04-23 07:32:49 +05:30