# Clothes changer model comparison — 2026-10-20

These are real model outputs generated for PhotoGenerAI's clothes changer comparison. The input people are fictional adults from our own sample library. No customer images, private generations or customer prompts were used.

## Inputs and frozen experiment

- [plan.json](./plan.json): six cases, exact user prompts, full adapter-wrapped prompts, model configurations, source paths, input dimensions and SHA-256 hashes. The plan was saved before the first provider call.
- [results.json](./results.json): all 48 attempts, including failures, start/end timestamps, elapsed times, configuration prices in site credits, available provider receipts, actual dimensions and original-output SHA-256 hashes.
- [assessments.json](./assessments.json): one assessment per attempt, with four quality criteria, reference-garment fidelity when applicable, concrete reasons, reviewer/method metadata and the evaluated image hash. No-image attempts are not assessed and stay in the task-pass denominator.
- [Input image provenance](../outfit-changer/PROVENANCE.md): the existing subscribed ImageGen source-photo and garment-reference prompts. Existing marketing after-images were not used as comparison outputs.

The Qwen adapter did not return provider request IDs for these twelve outputs; their `receiptId` values are null. Their original returned files, dimensions, timestamps and SHA-256 hashes are retained. Failed Gateway calls also have no captured receipt ID.

Configurations requested 3:4: `gpt-image-2/low/3:4`, `nano-banana-2/1K/3:4`, `flux-2-pro/1K/3:4`, and `qwen-image-2.1-turbo/2K/3:4`. Underlying models were `openai/gpt-image-2`, `google/gemini-3.1-flash-image`, `bfl/flux-2-pro`, and `qwen-image-2.1-turbo` respectively. This tests available PhotoGenerAI configurations; output resolutions and input preprocessing differ.

## Execution and display

We called the same `generateMakeover` adapter used by PhotoGenerAI, directly from a local research runner. GPT, Nano and FLUX used Vercel AI Gateway; Qwen used the Qianwen endpoint. Calls did not go through customer billing, private history or production database writes. Listed credits are the configuration's tool price on the test date. The planned total was 98 site credits; this is not a customer-wallet charge or a verified provider-dollar total. Failed-request provider charges are unknown.

Each of the 24 case/model cells has two independent attempts from the original inputs. Model order rotates by case. The first attempt is always the main image, including failures. Second attempts are available separately. No output was selected as a replacement for a failed or weaker attempt.

FLUX's existing adapter converts all inputs to JPEG with a maximum 960-pixel long edge. Other models receive the original source WebP bytes. Successful output files are the original returned bytes, without content edits, face correction, sharpening, resizing or re-encoding. Web display uses Next.js image optimization; opening an original accesses the retained file. Detail view applies the same CSS enlargement and position to every output.

## Timing and failures

Elapsed time starts immediately before the adapter call and ends when image bytes return. It includes input preparation and provider response, excluding browser uploads, abuse verification, billing and history storage. Summary medians/ranges include successful requests only; failed attempt times remain in the records. A returned image is not a quality pass.

Two first-run Nano requests (`linen-nano-1` and `reference-nano-1`) waited approximately 271 and 590 seconds before a GatewayResponseError (HTTP 200), without image bytes. After those failures, the runner added a four-minute deadline to remaining requests through the adapter's existing injectable SDK functions. FLUX already had its adapter polling deadline and Qwen its existing request timeout. The production adapter was not changed. Every later failure is retained with its actual error class and elapsed time in results.json; no failed paid call was automatically repeated.

Visual judgments are a single unblinded AI-assisted editorial inspection by PhotoGenerAI (Codex), using all returned full photos and aligned face/clothing crops against the inputs. Pass means no material visible departure at the inspected scale; Partial means a recognizable edit with a visible unmet requirement; Fail means a major missed requirement or substantial unrequested change. A task passes only when all four criteria pass, plus garment fidelity for the reference case. All twelve attempts per model form the denominator. The original proposal suggested blind human review; this release explicitly uses AI-assisted judgments instead, with no independent human rating or universal-winner claim.

The square business and landscape street inputs conflict with fixed portrait output and the prompt's request to preserve framing. That preservation shortfall is marked for every affected model; it is not a model-only causal finding. Judgments remain editorial and can be checked against the retained originals and reasons.

The reproducible runner and verification report live in this repository under `docs/research/sources/2026-10-20-clothes-comparison-run.mts` and `docs/research/2026-10-20-clothes-comparison-implementation.md`.