Absorption v2 — "It becomes you"

Proposal · 2026-08-29 · written by Claude Fable 5 (plan/creative), build targeted at Claude Opus 5

App: https://clone.arnao.ai · source /Users/byronarnao/clone-mvp · worker /Users/byronarnao/clone-absorb

---

0. What happened today (already live)

Smoke-test artifact: [byron-talkinghead-quality.mp4](assets/byron-talkinghead-quality.mp4) (also at file:///Users/byronarnao/clone-smoketest/byron-talkinghead-quality.mp4)

---

1. Critique of the current app (honest)

The pipeline is better-built than it looks — it worked end-to-end exactly once (Aug 7: real Nano Banana avatar, real Chatterbox hello, 58.9s, ~$0.13) and then died of operational neglect, not design flaws. The real defects:

1. **The interview feels endless but isn't.** It's a hardcoded array of 10 questions with an early exit after Q5 — but the UI never shows progress, so it reads as infinite. A counter is not enough; the fix is making progress visceral (§2.3).
2. Voice recording starts at SCAN with zero indicator. The recorder runs through scan, capture, and interview; the only "listening…" dot appears mid-interview and is wired to the wrong thing (speech recognition, not the recorder). Byron noticed this cold — it's the app's biggest trust defect.
3. No face focus. MediaPipe FaceMesh is already tracking 468 landmarks, and none of that is used to tell the user the face is what's being absorbed. The background stays fully visible.
4. The endpoint is anticlimactic. "Absorbed." + a static avatar. The clone never speaks to you in your voice as the finale — the one beat that would prove the premise.
5. Silent twin. Twin replies are text-only; the older /c/:id surface speaks every reply, the flagship absorption surface speaks never.
6. Voice floor unenforced. The 18–20s clean-speech minimum was designed and never wired; the Aug 7 run cloned from 6.0s of audio. Explains "not great" voice quality.
7. Fidelity dial under-claims. 4 rungs shown, 5 designed; "Reactive (mirrors your movement)" is implemented and working but never surfaced.
8. Operational fragility. The worker dies on every reboot (TCC blocks cron/launchd) and nothing notices. This is what actually killed the app for 3 weeks.
9. Hygiene: DEV_LOGIN_CODE backdoor still set in prod (its own header says delete it); README 4 weeks stale; api/avatar.js dead code.

---

2. The experience: walking up to a mirror that takes you in

Design target: phone-first, one hand, zero wasted pixels. Every stage is one full-bleed screen with at most one instruction and one action.

2.1 Face focus — "it's the face being absorbed"

The moment FaceMesh locks (landmarks already streaming):

• Background falls away: the camera feed outside the face oval desaturates and blurs (canvas composite: draw frame → overlay a radial mask from the landmark hull → filter: blur(14px) saturate(0.3) outside, crisp inside). Cost ≈ one extra canvas draw; no new deps.
• The existing amber particle motes keep targeting landmarks — but now they visibly pull from the darkened edges toward the face, so the read is "the face is being taken."
• Lock/loss states: face lost >1s → background gently returns, ring dims. Re-lock → collapse again. The world literally closes in around you.

2.2 Recording honesty — REC from the first frame

• A persistent ● REC voice + face chip (top edge, safe-area aware) appears the instant MediaRecorder starts at SCAN and stays until synthesis. Live 5-bar waveform driven by an AnalyserNode — real mic energy, not decoration.
• The DISCLOSE beat gains one line, before the camera ever opens: "This won't just take your picture. It learns your voice, your mannerisms, the way you decide things. That's what a clone is." — consent that matches what absorption actually captures.
• Voice floor enforced: synthesis blocked (gently — "keep talking, I need 20 clean seconds of you") until ≥18s of speech energy is banked. Kills the 6-second-reference problem at the root.

2.3 A finite, visible interview — 8 questions, adaptive down to 6

Grounded in Stanford's generative-agents result: a single finite structured interview reproduced people's real answers at ~85% of their own test–retest consistency — beating trait quizzes and data dumps (Stanford HAI, genagents). Endless questions aren't just annoying, they're scientifically unnecessary.

• 8 base questions, hard-capped, shown as 4 / 8 — plus one Flash-classified follow-up max per answer (the contract's original design, never built).
• Rich answers shorten the set to 6 ("You've given me more than most people do in twenty. Two more.").
• Absorption % is the hero metric, not the question count: each answer, each banked second of voice, each captured expression visibly fills the same single number. The interview reads as feeding, not form-filling.

2.4 Staged encoding — the clone finishes cooking while you're still talking

Today everything renders after the interview. Instead:

• Q1 is the voice anchor: "Say your full name, then tell me something you'd never put on a resume." Name + free speech = clean reference seed. At ~40% absorption (≈ Q4), the client makes an early third call to absorb-create shipping voice-so-far → the Mini starts Chatterbox encode + FasterLivePortrait render in parallel with the remaining questions (the two-phase job API already supports this shape; add one stage:"voice-early").
• The endpoint beat: on the final question, the twin interrupts — speaking, in your cloned voice, through your moving face — answering that last question the way you would have. That is the finish line: not "Absorbed." but you, talking back. To make the timing real: the first line is capped at ≤6s of audio (<40s local render, proven today), and if it's still not ready a designed 45s holding beat plays (reactive avatar mirrors you, "finding your voice…") before degrading honestly to text-first with a live mid-conversation upgrade (§8.1–8.2).
• The existing post-handoff upgrade poll stays as the safety net: if the Mini is slow, the twin appears in text and upgrades to face+voice mid-conversation — which is its own kind of uncanny-good.

2.5 Talking head everywhere (proven today)

FasterLivePortrait-MLX (repo, MIT, Apple-native) replaces the Wav2Lip path in ~/avatar and joins the worker: every twin reply can be a ~5×-real-time rendered clip of the user's own face speaking. Fidelity ladder becomes honest and complete: Text → Voice → Reactive → Talking head → Live face (HeyGen seam stays as the paid "Live" rung until local realtime matures).

---

3. Mirror — the killer test app

mirror mode: you, talking to yourself. Split screen: live camera (you) on the left, your clone's talking head on the right. You ask anything; both of you answer — you out loud, it in your voice a beat later. The product demo, the QA harness, and the philosophical gut-punch are the same feature.

• Turn-based today (proven): your question → clone's rendered reply clip in ~30–60s. Fill the gap with the Reactive avatar mirroring your movements (already implemented, currently unclaimed).
• Live puppet (experimental): FasterLivePortrait's realtime webcam mode runs ~6 fps on the M4 Pro — deliberately frame it as "the mirror is waking up" aesthetic, not broken video.
• Fidelity score: after each exchange, one tap — "was that you?" Yes/No/Almost. That single signal becomes the clone's training loss, logged per-persona. Mirror sessions are how a clone earns its fidelity rungs.

4. Deploy pieces of yourself — the agent layer

The absorbed bundle (persona + voice + face + mannerisms) becomes a deployable unit. Three use cases, strongest first:

1. Stand-in: "send my clone to this meeting." Google Meet's official Media API is receive-only (docs) — so presence = a browser bot with fake mic/cam, which is exactly what open-source joinly (MIT) does today: joins Meet/Zoom/Teams, listens, and speaks through any OpenAI-compatible TTS endpoint — our Chatterbox slots straight in. Phase 1 (buildable now): audio-only stand-in with your voice + portrait tile. Phase 2: idle-loop fake-webcam video. Phase 3: real-time lip-sync via a rented GPU running LiveTalking (~$0.34/hr on RunPod). UI: a "Deploy" sheet on the twin — paste a Meet link, set a brief ("listen, answer questions about my SPIRES work, take notes, never commit me to anything"), get a transcript + what-I-said report after.
2. Direct Line: the clone answers the front door. Visitors to clone.arnao.ai meet your talking head live — recruiters ask real questions, it answers from persona + your public record, and escalates the good ones to you via Signal. Ties directly into Career Triage and the brand front door. This is the always-on deployment that costs pennies (metered at $1/visitor).
3. The Double: your clone reads your dailies. The Signal AM podcast, delivered as a 60-second talking-head clip of you — shareable, on-brand, and generated from a pipeline that already exists end-to-end on the Mini.

5. Premium model routing (OpenRouter — live key in prod)

Per the fresh model research (see refreshed https://benchmarks.arnao.ai):

6. Platform plumbing (mentor patterns, copied faithfully)

• Feedback FAB (🐞 Bug · 💡 Idea · 🤔 Confusing · ❤️ Love + "point at what you mean" element picker) → feedback/<id>.json on Blob + notify.arnao.ai event per batch — mentor's www/api/feedback.js copied, minus its mistake of hardcoding the shared secret in the client bundle (ours comes from /api/config).
• Analytics: Plausible ✅ (done today) + the fleet beacon snippet → fleet-analytics hits/ so clone joins the unified stats pane.
• **Gia dailies (so Byron hears it): nothing auto-discovers a new app — four explicit edits: projects/clone-app/VERSION.md in the workspace, a cat line in scripts/brief/prefetch-brief.sh, a loader in assemble-brief.py, and a source line + AM-structure mention in skills/signal-daily-podcast/SKILL.md §1/§3. Traffic + new feedback + shipped features then read out in the AM/PM episodes exactly like mentor's. Plus: clone-absorb worker liveness → fleet-health.json** (the Aug 22 silent death becomes a headline the next morning, never a 3-week mystery).
• CI/CD: git init + Gitea remote + the vercel-deploy skill as the single deploy path + product-ops contract repo (byron/product-clone) for releases/changelog; version bump rule in VERSION.md. (True push-triggered CI needs Gitea Actions — flagged as optional, the fleet standard is skill-driven deploys.)

7. Build plan (Opus 5 executes; Fable has done the creative)

Open decisions for Byron

1. Remove DEV_LOGIN_CODE from prod? (its own header says yes; Google auth is live; I left it untouched)
2. Cloud-GPU premium rung: fal.ai OmniHuman photoreal at $0.16/sec of video — offer as paid tier, or stay 100% local?
3. Chatterbox → Turbo + Multilingual v3 upgrade (same MIT family, streaming-first) — test next to current voice before swapping?
4. Meet stand-in: green-light P3 now or after Mirror ships?
5. Anthropic API credit top-up — the fleet key is out of credits (blocks fable-critique skill, Gia Signal synthesis on Fable, and any API-side Claude call). One tap: https://console.anthropic.com/settings/billing

8. Self-critique — Fable adversarial pass

(Run in-session by Fable 5. The API-based critique pass is blocked: the Anthropic API key's credit balance is too low — same unblock flagged Aug 2. Top-up restores the fleet's Fable pipeline, including this skill and Gia's Signal synthesis.)

What the critique caught, now integrated:

1. The finale beat was over-promised. "Voice ~60s + face ~50s fits inside 3–4 questions" fails whenever answers come fast. Fixes now in §2.4: the first line is capped at ≤6s of audio (guarantees <40s render), and if the clone still isn't ready at the last answer, a designed 45-second holding beat plays — the reactive avatar mirrors you while "finding your voice…" — then degrades honestly to text-first with the voice arriving mid-conversation. The refusal to substitute generic TTS stands.
2. Definitive call — the wow moment must never depend on the Mini. Resolved: never block, degrade honestly, upgrade live. Concretely: the worker writes a heartbeat blob every cycle; absorb-status exposes it; the UI distinguishes "still cooking" from "the forge is cold" (today both render as fake patience); worker-down becomes a fleet-health headline in the next AM podcast. The Aug 22 three-week silent death becomes structurally impossible to repeat.
3. The $1 cap under-meters by ~2×. Opus 5 at $5/$25 with the new tokenizer's ~30% overhead ≈ 1.7¢ per real chat turn vs the 1¢ flat charge; absorption's true cost was 13¢ vs 8¢ charged. P0 now includes COST.chat: 2, COST.absorb: 15 — so the cap means what it says.
4. Phone performance honesty. FaceMesh + particles + waveform + a per-frame blur composite is a lot for mid-range phones: the blur mask renders on a half-resolution canvas, and the whole face-focus effect degrades to a simple vignette below 24fps. Typed-answer fallback stays first-class (mobile SpeechRecognition is unreliable, especially iOS).
5. Scope discipline. Mirror is the point (it's the QA loop and the demo). Meet stand-in is the scope-creep risk — it stays strictly last (P3b), behind Direct Line, and ships audio-only first.

Bold idea the draft missed — the Clone Card: the absorption finale mints a shareable 10-second vertical clip — your talking head, saying one line you type — behind a share slug + QR (voice.arnao.ai's library pattern, reused). Every absorbed user becomes distribution. Cost: one extra render (~$0, local). Added to P1 alongside Mirror.

9. Research receipts

Avatar: FasterLivePortrait-MLX · FLOAT (better quality, CC-NC — non-commercial only) · SadTalker (legacy fallback) · SOTA-but-CUDA: EchoMimicV3, HunyuanVideo-Avatar, InfiniteTalk · cloud: fal OmniHuman, RunPod 4090 $0.34/hr

Voice: Chatterbox Turbo/v3 · Kyutai Pocket TTS (real-time streaming on CPU — the live-conversation engine) · Qwen3-TTS (quality challenger) · license landmines: F5-TTS, Fish/OpenAudio, IndexTTS-2 (non-commercial / gated)

Twin architecture: Second-Me (hierarchical memory, Apache) · WeClone (chat-log LoRA for surface style; AGPL) · clonellm (MIT RAG+trait skeleton) · genagents (the finite-interview science)

Meet presence: joinly · Attendee · Meet Media API is receive-only

---

Links