JEPA and the Non-Generative Counterargument

The fourth paradigm in the taxonomy rejects the premise of the previous one. Instead of predicting future pixels, predict the representation of the future: train an encoder and a predictor jointly so that the predictor, given the current embedding and an action, matches the embedding a target encoder produces on the true future. No pixel reconstruction appears anywhere, at training time or at planning time. This is the Joint-Embedding Predictive Architecture, JEPA, and it is the principal architectural alternative to generative world models. It is also a position with an institution behind it: Yann LeCun's AMI Labs.

The argument: predict representations, not pixels

The case against generative pixel prediction, stated as strongly as it can be:

1. Most of the information in a video frame is irrelevant to control: leaf texture, exact sensor noise, the precise fold of a sleeve. A generative objective forces the model to spend capacity modeling all of it.
2. The future is genuinely multimodal and partly unpredictable. A pixel-space model must either average over the possibilities, producing blur, or sample, committing to arbitrary detail. Predicting in representation space lets the model discard what it cannot predict, because the target is an embedding, not a frame.
3. Planning wants an abstract state, not an image. If you have to decode to pixels and re-encode to plan, the decoder is pure overhead.

The original V-JEPA showed the recipe works as a representation learner: a context encoder, an exponential-moving-average target encoder, and a predictor, trained by feature prediction alone on 2 million videos, with no reconstruction, no negative examples, no text, and no pretrained image encoder. The frozen backbone then performed well on both motion and appearance tasks without any adaptation of its parameters Bardes 2024Revisiting Feature Prediction for Learning Visual Representations from VideoAdrien Bardes, Quentin Garrido, Jean Ponce et al., 2024.

V-JEPA 2: the argument's strongest artifact

V-JEPA 2 turns the representation into a controller, in two stages Assran 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMido Assran, Adrien Bardes, David Fan et al., 2025.

First, action-free pretraining: self-supervised masked latent prediction on over 1 million hours of internet video, plus images, at roughly 1B parameters, using mask denoising and 3D rotary position embeddings. This alone sets the state of the art on motion understanding, 77.3 top-1 on Something-Something v2, with strong human-action anticipation on Epic-Kitchens-100, and the same encoder serves a VLM competitively on video question answering Assran 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMido Assran, Adrien Bardes, David Fan et al., 2025.

Second, action-conditioned post-training, producing V-JEPA 2-AC: freeze the encoder and train an action-conditioned predictor on less than 62 hours of unlabeled robot video from the Droid dataset, with no rewards, no task-specific demonstrations, and no per-environment tuning. The result plans zero-shot on Franka arms in labs never seen during training. Control is energy minimization in latent space: specify a goal image, encode it once, search action sequences whose predicted final latent lands closest to the goal latent, execute the first action, observe, and replan. Reported tasks are reaching, grasping, and pick-and-place Assran 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMido Assran, Adrien Bardes, David Fan et al., 2025.

The interactive below runs that loop in a 2-D projection of the embedding space. The goal image is a fixed point, the amber marker is the current latent, and each Plan step is one model-predictive-control iteration: the candidate fan is the searched action sequences, scored by predicted distance to the goal latent, and the readout contracts as steps execute. A larger search budget aligns the plan better with the true goal direction, so it reaches the goal in fewer steps. The predictor is imperfect on purpose, which is why the executed path wobbles off the planned one.

One follow-up makes the distance-as-value reading explicit: shaping the representation space during training so that embedding distance approximates the negative goal-conditioned value function significantly improves planning performance over standard JEPA models on simple control tasks Destrade 2026Value-guided action planning with JEPA world modelsMatthieu Destrade, Oumayma Bounou, Quentin Le Lidec et al., 2026.

The institutional bet: AMI Labs

The argument is no longer only a research position. LeCun left Meta in November 2025 and co-founded AMI Labs (Advanced Machine Intelligence), which raised $1.03 billion at a $3.5 billion pre-money valuation in March 2026, one of the largest seed rounds on record, explicitly to build world models rather than language models Heim 2026Yann LeCun's AMI Labs raises $1.03B to build world modelsAnna Heim, TechCrunch, 2026. CEO Alexandre LeBrun's own framing to TechCrunch was blunt: "'world models' will be the next buzzword. In six months, every company will call itself a world model to raise funding" Heim 2026Yann LeCun's AMI Labs raises $1.03B to build world modelsAnna Heim, TechCrunch, 2026.

The counterargument, stated fairly

1. Every currently usable interactive simulator is generative. Genie 3, Cosmos 3, Odyssey-2, and the Interactive World Simulator all render pixels you can inspect and steer Parker-Holder 2025Genie 3: A new frontier for world modelsJack Parker-Holder, Shlomi Fruchter, Google DeepMind, 2025NVIDIA 2026Cosmos 3: Omnimodal World Models for Physical AINVIDIA, 2026Cameron 2025Introducing Odyssey-2: A General-Purpose World ModelOliver Cameron, Odyssey, 2025Wang 2026Interactive World Simulator for Robot Policy Training and EvaluationYixuan Wang, Rhythm Syed, Fangyu Wu et al., 2026. Nobody has shipped an interactive, inspectable, action-controllable JEPA environment.
2. Latent-only objectives are hard to validate. Without a decoder you cannot see what the model believes, and a good world model and a collapsed one can look identical in the loss. Collapse prevention leans on architectural asymmetries, the EMA target encoder and stop-gradient, whose theory is still thin Bardes 2024Revisiting Feature Prediction for Learning Visual Representations from VideoAdrien Bardes, Quentin Garrido, Jean Ponce et al., 2024.
3. The demonstrated tasks are short-horizon single-arm pick-and-place. No contact-rich assembly, no deformables, no whole-body control Assran 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMido Assran, Adrien Bardes, David Fan et al., 2025.
4. It still needs action-conditioned post-training. The "learns from observation alone" framing does rhetorical work; the controllability comes from the action-conditioned stage, and that stage needs robot data with action labels Assran 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMido Assran, Adrien Bardes, David Fan et al., 2025.
5. Empirically the camps are comparable, not one dominant. In the 2026 survey's LIBERO table, latent-space world-model VLAs (VLA-JEPA at 97.2, JEPA-VLA at 96.4) sit alongside video-based ones (Cosmos Policy and LingBot-VA at 98.5, Say-Dream-ACT at 98.1) Hou 2026World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026. Nothing in the numbers adjudicates the architectural argument.
6. The generative camp partially concedes the efficiency point. Latent-space models like LeWorldModel exist precisely to cut the cost of full high-dimensional generation, and Fast-WAM keeps world modeling as a training objective while dropping future prediction at inference entirely Hou 2026World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026Yuan 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?Tianyuan Yuan, Zibin Dong, Yicheng Liu et al., 2026. That is the JEPA argument arriving through the back door.

What would settle it

Two results would break the stalemate. The first is an interactive, inspectable, action-controllable JEPA environment: something a user can steer and probe the way Genie 3 can be steered today. Whether its absence is fundamental or an engineering gap is the crux of the debate. The second is a demonstrated JEPA advantage on contact-rich tasks, where the generative camp's best evidence and the JEPA camp's best evidence both currently run out. Until one of those lands, the honest summary is that the JEPA line has the cheapest demonstrated path from observation to zero-shot planning, and the generative video line has the only usable simulators. The latent-dynamics module covers the paradigm that sits between them: no decoder either, but trained on the agent's own experience rather than internet video.