JEPA and the Non-Generative Counterargument

The fourth paradigm in the taxonomy rejects the premise of the previous one. Instead of predicting future pixels, predict the representation of the future: train an encoder and a predictor jointly so that the predictor, given the current embedding and an action, matches the embedding a target encoder produces on the true future. The core representation and action-conditioned predictor objectives do not reconstruct pixels, and control planning compares feature maps. V-JEPA 2 separately trains an interpretability decoder in its appendix Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025. This is the Joint-Embedding Predictive Architecture, JEPA, and it is the principal architectural alternative to generative world modelsworld modelA model of how an environment evolves that an agent can query to make decisions: given the current observation or state and a candidate action, it predicts what happens next, so a policy can be trained, evaluated, or planned against the model instead of the real world. The 2026 robotics survey draws the functional line: producing plausible future images is not enough, because a system qualifies only if its predictions change under the agent's action in a way that is useful for decision-making. The single name covers at least six architecturally distinct paradigms, from compact latent dynamics models to action-conditioned video generators.. It is also a position with an institution behind it: Yann LeCun's AMI Labs.

The interactive below illustrates goal-directed replanning with a deterministic teaching model. Raising the search budget changes a prescribed candidate fan and improves alignment by construction. Its synthetic coordinates and distance trace are not learned robot embeddings or measured planning performance.

The argument: predict representations, not pixels

The case against generative pixel prediction, stated as strongly as it can be:

1. Most of the information in a video frame is irrelevant to control: leaf texture, exact sensor noise, the precise fold of a sleeve. A generative objective forces the model to spend capacity modeling all of it.
2. The future is genuinely multimodal and partly unpredictable. A pixel-space model must either average over the possibilities, producing blur, or sample, committing to arbitrary detail. Predicting in representation space lets the model discard what it cannot predict, because the target is an embedding, not a frame.
3. Planning wants an abstract state, not an image. If you have to decode to pixels and re-encode to plan, the decoder is pure overhead.

The original V-JEPA learns masked video features with a context encoder, a predictor, and an exponential-moving-average target encoder whose outputs are stop-gradient targets. Its representation pretraining uses feature prediction without pixel reconstruction, negative examples, text, or a pretrained image encoder. The VideoMix2M pretraining set contains approximately 2 million videos from HowTo100M, Kinetics-400/600/700, and Something-Something-v2 after removing overlap with the Kinetics and Something-Something-v2 validation sets. For frozen-backbone evaluation, the encoder stays fixed while an attentive pooling head and classifier are trained on the downstream task. The paper evaluates appearance-oriented action recognition on Kinetics-400 and motion classification on Something-Something-v2; end-to-end fine-tuning is a separate evaluation protocol Bardes 2024↓Revisiting Feature Prediction for Learning Visual Representations from VideoAdrien Bardes, Quentin Garrido, Jean Ponce et al., 2024.

V-JEPA 2: the argument's strongest artifact

V-JEPA 2 separates action-free representation learning from action-conditioned world-model learning; a model-predictive-control planner then selects robot actions Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

First, action-free pretraining uses masked representation prediction on over 1 million hours of internet video plus images in the VM22M mixture. The largest encoder is ViT-g, listed as 1B parameters in Table 12; its separate pretraining predictor is 22M. The mask-denoising objective predicts target-encoder features with an L1 loss, and 3D rotary position embeddings encode time, height, and width. These are pretraining components, not the later action-conditioned predictor Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

For motion classification, Table 4 reports 77.3% top-1 accuracy on Something-Something v2 for ViT-g384 with a frozen encoder and a learned four-layer attentive probe. Evaluation uses 384-by-384 images, 64 frames per clip, two temporal crops, and three spatial crops; logits are averaged across clips. Appendix C.1 trains the probe on the task training data and reports the best of 20 classifier heads. Appendix A.4.2 separately lists 76.5 for the 384-resolution cooldown ablation in Table 14. Its protocol is not explicitly reconciled with Table 4, so these two reported results are not interchangeable Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

For human-action anticipation, Table 5 reports 39.7% mean-class recall-at-5 for actions on the Epic-Kitchens-100 validation set, using ViT-g384. A learned probe reads both the frozen encoder and the frozen action-free predictor. Its video context is 32 frames at 8 fps and 384-by-384 resolution, ending one second before the action begins. During probe training, anticipation time is sampled from 0.25 to 1.75 seconds; the validation interval is fixed at one second. This is closed-vocabulary kitchen-action classification, not robot control or an action-conditioned robot predictor result Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

Second, V-JEPA 2-AC freezes the ViT-g encoder and trains an approximately 300M-parameter action-conditioned predictor on less than 62 hours of Droid robot video. The retained data are left-camera videos at least four seconds long; training samples four-second clips at 4 fps and 256-by-256 resolution. Here “unlabeled” excludes reward, task-type, and success labels, not robot state information: end-effector position, orientation, and gripper state are supplied, and action vectors are computed from changes between adjacent states Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

The model is deployed zero-shot on Franka Emika Panda arms with RobotiQ grippers in two labs absent from Droid, using identical weights and inference code and no training data from those evaluation robots. Droid itself contains Franka Panda teleoperation, so this is not evidence of arbitrary new-embodiment transfer. The authors manually selected camera positions that worked well; “zero-shot” does not mean setup-free. Their reaching, grasping, reach-with-object, and pick-and-place evaluations use image goals, with two additional supplied subgoal images for pick-and-place. Table 2 reports ten trials per task setting Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

Control uses energy minimization in latent space: the Cross-Entropy Method searches actions that minimize the L1 distance between a predicted future feature map and the goal-image feature map, executes the first action, observes, and replans. Appendix B.2 uses 800 candidate samples, ten refinement iterations, and planning horizon one for these greedy tasks; the pick-and-place sequence relies on the supplied subgoals rather than a long-horizon solution Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

The interactive above is a deterministic two-dimensional teaching model, not a projection of learned V-JEPA 2 features. It uses synthetic points, Euclidean distance, prescribed candidate directions, and a deterministic execution perturbation. Its shrinking trace is guaranteed by construction, not a measured robot result, a learned value function, or a reproduction of the paper’s Cross-Entropy Method optimizer.

V-JEPA 2-AC’s goal-distance energy is a planning objective, not a demonstrated calibration to a goal-conditioned value function. Its predictor learns feature prediction without reward labels; the planning loss compares predicted and target feature maps. The paper also trains a separate pixel decoder to visualize representations in Appendix B.3, but that interpretability decoder is not used to select control actions Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.

The institutional bet: AMI Labs

The argument is no longer only a research position. LeCun left Meta and co-founded AMI Labs (Advanced Machine Intelligence), which raised $1.03 billion at a $3.5 billion pre-money valuation in March 2026 to build world models Heim 2026↓Yann LeCun's AMI Labs raises $1.03B to build world modelsAnna Heim, TechCrunch, 2026. CEO Alexandre LeBrun's own framing to TechCrunch was blunt: "'world models' will be the next buzzword. In six months, every company will call itself a world model to raise funding" Heim 2026↓Yann LeCun's AMI Labs raises $1.03B to build world modelsAnna Heim, TechCrunch, 2026.

The counterargument, stated fairly

1. Every currently usable interactive simulator is generative. Genie 3, Cosmos 3, Odyssey-2, and the Interactive World Simulator all render pixels you can inspect and steer Parker-Holder 2025↓Genie 3: A new frontier for world modelsJack Parker-Holder, Shlomi Fruchter, Google DeepMind, 2025NVIDIA 2026↓Cosmos 3: Omnimodal World Models for Physical AINVIDIA, Aditi, Niket Agarwal et al., 2026Cameron 2025↓Introducing Odyssey-2: A General-Purpose World ModelOliver Cameron, Odyssey, 2025Wang 2026↓Interactive World Simulator for Robot Policy Training and EvaluationYixuan Wang, Rhythm Syed, Fangyu Wu et al., 2026. Nobody has shipped an interactive, inspectable, action-controllable JEPA environment.
2. Latent-only objectives are hard to validate. Without a decoder you cannot see what the model believes, and a good world model and a collapsed one can look identical in the loss. Collapse prevention leans on architectural asymmetries, the EMA target encoder and stop-gradient, whose theory is still thin Bardes 2024↓Revisiting Feature Prediction for Learning Visual Representations from VideoAdrien Bardes, Quentin Garrido, Jean Ponce et al., 2024.
3. The reported robot tests are bounded tabletop skills. The two-lab Franka evaluation covers reaching, grasping, reach-with-object, and pick-and-place with supplied image goals and subgoals. It does not establish contact-rich assembly, deformable manipulation, or whole-body control Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.
4. Action-free pretraining is not the robot-control stage. V-JEPA 2-AC uses robot video with end-effector state signals and constructs action vectors from successive state differences. The excluded labels are rewards, task types, and success annotations Assran 2025↓V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and PlanningMahmoud Assran, Adrien Bardes, David Fan et al., 2025.
5. Empirically the camps are comparable, not one dominant. In the 2026 survey's LIBERO table, latent-space world-model VLAs (VLA-JEPA at 97.2, JEPA-VLA at 96.4) sit alongside video-based ones (Cosmos Policy and LingBot-VA at 98.5, Say-Dream-ACT at 98.1) Hou 2026↓World Model for Robot Learning: A Comprehensive SurveyBohan Hou, Gen Li, Jindou Jia et al., 2026. Nothing in the numbers adjudicates the architectural argument.
6. Fast-WAM separates video co-training from test-time generation. It retains a future-video training objective but omits future-video tokens and their denoising at inference. The video backbone still encodes the current frame once, and the action expert still denoises an action chunk. In its controlled LIBERO and RoboTwin 2.0 comparisons, removing video co-training hurt success more than removing explicit future generation. That finding concerns these tested WAM variants; it does not establish a general advantage for JEPA over generative models. Yuan 2026↓Fast-WAM: Do World Action Models Need Test-time Future Imagination?Tianyuan Yuan, Zibin Dong, Yicheng Liu et al., 2026

What would settle it

Two results would break the stalemate. The first is an interactive, inspectable, action-controllable JEPA environment: something a user can steer and probe the way Genie 3 can be steered today. Whether its absence is fundamental or an engineering gap is the crux of the debate. The second is a demonstrated JEPA advantage on contact-rich tasks, where the generative camp's best evidence and the JEPA camp's best evidence both currently run out. Until one of those lands, the honest summary is that the JEPA line has the cheapest demonstrated path from observation to zero-shot planning, and the generative video line has the only usable simulators.

A deployment rule for representation prediction

Use a JEPA-style model when the downstream decision depends on task-relevant change and reconstructing pixels would consume capacity without adding evidence. Goal-conditioned planning is a natural fit: compare predicted future representations with a goal representation and replan after each observation. Synthetic video generation, visual debugging and simulator replacement are not natural fits because the decoder-free state does not expose what it omitted.

Evaluation must therefore probe the representation. Test whether nearby embeddings correspond to equivalent control outcomes, whether different actions separate when their consequences matter, and whether the induced action ranking matches hardware. Add explicit out-of-distribution detectors or planner fallbacks because low embedding distance is not calibrated physical confidence. The model-based robot-learning guide shows how the representation enters planning; the evaluation guide supplies action-sensitivity and policy-ranking tests that do not require a pixel decoder.