Generative World Models for Sim-to-Real Transfer and Data-Efficient Reinforcement Learning in Autonomous Mobile Robotics
Keywords:
generative world models; sim-to-real transfer; data-efficient reinforcement learning; autonomous mobile robotics; model-based control; robotic infrastructure; safety governanceAbstract
Generative world models have become a promising computational paradigm for autonomous mobile robotics because they can learn compressed predictive representations of environment dynamics and use them for planning, control, and policy improvement. This paper examines the system-level implications of generative world models for sim-to-real transfer and data-efficient reinforcement learning in mobile robotic platforms. It argues that their principal value lies not only in sample efficiency but also in their capacity to mediate between simulated and physical worlds through latent dynamics, uncertainty-aware rollouts, and generative augmentation. The analysis considers recurrent state-space models, transformer-based sequence prediction, diffusion and adversarial generation, and hybrid model-based and model-free control. It then evaluates how domain randomization, dynamics adaptation, residual correction, and real-world fine-tuning can be integrated with generative rollouts to reduce distribution shift. The paper also discusses deployment constraints such as onboard computation, memory bandwidth, latency, safety certification, fleet learning, and long-term maintenance. It emphasizes structural trade-offs among fidelity, generalization, interpretability, and cost. Governance concerns include data provenance, simulation bias, liability, and fairness in access to robotic infrastructure. Through cross-domain comparisons with autonomous driving, manipulation, and aerial robotics, the paper identifies a research agenda for robust, sustainable, and policy-aware world-model systems. The conclusion is that generative world models should be treated as socio-technical infrastructure rather than isolated algorithms, requiring coordinated advances in modeling, verification, deployment, and governance.
References
1. Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017). Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 23–30). IEEE. https://doi.org/10.1109/IROS.2017.8202133
2. Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018). Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE International Conference on Robotics and Automation (pp. 3803–3810). IEEE. https://doi.org/10.1109/ICRA.2018.8460528
3. Ha, D., & Schmidhuber, J. (2018). Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems 31 (pp. 2450–2462). Curran Associates.
4. Hafner, D., Lillicrap, T., Ba, J., & Norouzi, M. (2020). Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations.
5. Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023). Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104.
6. Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016). End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17(39), 1–40.
7. Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2), 99–134. https://doi.org/10.1016/S0004-3702(98)00023-X
8. Sadeghi, F., & Levine, S. (2017). CAD2RL: Real single-image flight without a single real image. In Robotics: Science and Systems. https://doi.org/10.15607/RSS.2017.XIII.034
9. Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., & Fox, D. (2019). Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (pp. 8973–8979). IEEE. https://doi.org/10.1109/ICRA.2019.8793789
10. Xiong, Zhexiao, et al. "UniDrive-WM: Unified understanding, planning and generation world model for autonomous driving." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
11. Todorov, E., Erez, T., & Tassa, Y. (2012). MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 5026–5033). IEEE. https://doi.org/10.1109/IROS.2012.6386109
12. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.
13. Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2016). Continuous control with deep reinforcement learning. In International Conference on Learning Representations.
14. Rao, K., Harris, C., Irpan, A., Levine, S., Ibarz, J., & Khansari, M. (2020). RL-CycleGAN: Reinforcement learning aware simulation-to-real. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11157–11166). https://doi.org/10.1109/CVPR42600.2020.01117
15. Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (pp. 1597–1607). PMLR.
16. Ho, J., & Ermon, S. (2016). Generative adversarial imitation learning. In Advances in Neural Information Processing Systems 29 (pp. 4565–4573). Curran Associates.
17. Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning (pp. 1126–1135). PMLR.
18. Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023). Model-based reinforcement learning: A survey. Foundations and Trends in Machine Learning, 16(1), 1–118. https://doi.org/10.1561/2200000091
19. Sutton, R. S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning (pp. 216–224). Morgan Kaufmann. https://doi.org/10.1016/B978-1-55860-141-3.50030-4
20. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.
21. Dietterich, T. G. (2017). Steps toward robust artificial intelligence. AI Magazine, 38(3), 3–24. https://doi.org/10.1609/aimag.v38i3.2756
22. Sarker, I. H. (2021). Machine learning: Algorithms, real-world applications and research directions. SN Computer Science, 2(3), 160. https://doi.org/10.1007/s42979-021-00592-x
23. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 33–44). ACM. https://doi.org/10.1145/3351095.3372873
24. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (pp. 1097–1105). Curran Associates.
25. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489. https://doi.org/10.1038/nature16961
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Intelligent Engineering Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.