Scale-Separated Feature Conditioning for Temporally Consistent Stylized Video Generation
Keywords:
stylized video generation; temporal consistency; scale-separated conditioning; diffusion models; socio-technical systems; governance; robustness; infrastructureAbstract
Stylized video generation sits at the intersection of generative modeling, temporal representation learning, and large-scale media infrastructure. Although diffusion-based image synthesis has achieved remarkable visual fidelity and controllability, extending these capabilities to video introduces persistent challenges in temporal coherence, computational cost, and governance. This paper examines scale-separated feature conditioning as a system-level strategy for temporally consistent stylized video generation. The central argument is that conditioning style and motion at distinct spatial and temporal scales reduces the interference between content preservation, stylistic transformation, and frame-to-frame stability. Rather than treating style transfer as a single monolithic operation, a scale-separated approach assigns coarse scales to global aesthetic properties, intermediate scales to structural and semantic cues, and fine scales to texture and high-frequency detail. When coupled with temporal attention, latent propagation, and selective keyframe anchoring, this separation can improve consistency while keeping memory and latency within deployable bounds. The paper also analyzes the structural trade-offs among fidelity, controllability, throughput, and robustness in cloud, edge, and hybrid deployments. It further considers the socio-technical implications of stylized video pipelines, including provenance, content moderation, fairness, energy consumption, and lifecycle governance. By connecting architectural choices to infrastructure constraints and policy requirements, the discussion positions scale-separated feature conditioning not merely as a model design pattern but as an organizing principle for accountable, scalable, and sustainable video generation systems.
References
1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, 27, 2672–2680.
2. Gatys, L. A., Ecker, A. S., & Bethge, M. (2016). Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2414–2423). IEEE.
3. Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision (pp. 694–711). Springer.
4. Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022.
5. Huang, X., & Belongie, S. (2017). Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1501–1510). IEEE.
6. Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1125–1134). IEEE.
7. Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2223–2232). IEEE.
8. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 6840–6851.
9. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations.
10. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10684–10695). IEEE.
11. Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023). Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 22563–22575). IEEE.
12. Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., & Fleet, D. J. (2022). Video diffusion models. Advances in Neural Information Processing Systems, 35, 8633–8646.
13. Esser, P., Chiu, J., Atighehchian, P., Granskog, J., & Germanidis, A. (2023). Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7346–7356). IEEE.
14. Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., & Shou, M. Z. (2023). Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7623–7633). IEEE.
15. Zhang, Jingtao, et al. "Scale-separated conditioning for style-encoder-free diffusion stylization." arXiv preprint arXiv:2608.19719 (2026).
16. Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R., Wang, Z., Navasardyan, S., & Shi, H. (2023). Text2Video-Zero: Text-to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 15954–15964). IEEE.
17. Chen, H., Xia, M., He, Y., Zhang, Y., Cun, X., Yang, S., Xing, J., Liu, Y., Chen, Q., Wang, X., Weng, C., & Shan, Y. (2023). VideoCrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512.
18. Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D., & Dai, B. (2023). AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725.
19. Poole, B., Jain, A., Barron, J. T., & Mildenhall, B. (2023). DreamFusion: Text-to-3D using 2D diffusion. In International Conference on Learning Representations.
20. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (pp. 405–421). Springer.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Intelligent Engineering Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.