GliaCloud Open-Sources an Omniverse-ComfyUI Bridge to Feed Rendered Scene Data Into Video Generation Models

GliaCloud, a Taipei-based AI startup in the NVIDIA Inception program focused on autonomous marketer content creation, open sourced Omniverse-ComfyUI Bridge, according to a July 2025 post on NVIDIA's technical blog. This extension template connects visual data rendered in NVIDIA Omniverse—including RGB images, normal maps, depth maps, instance ID segmentation, and semantic segmentation—with ComfyUI for enhanced AI image processing workflows. The idea for the Omniverse-ComfyUI Bridge first emerged from the GliaCloud team's experience in experimentation with various video-to-video (v2v) generative AI models in ComfyUI. They recognized the increasing demand for detailed visual data, such as depth maps and segmentation outputs, in current generative AI workflows. Within internal experiments, the GliaCloud team has observed noticeable improvements in the refinement and object control of generated outputs by leveraging different data rendered from Omniverse RTX Renderer as v2v model inputs. This was possible using the NVIDIA Omniverse Replicator framework, which greatly improved the overall outputs from a range of open source video generation models by enabling dynamic masks to be created based on segmentation inputs. The v2v models can now iterate through multiple passes to one single input with separate prompts reserved for dedicated regions, resulting in object-based style controls. Core functionalities of the Omniverse-ComfyUI Bridge include single and multi-frame captures for precise edits or sequences for animations, custom node integration for ComfyUI with direct streaming of Omniverse RTX Renderer data, and real-time data processing for faster experimentation and reduced iteration time. "We see significant potential in hybrid workflows connecting advanced physical-AI enabled applications built with NVIDIA Omniverse to generative video models," said David Chen, GliaCloud's chief executive and co-founder. Planned additions include advanced animation controls, integration with human pose estimation models and edge detection.