Toggle search
Toggle menu
5.5K
6.1K
231
53.7K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Text-to-3D generation

vrarwiki.com · VR & AR Wiki
More actions
See also: Skybox AI and Procedural generation

Text-to-3D generation is the creation of three-dimensional content from a written description by generative AI models. Depending on the system, the result is a voxel grid, a point cloud, a neural radiance field (NeRF), a set of 3D Gaussians of the kind used in Gaussian splatting, or a textured polygon mesh.[1][2][3] The related task of image-to-3D generation starts from one or more pictures instead, and several systems accept either kind of input or turn a text prompt into an image before building the 3D model.[2][3]

Optimization-based methods from 2022, such as DreamFusion and Magic3D, created a separate 3D model for every prompt under the guidance of a pretrained 2D image model, taking about 1.5 hours and 40 minutes per object respectively.[4][5] Later models trained on large 3D datasets produce an object from a single image in seconds or less.[6][7] Open-weight models include Microsoft's TRELLIS and Tencent's Hunyuan3D, and commercial services include Meshy.[8][9][10] Generators have also been built into creation tools: Meta described mesh generation in the Horizon Worlds desktop editor in April 2025, Roblox announced a mesh generation beta for Roblox Studio in March 2025, and Unity included a 3D Object Generator in the beta of its AI tools, which it opened to all Unity 6 developers in May 2026.[11][12][13][14] Other systems generate whole scenes, among them World Labs' Marble and Meta's WorldGen research.[15][16]

Vendors and researchers have applied these models to virtual reality (VR) and augmented reality (AR) content.[17] Meta said the generative tools in its Horizon Worlds desktop editor could build mixed reality and mobile worlds; Masterpiece Studio paired a text-to-3D service with a VR editing app for Meta Quest; and TechCrunch reported that every world made with Marble could be viewed in VR.[11][18][19] Real-time use adds constraints. Microsoft's TRELLIS.2 model card warns that raw meshes may contain small holes, and Meta's optimization guidance for Meta Quest 3 recommends fewer than 1.5 million triangles per frame.[20][21]

Reviewed 1 November 2026. Paper authors, venues and reported results, model repositories and cards, vendor documentation and announcements, dated prices and statuses, licenses and the US Copyright Office notice against their cited sources. About review dates.

History

Early data-driven work paired 3D shapes with natural-language descriptions. Text2Shape, posted to arXiv in March 2018 by Kevin Chen and colleagues and published in the ACCV 2018 proceedings, used descriptions that the authors collected for objects in the ShapeNet dataset to learn a joint embedding of text and colored 3D shapes, and generated shapes with a conditional Wasserstein GAN.[22] A 2024 survey by Han-Hung Lee, Manolis Savva and Angel X. Chang notes that the method represented shapes as voxel grids storing occupancy and RGB color.[1]

Dream Fields, by Ajay Jain and colleagues (CVPR 2022), generated objects from captions with no 3D training data. It optimized a NeRF so that its renderings from many camera views scored highly against the caption under a pre-trained CLIP model.[23] Where Dream Fields relied on CLIP, DreamFusion (Ben Poole, Ajay Jain, Jonathan T. Barron and Ben Mildenhall, posted in September 2022) used a pretrained text-to-image diffusion model, Imagen, and introduced score distillation sampling (SDS).[4][24] It was presented as an oral paper at ICLR 2023, and its conference page carries an "Outstanding Paper" label.[25]

Work in late 2022 and 2023 addressed the speed, resolution and view consistency of this approach, for example Magic3D's two-stage method and the MVDream multi-view diffusion model.[5][26] OpenAI's Point-E (December 2022) generates point clouds in one to two minutes on a single GPU, and its Shap-E (May 2023) generates assets in seconds.[2][27]

Objaverse, released in December 2022, collected more than 800,000 3D models from Sketchfab, and its authors listed training generative 3D models among its applications.[28] Later models trained in part on Objaverse include the Large Reconstruction Model (LRM, 2023), which predicts a NeRF from one image within five seconds, and Microsoft's TRELLIS (December 2024), which generates 3D assets from text or images.[6][8] Open-weight releases in 2025 included Tencent's Hunyuan3D 2.0 and 2.1 and Roblox's Cube 3D, and Microsoft described TRELLIS.2 in a December 2025 technical report.[9][29][12][30] HunyuanWorld 1.0, WorldGen and Marble, all from 2025, generate scenes and worlds rather than single objects.[31][16][15]

How it works

The Lee, Savva and Chang survey, published in Computer Graphics Forum, groups text-to-3D methods by the supervision they require: paired text and 3D data; unpaired 3D data, with images bridging text and 3D; no 3D data, with guidance from CLIP or a text-to-image diffusion model; and hybrid methods that combine text-to-image and image-to-3D models.[1]

Optimization with 2D image priors

Methods without 3D training data optimize one 3D representation for each prompt, guided by a pretrained 2D model; the survey summarizes this group as "per-prompt optimization, slow".[1] DreamFusion's project page describes SDS as "a way to generate samples from a diffusion model by optimizing a loss function". DreamFusion uses it with Imagen to optimize a 3D scene representation similar to a NeRF, built on mip-NeRF 360, and the resulting models can be exported to meshes with the marching cubes algorithm.[24][4] The paper used only Imagen's 64x64 base model and reported 15,000 optimization iterations per prompt, about 1.5 hours on a TPUv4 machine with four chips. Its authors note that SDS "often produces oversaturated and oversmoothed results", that results vary little between random seeds, and that the low-resolution model limits fine detail.[4]

Magic3D (CVPR 2023) splits the process into two stages. A coarse model is optimized with a low-resolution diffusion prior and accelerated with a sparse 3D hash grid; it then initializes a textured mesh that is refined with a differentiable renderer and a high-resolution latent diffusion model. The authors reported 40 minutes per model, about twice as fast as DreamFusion, and 61.7 percent of raters preferred its results to DreamFusion's.[5]

According to Susung Hong, Donghoon Ahn and Seungryong Kim (NeurIPS 2023), score-distillation methods often produce objects that are inconsistent across views, and one of the most notable cases is the Janus problem, in which a subject's canonical view "appears in other views".[32] The survey describes the Janus problem as models displaying "multiple front faces", likely because the datasets used to train text-to-image models mostly show objects from the front.[1] MVDream (ICLR 2024) is a diffusion model trained on both 2D and 3D data that generates consistent multi-view images from a text prompt; its authors report that using it with SDS makes 2D-lifting methods more consistent and stable.[26]

Image generation and reconstruction

A second group of methods works through images: a text-to-image or multi-view model produces pictures of the object, and a reconstruction model turns pictures into 3D. The survey calls combinations of text-to-image and image-to-3D models hybrid methods.[1] OpenAI's Point-E generates a single synthetic view with a text-to-image diffusion model and then a point cloud with a second diffusion model conditioned on that image. The authors' preprint reports that sample quality falls short of the state of the art but that sampling is one to two orders of magnitude faster.[2] Zero-1-to-3 (ICCV 2023) is a diffusion model that changes the camera viewpoint of an object shown in a single RGB image; it was trained on synthetic data and also applied to single-image 3D reconstruction.[33]

The Large Reconstruction Model replaces per-object optimization with a single forward pass. A transformer with 500 million learnable parameters predicts a NeRF from one image within five seconds, after end-to-end training on roughly one million objects from Objaverse (synthetic renderings) and MVImgNet (real captures).[6] TripoSR, which its repository describes as "collaboratively developed by Tripo AI and Stability AI", builds on the LRM architecture and produces a mesh from a single image in under 0.5 seconds; the repository gives that time for an NVIDIA A100 GPU and about 6 GB of VRAM for the default options.[7][34] Stability AI's Stable Fast 3D (SF3D, August 2024) outputs a UV-unwrapped textured mesh with material parameters and normal maps in about 0.5 seconds, and SPAR3D (January 2025) first samples a sparse point cloud, which users can edit, before producing a mesh in about 0.7 seconds.[35][36]

Meta's 3D AssetGen (NeurIPS 2024) generates several views of an object with separate shaded and albedo channels, reconstructs its shape as a signed distance function and refines the texture in UV space after extracting the mesh. The output has physically based rendering (PBR) materials (color, metalness and roughness), so it can be relit. Its authors report improvements of 17 percent in Chamfer distance and 40 percent in LPIPS over the best concurrent work for few-view reconstruction, and a 72 percent human preference over the best industry competitors "of comparable speed".[37] The Meta 3D Gen pipeline combines AssetGen with the TextureGen text-to-texture model and generates assets in under a minute; in its authors' preprint, the combined two-stage pipeline won 68 percent of evaluations against the single-stage model.[38]

Native 3D generative models

Another line of work trains generative models directly on 3D data. Shap-E first trains an encoder that maps 3D assets to the parameters of implicit functions, then trains a conditional diffusion model on those parameters; the outputs can be rendered as textured meshes or NeRFs. Its authors' preprint reports that it converges faster than Point-E and reaches comparable or better sample quality.[27] OpenAI's repository describes the released models as generating "3D objects conditioned on text or images".[39]

CLAY, published in ACM Transactions on Graphics in 2024, combines a geometry generator with 1.5 billion parameters, built from a multi-resolution variational autoencoder and a latent diffusion transformer, with a multi-view material diffusion model that produces 2K PBR textures (diffuse, roughness and metallic maps). It accepts text, images and 3D-aware controls such as voxels, bounding boxes and point clouds.[40] The Decoder reported that scientists at ShanghaiTech University developed it, that it was trained on more than 500,000 3D models and generates an asset in about 45 seconds, and that a version can be accessed through the Rodin service.[41]

Microsoft's TRELLIS (arXiv December 2024, listed in its repository as a CVPR 2025 spotlight paper) uses a structured latent representation, SLAT, that combines a sparse 3D grid with dense multi-view visual features and can be decoded to radiance fields, 3D Gaussians or meshes. Its rectified-flow transformers have up to 2 billion parameters and were trained on about 500,000 objects.[3] According to the repository, the TRELLIS-500K training set was curated from Objaverse(XL), ABO, 3D-FUTURE, HSSD and Toys4k. The released models are an image-conditioned model with 1.2 billion parameters and text-conditioned models with 342 million, 1.1 billion and 2.0 billion parameters, the text models published on 25 March 2025.[8] TRELLIS.2, described in a December 2025 technical report, uses a sparse "O-Voxel" structure that encodes both geometry and appearance, including PBR attributes. The released model has 4 billion parameters and generates a mesh with PBR materials, including transparency, from a single image at voxel-grid resolutions from 5123 to 15363.[30][20]

Tencent's Hunyuan3D 2.0 pairs Hunyuan3D-DiT, a flow-based diffusion transformer that generates shapes aligned with a condition image, with Hunyuan3D-Paint, which produces texture maps for generated or hand-made meshes.[42] Tencent released the inference code and weights on 21 January 2025 (1.1 billion parameters for the shape model and 1.3 billion for the texture model), followed by a Blender add-on on 27 January and the 0.6-billion-parameter Hunyuan3D-2mini on 18 March 2025.[9] Hunyuan3D 2.1, released on 13 June 2025, added full model weights and training code and replaced the RGB texture model with PBR texture synthesis; its shape and texture models have 3.3 billion and 2 billion parameters.[29]

Roblox's Cube 3D, open-sourced in March 2025, tokenizes 3D shapes so that a transformer can predict the next shape token, much as a language model predicts the next word, and is trained on native 3D data rather than on reconstructions from images.[12] Roblox's accompanying preprint describes the tokenizer as supporting text-to-shape, shape-to-text and text-to-scene generation.[43] Meta's 3D AssetGen 2.0, announced in May 2025, consists of two models, one for 3D meshes and one for textures. Unlike the first version, it estimates geometry with a "single-stage 3D diffusion model" trained on "a large corpus of 3D assets", and Meta said its texture generation added methods for better view consistency, texture in-painting and higher texture resolution.[44]

Training data

Objaverse contains more than 800,000 3D assets by over 100,000 artists, obtained from Sketchfab through its public API; only models with a distributable Creative Commons license were selected.[28] Objaverse-XL (July 2023) expanded the collection to more than 10 million objects, including manually designed objects, photogrammetry scans of landmarks and everyday items, and professional scans of historic artifacts.[45] The Lee, Savva and Chang survey identifies the scarcity of 3D data paired with natural-language descriptions as a constraint on text-to-3D methods.[1]

Selected models

Year Model Developer Input Output Release or availability
2018 Text2Shape Kevin Chen et al. Text Colored voxel shapes Paper (ACCV 2018)[22]
2021 Dream Fields Ajay Jain et al. Text NeRF optimized per prompt with CLIP guidance Paper (CVPR 2022)[23]
2022 DreamFusion Ben Poole et al. Text NeRF, exportable to a mesh Paper (ICLR 2023)[4]
2022 Magic3D Chen-Hsuan Lin et al. Text Textured mesh Paper (CVPR 2023)[5]
2022 Point-E OpenAI Text, through a generated image Point cloud Code and models on GitHub[2]
2023 Shap-E OpenAI Text or image Implicit function rendered as a textured mesh or NeRF Weights and code on GitHub[27][39]
2023 LRM Yicong Hong et al. Single image NeRF Paper (ICLR 2024)[6]
2024 TripoSR Tripo AI and Stability AI Single image Mesh with vertex colors or a baked texture MIT license[34]
2024 Meta 3D AssetGen Meta Text Mesh with PBR materials Paper (NeurIPS 2024)[37]
2024 CLAY Longwen Zhang et al. Text, image or 3D controls Geometry with 2K PBR textures Paper (ACM TOG); a version in the Rodin service, per The Decoder[40][41]
2024 Stable Fast 3D Stability AI Single image UV-unwrapped textured mesh Stability AI Community License; API[35][46]
2024 TRELLIS Microsoft Text or image Radiance field, 3D Gaussians or mesh MIT license; text models March 2025[8]
2025 Hunyuan3D 2.0 and 2.1 Tencent Image Textured mesh, with PBR textures from 2.1 Weights and code on GitHub; 2.0 under a Tencent community license[9][29][47]
2025 SPAR3D Stability AI Single image Mesh through an editable point cloud Paper (CVPR 2025)[36]
2025 Cube 3D Roblox Text Mesh Open-sourced on GitHub and Hugging Face[12]
2025 Meta 3D AssetGen 2.0 Meta Text or image Mesh with generated textures Used internally; in May 2025 Meta said it would roll out to Horizon creators later that year[44]
2025 TRELLIS.2 Microsoft Single image Mesh with PBR materials MIT license[20]

Scene and world generation

Tencent's HunyuanWorld 1.0 (technical report, July 2025) builds 3D scenes from text or images through panoramic "world proxies" that are decomposed into a semantically layered mesh. Its worlds can be exported as meshes, objects are represented separately from the scene, and the report lists virtual reality, physical simulation, game development and interactive content creation as applications.[31] Tencent's Hunyuan3D repository dates the release to 26 July 2025.[9]

Meta's WorldGen (preprint November 2025, listed as a CVPR 2026 poster) produces traversable worlds from a text prompt in stages. A large language model reasons about the scene layout, a procedural step builds a blockout from which a navigation mesh is extracted with a standard algorithm such as Recast, a single reference image of the scene is generated, and image-to-3D generation based on AssetGen2 reconstructs the scene.[16][48] The paper reports scenes of approximately 50 by 50 meters, a pipeline time of about five minutes when its submodules run in parallel on enough GPUs, and output "compatible with standard game engines such as Unreal and Unity".[16] UploadVR noted that the result is a layout of conventional triangle-mesh assets with a navmesh, not Gaussian splats, and that WorldGen was not yet available in Horizon Worlds tools.[49]

World Labs' Marble, generally available since 12 November 2025, creates worlds from text, a single image, several images of a scene, short video, or coarse 3D layouts built in an experimental editing mode called Chisel. It outputs Gaussian splats, triangle meshes (collider meshes for physics and higher-quality meshes) and rendered video.[15] Its documentation states that "Exporting requires a paid plan": the Standard plan covers splat exports in the SPZ and PLY formats, 360-degree panoramas and collider meshes, while textured GLB meshes and commercial rights come with the Pro plan.[50] At launch, TechCrunch listed the plans as Free (four generations), Standard (US$20 per month, 12 generations), Pro (US$35 per month, 25 generations, with commercial rights) and Max (US$95 per month, 75 generations).[19]

Meta's Environment Generation, which Meta described in August 2025 as available to all creators in the Worlds Desktop Editor, produces themed landscapes with 3D meshes, textures and decomposable assets from a text prompt.[51] UploadVR wrote in November 2025 that it could generate only one specific kind of island.[49]

Interactive world models are a different technique. Google DeepMind's Genie 3 (August 2025) generates navigable environments frame by frame at 720p and 24 frames per second, and its environments stay largely consistent for a few minutes; DeepMind contrasts this with NeRFs and Gaussian splatting, which "depend on the provision of an explicit 3D representation".[52] Project Genie, an experimental prototype built on Genie 3 that Google announced on 29 January 2026 for Google AI Ultra subscribers in the United States aged 18 and over, limits generations to 60 seconds and lets users download videos of their worlds.[53]

Tools and services

Tool Provider What it generates Status
Mesh Generation in the Horizon Worlds desktop editor Meta 3D objects from a prompt, chosen from several results and placed in a world Access expanding to the UK and Canada in addition to the US (April 2025); UploadVR listed the US, UK, Canada, EU, Australia and New Zealand (November 2025); the editor was to be rebranded Horizon Studio (September 2025)[11][49][54]
Environment Generation Meta Themed landscapes with meshes and textures Available to all creators in the Worlds Desktop Editor (August 2025)[51]
Mesh Generation API Roblox Meshes from text, in Roblox Studio and through an in-experience Lua API Beta announced March 2025; the engine API reference lists the original text-to-mesh method as scheduled for deprecation (November 2026)[12][55]
3D Object Generator Unity (Unity AI) Static mesh prefabs with materials from text, a reference image or both Open beta for Unity 6 (May 2026)[13]
Meshy Meshy Meshes from text or images, with textures, remeshing and low-poly models API lists meshy-7.1, meshy-6, meshy-6-lite and two Smart Topology models as active (November 2026)[10][56][57]
Masterpiece X - Generate Masterpiece Studio Textured GLB models from short descriptions Launched October 2023 as a browser-based service; the VR app's store listing says it "is now in VR" (November 2026)[18][58]
Genie Luma AI Text-to-3D models as GLB files Free research preview on Discord (November 2023); sunset on 1 January 2026[59][60]
Stable Fast 3D Stability AI Textured meshes from a single image Weights, demo and API (August 2024)[35][46]
Marble World Labs Worlds as Gaussian splats, meshes and video Generally available since 12 November 2025[15]

Unity states that the 3D Object Generator, one of its AI tools for the editor, "creates static 3D mesh prefabs" and is "well-suited for simple, single-part props", and it warns that the beta features it describes "may change, be limited, or be discontinued without notice".[13] The documentation for its Assistant package (version 2.21.0-pre.2) lists 3D objects and terrain layers among the generator types, alongside sprites, textures, materials and cubemaps.[61] Roblox said in March 2025 that the beta of Cube 3D mesh generation would be available in Roblox Studio and as an in-experience Lua API, and that developers using its Assistant could type a prompt such as "/generate orange safety cone", receive a mesh within seconds and then add texture and color. It said it planned to extend the system to scene generation and to what it calls "4D creation, where the fourth dimension is interaction between objects, environments, and people".[12] As of November 2026, Roblox's engine API reference describes GenerationService as generating 3D objects from text prompts with the Cube 3D model, lists its original text-to-mesh method as scheduled for future deprecation, and points to a newer method that generates multi-mesh models from a text prompt, an image or both, with an optional maximum triangle count and an option to skip textures.[55]

Meshy's changelog dates its Meshy 7 image-to-3D model to August 2026 and Meshy 7.1 to September 2026, and its deprecation table records that meshy-4 was retired on 20 March 2026 and meshy-5 on 10 October 2026, and that meshy-7 was marked deprecated on 18 September 2026 with its retirement date still to be announced.[56][57] Meshy lists integrations with the game engines Unity, Unreal Engine, Godot and Roblox Studio, and with tools including Blender, Maya, 3ds Max and ZBrush.[62] Luma AI's Genie, a free research preview on Discord from November 2023, produced four simple models per prompt in about 20 seconds, which could be refined in about 20 minutes. Luma's App Store listing states that Genie was sunset on 1 January 2026 and that users can still export their content.[59][60]

Use in VR and AR content creation

In April 2025 Meta described Mesh Generation in the Horizon Worlds desktop editor as three steps: the creator enters a prompt in the editor's generative AI toolbar, chooses among several generated objects or refines the prompt, and places the object in the world. Meta said the tools could build both mixed reality and mobile worlds and claimed that they "dramatically reduce development time from weeks to as little as hours".[11] In May 2025 it said AssetGen 2.0 was used internally for 3D world creation, would "roll out later this year to Horizon creators", and would become a building block for "auto-regressive generation of entire 3D scenes".[44] UploadVR reported in September 2025 that Meta would rebrand the desktop editor, a Windows application released in early access in February 2025, as Horizon Studio, and that worlds built with it would use the new Horizon Engine, which replaces the Unity runtime of Horizon Worlds. By then, UploadVR wrote, creators in the US, UK, Canada, EU, Australia and New Zealand could use AI to generate 3D meshes, textures, skyboxes, sound effects, ambient audio, TypeScript and islands.[54] Meta's VR plans for Worlds then changed. In February 2026 it said it was shifting the focus of Worlds to be "almost exclusively mobile" and described Meta Horizon Studio and Meta Horizon Engine as "custom-designed for this purpose". In March 2026 its chief technology officer, Andrew Bosworth, said Horizon Worlds would keep working in VR for existing games while "most of our energy is going towards mobile and the Meta Horizon Engine".[63][64] At Meta Connect in September 2026, Meta presented Horizon Create, a mobile app that builds playable 2D or 3D games, including environments and characters, from prompts or templates, and Horizon Studio, a browser-based environment that adds visual editing to the same prompting; Meta said both were in testing with its creator community and that early access was rolling out gradually.[65]

Masterpiece Studio launched Masterpiece X - Generate in October 2023 as a browser-based service that turns short descriptions into textured models, downloadable as GLB files for Unity, Unreal Engine or Blender, in two to eight minutes per model. CG Channel reported that generation was limited to predefined categories of objects, humans and animals, that the results were "lightweight 3D assets" for AR projects and indie games, and that the service combined the company's own AI with NVIDIA's Picasso platform. Generated models could be edited, including their geometry, textures, rigs and animation, in the free Masterpiece X VR app for Meta Quest and Meta Quest Pro. Credits cost US$14.99 for 750 at launch, at 50 credits per model.[18] As of November 2026, the app's store listing says that Masterpiece X - Generate "is now in VR", describes a generator that creates assets with meshes, textures and animations (animations for human models only), and offers 250 free bonus credits.[58]

TechCrunch reported at Marble's launch that it was compatible with the Apple Vision Pro and Meta Quest 3 and that every generated world could be viewed in VR. According to the report, World Labs co-founder Justin Johnson expected Marble's first uses to be in gaming, visual effects for film and virtual reality; he said the company was not focusing on VR applications at the time but that the VR industry was "starved for content", and that Marble was "not designed to replace the entire existing pipeline for gaming" but to provide assets that could be dropped into it.[19] World Labs' export documentation calls the Spark library for Three.js "Perfect for creating VR experiences, interactive games, and custom visualization tools" and links separate guides for Unreal Engine, Unity, Houdini and Blender.[50] Stability AI aimed Stable Fast 3D at "enterprises and indie developers in gaming and virtual reality" and listed "Fast model creation for AR/VR" and static game assets such as background objects and furniture among its uses.[46]

Research prototypes have placed generation inside the headset. Dreamcrafter (CHI 2025) is a VR system for editing radiance-field scenes such as NeRFs and Gaussian splats, built in Unity with the Unity XR toolkit and Mixed Reality Toolkit plugins and used on a Meta Quest 3 with PC link. Prompted objects are previewed as stylized images of a low-fidelity Shap-E mesh, and the full objects are generated afterwards with image-to-3D models such as LGM or GRM; edits to Gaussian-splat objects use Instruct-GS2GS, with Instruct-Pix2Pix images as previews. These 2D image proxies keep the scene interactive while the slower 3D generation runs. The paper reports that edit previews take about 10 seconds and object previews about 15 seconds. In a first-use study with seven participants, participants created more objects by prompting but reported feeling "more in control" when sculpting and then stylizing, and six of the seven relied mainly on image previews to judge composition.[17]

VRCopilot (UIST 2024) brings pre-trained generative models into VR authoring of room layouts, using the ATISS layout model and selecting furniture from a catalog of 7,302 3D-FRONT models rather than generating new geometry. It was built in Unity 2021.3 for Meta Quest and Rift headsets. In a within-subject study with 15 participants on a Meta Quest 2, perceived agency was highest for manual creation, followed by scaffolded creation with wireframes and then fully automatic generation; in an earlier study with 14 participants, generated suggestions sometimes did not match participants' preferences.[66]

Limitations

The Lee, Savva and Chang survey says that current text-to-3D methods "do not afford natural editability" of their outputs, that generating larger 3D scenes composed of several objects "also remains challenging", and that per-prompt optimization is "very slow, and currently highly impractical for real-world deployment". It describes the Janus problem as a notable limitation of SDS-based generation and notes a tendency of SDS results toward "excessive smoothness and limited variation".[1]

Sources describe several properties of generated geometry that matter for real-time use. Microsoft's TRELLIS.2 model card states that "the generated raw meshes may occasionally contain small holes or minor topological discontinuities" and points to post-processing such as hole filling where watertight geometry is required.[20] The SF3D paper notes that most existing methods rely on vertex colors and adds a delighting step that removes low-frequency illumination effects so that meshes can be relit; TripoSR's default output also uses vertex colors, with an option to bake a texture instead.[35][34] Commercial services expose controls for these properties. Meshy's text-to-3D API accepts a target polygon count from 100 to 300,000 (default 30,000) when remeshing, quad-dominant or triangle topology, and exports to GLB, OBJ, FBX, STL, USDZ and 3MF; its triangle-only Smart Topology models meshy-t2 and meshy-t2.5 accept 100 to 15,000 and 100 to 30,000 polygons.[10] Its changelog describes Smart Topology, added in July 2026, as "a new AI model for Image to 3D that generates low-poly models".[56]

Platform guidance sets budgets for what a headset draws each frame. Meta's device optimization guidance recommends fewer than 1.5 million triangles per frame and fewer than 200 draw calls on Meta Quest 3, and fewer than 750,000 triangles and 100 draw calls on Meta Quest 2, with a frame budget of 13.8 ms at 72 Hz. The triangle figure is a budget per frame, so it covers everything drawn in a frame rather than a single asset.[21] The WorldGen authors write that relying on a single reference view restricts the scale of scenes, that worlds "spanning kilometers are not supported natively", that the single-view conditioning limits multi-layered environments such as multi-floor dungeons, and that because each object is represented "without geometry or texture reuse", rendering efficiency may become a concern in very large scenes.[16] Unity describes its generator as suited to simple, single-part props and its output as static.[13]

The open models' repositories document their hardware requirements. TRELLIS requires an NVIDIA GPU with at least 16 GB of memory and TRELLIS.2 at least 24 GB, and both are tested only on Linux.[8][20] Tencent gives 6 GB of VRAM for shape generation and 16 GB for shape and texture generation with Hunyuan3D 2.0, and 10 GB, 21 GB and 29 GB for shape, texture and both with Hunyuan3D 2.1.[9][29] In VR authoring studies, six of Dreamcrafter's seven participants had difficulty rotating and arranging objects, and the VRCopilot authors note that both of their studies were short lab sessions with small samples.[17][66]

Licensing and rights

Training data carries its own licenses. Objaverse's dataset card licenses the dataset as a whole under ODC-By v1.0, while each object keeps its own Creative Commons license: 721K objects under CC-BY 4.0, 25K under CC-BY-NC 4.0, 52K under CC-BY-NC-SA 4.0, 16K under CC-BY-SA 4.0 and 3.5K under CC0 1.0, with the license for each object given in the metadata.[67]

Model licenses differ. TRELLIS's models and most of its code, the TRELLIS.2 model and TripoSR are released under the MIT License.[8][20][34] Stable Fast 3D uses the Stability AI Community License, which "allows non-commercial use and commercial use for individuals or organizations with up to $1M in annual revenue".[46] The Tencent Hunyuan 3D 2.0 Community License Agreement, which gives a release date of 21 January 2025, defines its territory as worldwide excluding the European Union, the United Kingdom and South Korea, and makes any use outside that territory unlicensed. A licensee whose products or services had more than 1 million monthly active users in the calendar month before the version's release date "must request a license from Tencent", and neither the model nor its outputs may be used "to improve any other AI model (other than Tencent Hunyuan 3D 2.0 or Model Derivatives thereof)".[47]

Services set their own terms for outputs. Meshy's pricing documentation places Free-plan output under CC BY 4.0, usable commercially with attribution, and paid-plan output under a "Private" license with full commercial use, and states "You own the output" for every plan.[68] Marble's commercial rights come with the Pro plan.[50]

On copyright in the United States, part 2 of the U.S. Copyright Office's report on artificial intelligence, released in January 2025, concluded that protection does not extend to "the mere provision of prompts". It can cover outputs where "a human-authored work is perceptible in an AI output" or where a person makes "creative arrangements or modifications of the output". The Office found existing principles "flexible enough to apply to this new technology" and concluded that "the case has not been made for changes to existing law".[69]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 Han-Hung Lee, Manolis Savva, Angel X. Chang (2024). "Text-to-3D Shape Generation". Computer Graphics Forum, vol. 43, no. 2 (arXiv version). https://arxiv.org/abs/2403.13289. Retrieved 2026-11-01.
  2. ↑ 2.0 2.1 2.2 2.3 2.4 Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, Mark Chen (2022-12-16). "Point-E: A System for Generating 3D Point Clouds from Complex Prompts". arXiv preprint. https://arxiv.org/abs/2212.08751. Retrieved 2026-11-01.
  3. ↑ 3.0 3.1 3.2 Jianfeng Xiang et al. (2024-12-02). "Structured 3D Latents for Scalable and Versatile 3D Generation". CVPR 2025 (arXiv version). https://arxiv.org/abs/2412.01506. Retrieved 2026-11-01.
  4. ↑ 4.0 4.1 4.2 4.3 4.4 Ben Poole, Ajay Jain, Jonathan T. Barron, Ben Mildenhall (2022-09-29). "DreamFusion: Text-to-3D using 2D Diffusion". ICLR 2023 (arXiv version). https://arxiv.org/abs/2209.14988. Retrieved 2026-11-01.
  5. ↑ 5.0 5.1 5.2 5.3 Chen-Hsuan Lin et al. (2022-11-18). "Magic3D: High-Resolution Text-to-3D Content Creation". CVPR 2023 (arXiv version). https://arxiv.org/abs/2211.10440. Retrieved 2026-11-01.
  6. ↑ 6.0 6.1 6.2 6.3 Yicong Hong et al. (2023-11-08). "LRM: Large Reconstruction Model for Single Image to 3D". ICLR 2024 (arXiv version). https://arxiv.org/abs/2311.04400. Retrieved 2026-11-01.
  7. ↑ 7.0 7.1 Dmitry Tochilkin et al. (2024-03-04). "TripoSR: Fast 3D Object Reconstruction from a Single Image". arXiv preprint (technical report). https://arxiv.org/abs/2403.02151. Retrieved 2026-11-01.
  8. ↑ 8.0 8.1 8.2 8.3 8.4 8.5 "microsoft/TRELLIS". GitHub. Microsoft. https://github.com/microsoft/TRELLIS. Retrieved 2026-11-01.
  9. ↑ 9.0 9.1 9.2 9.3 9.4 9.5 "Tencent-Hunyuan/Hunyuan3D-2". GitHub. Tencent. https://github.com/Tencent-Hunyuan/Hunyuan3D-2. Retrieved 2026-11-01.
  10. ↑ 10.0 10.1 10.2 "Text to 3D". Meshy API documentation. https://docs.meshy.ai/en/api/text-to-3d. Retrieved 2026-11-01.
  11. ↑ 11.0 11.1 11.2 11.3 "Build Faster and Smarter with GenAI Tools in Meta Horizon Worlds". Meta for Developers. Meta Platforms. 2025-04-14. https://developers.meta.com/vr/blog/worlds/build-faster-smarter-genai-tools-meta-horizon-worlds-mesh-gen/. Retrieved 2026-11-01.
  12. ↑ 12.0 12.1 12.2 12.3 12.4 12.5 Anupam Singh, Nick Tornow (2025-03-17). "Introducing Roblox Cube: Our Core Generative AI System for 3D and 4D". Roblox Newsroom. https://about.roblox.com/newsroom/2025/03/introducing-roblox-cube. Retrieved 2026-11-01.
  13. ↑ 13.0 13.1 13.2 13.3 "Unity's AI tools in beta: Create props with the 3D Object Generator". Unity Blog. Unity Technologies. 2026-05-21. https://unity.com/blog/unity-ai-3d-object-generator. Retrieved 2026-11-01.
  14. ↑ "Unity AI's Open Beta Now Live for Unity 6". Unity Discussions. Unity Technologies. 2026-05-01. https://discussions.unity.com/t/unity-ai-s-open-beta-now-live-for-unity-6/1718560. Retrieved 2026-11-01.
  15. ↑ 15.0 15.1 15.2 15.3 "Marble: A Multimodal World Model". World Labs. 2025-11-12. https://www.worldlabs.ai/blog/marble-world-model. Retrieved 2026-11-01.
  16. ↑ 16.0 16.1 16.2 16.3 16.4 Dilin Wang et al. (2025-11-20). "WorldGen: From Text to Traversable and Interactive 3D Worlds". CVPR 2026 (arXiv version). https://arxiv.org/abs/2511.16825. Retrieved 2026-11-01.
  17. ↑ 17.0 17.1 17.2 Cyrus Vachha et al. (2025). "Dreamcrafter: Immersive Editing of 3D Radiance Fields Through Flexible, Generative Inputs and Outputs". CHI 2025 (Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems), arXiv version. https://arxiv.org/abs/2512.20129. Retrieved 2026-11-01.
  18. ↑ 18.0 18.1 18.2 Jim Thacker (2023-10-16). "Check out text-to-3D service Masterpiece X - Generate". CG Channel. https://www.cgchannel.com/2023/10/check-out-text-to-3d-service-masterpiece-x-generate/. Retrieved 2026-11-01.
  19. ↑ 19.0 19.1 19.2 Rebecca Bellan (2025-11-12). "Fei-Fei Li's World Labs speeds up the world model race with Marble, its first commercial product". TechCrunch. https://techcrunch.com/2025/11/12/fei-fei-lis-world-labs-speeds-up-the-world-model-race-with-marble-its-first-commercial-product/. Retrieved 2026-11-01.
  20. ↑ 20.0 20.1 20.2 20.3 20.4 20.5 "microsoft/TRELLIS.2-4B model card". Hugging Face. Microsoft. https://huggingface.co/microsoft/TRELLIS.2-4B/blob/main/README.md. Retrieved 2026-11-01.
  21. ↑ 21.0 21.1 "Device-specific optimization (Quest 3 vs Quest 2)". Meta Horizon OS Developers. Meta Platforms. 2026-05-11. https://developers.meta.com/horizon/resources/device-optimization-comparison/. Retrieved 2026-11-01.
  22. ↑ 22.0 22.1 Kevin Chen, Christopher B. Choy, Manolis Savva, Angel X. Chang, Thomas Funkhouser, Silvio Savarese (2018-03-22). "Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings". arXiv preprint; published in Computer Vision - ACCV 2018, Lecture Notes in Computer Science, pp. 100-116. https://arxiv.org/abs/1803.08495. Retrieved 2026-11-01.
  23. ↑ 23.0 23.1 Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, Ben Poole (2021-12-02). "Zero-Shot Text-Guided Object Generation with Dream Fields". CVPR 2022 (arXiv version). https://arxiv.org/abs/2112.01455. Retrieved 2026-11-01.
  24. ↑ 24.0 24.1 "DreamFusion: Text-to-3D using 2D Diffusion". DreamFusion project page. https://dreamfusion3d.github.io/. Retrieved 2026-11-01.
  25. ↑ "DreamFusion: Text-to-3D using 2D Diffusion". ICLR 2023. https://iclr.cc/virtual/2023/oral/12607. Retrieved 2026-11-01.
  26. ↑ 26.0 26.1 Yichun Shi et al. (2023-08-31). "MVDream: Multi-view Diffusion for 3D Generation". ICLR 2024 (arXiv version). https://arxiv.org/abs/2308.16512. Retrieved 2026-11-01.
  27. ↑ 27.0 27.1 27.2 Heewoo Jun, Alex Nichol (2023-05-03). "Shap-E: Generating Conditional 3D Implicit Functions". arXiv preprint. https://arxiv.org/abs/2305.02463. Retrieved 2026-11-01.
  28. ↑ 28.0 28.1 Matt Deitke et al. (2022-12-15). "Objaverse: A Universe of Annotated 3D Objects". CVPR 2023, pp. 13142-13153 (arXiv version). https://arxiv.org/abs/2212.08051. Retrieved 2026-11-01.
  29. ↑ 29.0 29.1 29.2 29.3 "Tencent-Hunyuan/Hunyuan3D-2.1". GitHub. Tencent. https://github.com/Tencent-Hunyuan/Hunyuan3D-2.1. Retrieved 2026-11-01.
  30. ↑ 30.0 30.1 Jianfeng Xiang et al. (2025-12-16). "Native and Compact Structured Latents for 3D Generation". arXiv preprint (technical report). https://arxiv.org/abs/2512.14692. Retrieved 2026-11-01.
  31. ↑ 31.0 31.1 HunyuanWorld Team (2025-07-29). "HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels". arXiv preprint (technical report). https://arxiv.org/abs/2507.21809. Retrieved 2026-11-01.
  32. ↑ Susung Hong, Donghoon Ahn, Seungryong Kim (2023-03-27). "Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation". NeurIPS 2023 (arXiv version). https://arxiv.org/abs/2303.15413. Retrieved 2026-11-01.
  33. ↑ Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, Carl Vondrick (2023-03-20). "Zero-1-to-3: Zero-shot One Image to 3D Object". ICCV 2023, pp. 9298-9309 (arXiv version). https://arxiv.org/abs/2303.11328. Retrieved 2026-11-01.
  34. ↑ 34.0 34.1 34.2 34.3 "VAST-AI-Research/TripoSR". GitHub. https://github.com/VAST-AI-Research/TripoSR. Retrieved 2026-11-01.
  35. ↑ 35.0 35.1 35.2 35.3 Mark Boss, Zixuan Huang, Aaryaman Vasishta, Varun Jampani (2024-08-01). "SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglement". CVPR 2025 (arXiv version). https://arxiv.org/abs/2408.00653. Retrieved 2026-11-01.
  36. ↑ 36.0 36.1 Zixuan Huang, Mark Boss, Aaryaman Vasishta, James M. Rehg, Varun Jampani (2025-01-08). "SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images". CVPR 2025 (arXiv version). https://arxiv.org/abs/2501.04689. Retrieved 2026-11-01.
  37. ↑ 37.0 37.1 Yawar Siddiqui et al. (2024-07-02). "Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials". Advances in Neural Information Processing Systems 37 (NeurIPS 2024), arXiv version. https://arxiv.org/abs/2407.02445. Retrieved 2026-11-01.
  38. ↑ Raphael Bensadoun et al. (2024-07-02). "Meta 3D Gen". arXiv preprint. https://arxiv.org/abs/2407.02599. Retrieved 2026-11-01.
  39. ↑ 39.0 39.1 "openai/shap-e". GitHub. OpenAI. https://github.com/openai/shap-e. Retrieved 2026-11-01.
  40. ↑ 40.0 40.1 Longwen Zhang et al. (2024). "CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets". ACM Transactions on Graphics, vol. 43, no. 4, pp. 1-20 (arXiv version). https://arxiv.org/abs/2406.13897. Retrieved 2026-11-01.
  41. ↑ 41.0 41.1 Maximilian Schreiner (2024-07-23). "CLAY creates detailed 3D objects from text and images". The Decoder. https://the-decoder.com/clay-creates-detailed-3d-objects-from-text-and-images/. Retrieved 2026-11-01.
  42. ↑ Zibo Zhao et al. (2025-01-21). "Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation". arXiv preprint (technical report). https://arxiv.org/abs/2501.12202. Retrieved 2026-11-01.
  43. ↑ Foundation AI Team, Roblox (2025-03-19). "Cube: A Roblox View of 3D Intelligence". arXiv preprint. https://arxiv.org/abs/2503.15475. Retrieved 2026-11-01.
  44. ↑ 44.0 44.1 44.2 Rakesh Ranjan, Andrea Vedaldi, Mahima Gupta, Christopher Ocampo, Ocean Quigley (2025-05-08). "Introducing Meta 3D AssetGen 2.0: A new foundation model for 3D content creation". Meta Horizon OS Developers. Meta Platforms. https://developers.meta.com/horizon/blog/AssetGen2. Retrieved 2026-11-01.
  45. ↑ Matt Deitke et al. (2023-07-11). "Objaverse-XL: A Universe of 10M+ 3D Objects". Advances in Neural Information Processing Systems 36 (NeurIPS 2023), arXiv version. https://arxiv.org/abs/2307.05663. Retrieved 2026-11-01.
  46. ↑ 46.0 46.1 46.2 46.3 "Introducing Stable Fast 3D: Rapid 3D Asset Generation From Single Images". Stability AI. https://stability.ai/news-updates/introducing-stable-fast-3d. Retrieved 2026-11-01.
  47. ↑ 47.0 47.1 "Tencent Hunyuan 3D 2.0 Community License Agreement". GitHub. Tencent. 2025-01-21. https://raw.githubusercontent.com/Tencent-Hunyuan/Hunyuan3D-2/main/LICENSE. Retrieved 2026-11-01.
  48. ↑ "WorldGen: From Text to Traversable and Interactive 3D Worlds". CVPR 2026 (poster). https://cvpr.thecvf.com/virtual/2026/poster/39770. Retrieved 2026-11-01.
  49. ↑ 49.0 49.1 49.2 David Heaney (2025-11-21). "Meta's WorldGen AI-Generates 3D Worlds From A Text Prompt". UploadVR. https://www.uploadvr.com/meta-worldgen-ai-generates-3d-worlds/. Retrieved 2026-11-01.
  50. ↑ 50.0 50.1 50.2 "Exporting from Marble". World Labs documentation. https://docs.worldlabs.ai/marble/export/gaussian-splat. Retrieved 2026-11-01.
  51. ↑ 51.0 51.1 "Meet the Newest GenAI Tools: Environment Generation and Embodied LLM NPCs". Meta Horizon OS Developers. Meta Platforms. 2025-08-27. https://developers.meta.com/horizon/blog/environment-generation-embodied-conversational-llm-npcs-genai-tools/. Retrieved 2026-11-01.
  52. ↑ "Genie 3: A new frontier for world models". Google DeepMind. 2025-08-05. https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/. Retrieved 2026-11-01.
  53. ↑ Diego Rivas, Elliott Breece, Suz Chambers (2026-01-29). "Project Genie: Experimenting with infinite, interactive worlds". Google Blog. https://blog.google/innovation-and-ai/models-and-research/google-deepmind/project-genie/. Retrieved 2026-11-01.
  54. ↑ 54.0 54.1 David Heaney (2025-09-18). "Meta Horizon Studio Will AI-Generate Just About Anything For Horizon Worlds". UploadVR. https://www.uploadvr.com/meta-horizon-studio-upgrade-ai-assistant-horizon-worlds/. Retrieved 2026-11-01.
  55. ↑ 55.0 55.1 "GenerationService (Engine API reference source)". Roblox Creator Documentation, GitHub. Roblox. https://github.com/Roblox/creator-docs/blob/main/content/en-us/reference/engine/classes/GenerationService.yaml. Retrieved 2026-11-01.
  56. ↑ 56.0 56.1 56.2 "Changelog". Meshy documentation. https://docs.meshy.ai/en/webapp/changelog. Retrieved 2026-11-01.
  57. ↑ 57.0 57.1 "Deprecations". Meshy API documentation. https://docs.meshy.ai/en/api/deprecations. Retrieved 2026-11-01.
  58. ↑ 58.0 58.1 "Masterpiece X". Meta. https://www.meta.com/experiences/masterpiece-x/5502306219889537/. Retrieved 2026-11-01.
  59. ↑ 59.0 59.1 Maximilian Schreiner (2023-11-02). "Genie by Luma AI is a free text-to-3D generator for Discord". The Decoder. https://the-decoder.com/genie-by-luma-ai-is-a-free-text-to-3d-generator-for-discord/. Retrieved 2026-11-01.
  60. ↑ 60.0 60.1 "Luma 3D Capture". App Store. Apple. https://apps.apple.com/us/app/luma-3d-capture/id1615849914. Retrieved 2026-11-01.
  61. ↑ "Create visual assets". Unity Documentation (Assistant package 2.21.0-pre.2). Unity Technologies. https://docs.unity3d.com/Packages/[email protected]/manual/Sprite/visual-assets.html. Retrieved 2026-11-01.
  62. ↑ "Integrations". Meshy. https://www.meshy.ai/integrations. Retrieved 2026-11-01.
  63. ↑ Samantha Ryan (2026-02-19). "Our Renewed Focus in 2026". Meta Horizon OS Developers. Meta Platforms. https://developers.meta.com/horizon/blog/2026-vr-state-of-the-union-horizon-mobile-focus/. Retrieved 2026-11-01.
  64. ↑ David Heaney (2026-03-19). "Meta Backtracks, Will Keep Horizon Worlds VR Support "For Existing Games"". UploadVR. https://www.uploadvr.com/meta-will-keep-horizon-worlds-vr-support-for-existing-games/. Retrieved 2026-11-01.
  65. ↑ "Meta Connect Recap: Sign Up for Early Access to Horizon Create and Horizon Studio". Meta for Developers. Meta Platforms. 2026-09-24. https://developers.meta.com/blog/meta-connect-recap-horizon-create-and-horizon-studio/. Retrieved 2026-11-01.
  66. ↑ 66.0 66.1 Lei Zhang, Jin Pan, Jacob Gettig, Steve Oney, Anhong Guo (2024). "VRCopilot: Authoring 3D Layouts with Generative AI Models in VR". UIST 2024 (Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology), arXiv version. https://arxiv.org/abs/2408.09382. Retrieved 2026-11-01.
  67. ↑ "allenai/objaverse". Hugging Face. https://huggingface.co/datasets/allenai/objaverse. Retrieved 2026-11-01.
  68. ↑ "Pricing & Credits". Meshy documentation. https://docs.meshy.ai/en/webapp/pricing. Retrieved 2026-11-01.
Retrieved from "https://vrarwiki.com/index.php?title=Text-to-3D_generation&oldid=53289"