Nvidia, a GPU manufacturer, has announced Magic3D, a generative AI that creates 3D models from a text prompt. This technology helps in the conceptualization of components with complex and organic shapes, which can be brought to life with the help of 3D printing.
NVIDIA’s Magic3D: Generating 3D Models from Text
AI in 3D Modeling
According to the annual 3D Printing Industry Executive Survey, the automatic generation of 3D models using AI is a growing trend. Magic3D creates a 3D mesh model with colored texture within 40 minutes from a text prompt, like “A blue poison-dart frog sitting on a water lily.”
The result can be used in CGI art scenes or video games.
Response to DreamFusion
Magic3D is Nvidia’s response to DreamFusion, a text-to-3D model released by Google researchers in September 2022. In comparison, Physna Inc. created a generative AI prototype for 3D models and scenes in two weeks using 8,000 models.
Easy 3D Model Creation
The researchers in the paper explain how Magic3D will allow anyone to create 3D models without any special training.
The technology could potentially speed up video game and VR development, as well as find applications in special effects for film and TV. Nvidia hopes to democratize 3D synthesis and open up everyone’s creativity in 3D content creation.
NVIDIA’s AI Expertise
Nvidia is well-equipped to progress AI. The company’s GPUs can create lifelike graphics using shaders, which calculate how each pixel in an image should display in specific lighting.
Nvidia GPUs can quickly render images due to their design for conducting multiple simple calculations at once, making them faster than Intel microprocessors or general-purpose CPUs.
Nvidia sees AI applications as a critical growth driver, with Bloomberg attributing a $4.6 billion increase in the wealth of Nvidia founder Jensen Huang to the popularity of ChatGPT, an AI chatbot.
Magic3D’s Capabilities
Magic3D uses a two-step approach that refines a low-resolution model to a higher resolution through a text-to-image model and then into volumetric NeRF data. This process allows Magic3D to produce 3D objects twice as fast as Google’s DreamFusion.
In addition to text-prompt-based 3D modeling, Magic3D can also perform prompt-based 3D mesh editing. By modifying the text prompt, the 3D model can be altered accordingly.
The authors of Magic3D have also demonstrated the ability to maintain subject consistency across multiple generations and apply the style of a 2D image to a 3D model.
Generative AI and 3D Printing: A Promising Future
Physna
Paul Powers, CEO of Physna Inc., believes that generative AI has the potential to revolutionize 3D printing. Despite being a 3D search and analysis company, Physna quickly built a basic generative AI prototype for 3D models and scenes in just two weeks with only three engineers and 8,000 models.
According to Powers, the main obstacle preventing the widespread adoption of generative AI in 3D printing is the limited availability of labeled 3D data and the difficulty of creating complex 3D models.
This has made 3D analysis historically challenging, with fewer companies focusing on the area in comparison to 2D analysis.
Google’s DreamFusion
Google’s DreamFusion team also highlights the lack of 3D data compared to 2D data as a challenge for generative AI in 3D printing. While training on NeRFs (Neural Radiance Fields) may be more helpful than training on 2D models, they are not a suitable substitute for labeled 3D models.
Without a solution, generative AI will not perform as well in 3D as it does in other areas in the near future.
Introduction to GPU Computing
GPU computing is a type of hybrid or heterogeneous computing that utilizes a graphics processing unit (GPU) as a co-processor to speed up the performance of CPU-based applications.
This is achieved by offloading some of the time-consuming and compute-intensive code to the GPU, which has hundreds of smaller cores in comparison to the 4 to 8 cores of a CPU. This results in an overall faster performance from the application from the user’s perspective.
NVIDIA’s CUDA Parallel Programming Model
Application developers can take advantage of the parallel GPU architecture’s performance by using NVIDIA’s “CUDA” parallel programming model. This model is supported by all NVIDIA GPUs, including GeForce, Quadro, and Tesla.
It allows developers to write parallel algorithms that can be executed on a GPU and combined with the rest of the application code running on the CPU.
NVIDIA’s Differentiable Interpolation-Based Renderer (DIB-R)
NVIDIA has introduced a rendering framework called the Differentiable Interpolation-based Renderer (DIB-R), which has the potential to improve different areas of 3D design and robotics.
DIB-R is capable of transforming 2D images into 3D models in seconds, by predicting the shape, color, texture, and lighting of an image. This is achieved by transforming the input from a 2D image into a map, which is then utilized to create a polygon sphere and generate a 3D model.
GPU Computing in 3D Printing
Daghan Cam, a former teaching fellow at the University College London’s Bartlett School of Architecture, utilized GPU computing to create a flawless 3D printed architecture.
He used his expertise with the CUDA parallel programming model and NVIDIA GPUs to teach his robotic fabrication system to utilize algorithms to finish his abstractly designed structures.
Conclusion
In conclusion, NVIDIA’s Magic3D is a revolutionary technology that enables users to transform 2D images into 3D models with ease. The technology is based on the NVIDIA rendering framework known as DIB-R, which is a differentiable interpolation-based renderer.
With the ability to render 3D models in seconds, NVIDIA’s Magic3D technology is poised to be a game-changer in the world of 3D printing.