Improved 3D Workflows with AI
Artificial intelligence is transforming creative workflows at an incredible pace, and 3D modelling is no exception. Every few months a new AI model arrives promising better quality, faster generation and more practical workflows. Naturally, we wanted to see whether these tools could genuinely improve our own VR modelling pipeline.
To put them to the test, we experimented with Meshy, using it to generate 3D models from reference images. The goal wasn’t simply to create impressive-looking assets, but to determine whether AI-generated models could become part of a professional production pipeline for immersive VR training applications.
The results were encouraging, but they also highlighted where human artists still make all the difference.
Starting with Shape
The first thing we looked at was how accurately Meshy could recreate the overall shape of an object from a reference image.
Using Meshy 6, the results were surprisingly impressive. The generated models closely resembled the original object, capturing the overall proportions and major features remarkably well. There were occasional inaccuracies, particularly where the AI had to infer details that weren’t visible in the reference image, but for a first pass the output was more than usable.
One particularly useful feature is the ability to generate multiple variations from the same reference, making it easy to select the strongest result. We also expect that supplying additional reference images from multiple angles would improve the accuracy even further.
For creating a high-poly starting point, AI performs exceptionally well.
The Polycount Problem
Unfortunately, a great-looking model isn’t necessarily a usable one.
VR applications require highly optimised assets, typically somewhere between 2,000 and 5,000 polygons. The models generated by Meshy, however, often exceeded 60,000 polygons, making them far too expensive for real-time use.
Meshy includes automatic polygon reduction tools and can even bake normal maps from the original high-resolution mesh. While this certainly helps, the results are mixed.
Like most automated optimisation tools, the algorithm doesn’t always know which areas deserve extra detail. Human artists instinctively preserve silhouettes and important forms, ensuring circular objects remain round and mechanical parts retain their defining features. Automatic reduction can sometimes simplify these details too aggressively, leaving previously clean geometry looking distorted. It also implements a blanket consistent polygon wrap, where Human artists would be more selective and not use polys where they aren’t needed, Every polygon saved is a step in the right direction when Framerate is considered.
In many cases we’ve achieved better results by treating the AI model as a high-poly source and manually rebuilding an optimised low-poly version before baking the high-resolution detail into normal maps. This hybrid workflow produces assets that are both visually convincing and performant enough for VR.
Where Texturing Starts to Fall Short
The generated textures themselves are often surprisingly good, but the biggest weakness lies in the UV mapping.
For anyone unfamiliar with the process, UV unwrapping is essentially flattening a 3D model into a 2D layout so textures can be applied correctly. A good UV layout ensures consistent texel density, clean seams and minimal texture stretching.
Currently, AI-generated UVs tend to resemble simple automatic projections rather than carefully planned layouts. This can introduce stretched textures, poorly placed seams and inconsistencies between different materials.
Fortunately, this is an area that’s likely to improve rapidly as AI tools continue to evolve.
In our workflow, spending around thirty minutes refining the textures in Substance Painter is usually enough to correct labels, improve material transitions and fix the small imperfections that AI currently struggles with.
Materials Are Good… But Not Great
Meshy also generates physically based rendering (PBR) maps including metallic, roughness and ambient occlusion textures.
These provide a solid foundation and successfully distinguish between different materials such as metal and plastic. However, they lack the subtle detail that gives assets realism.
Surface wear, scratches, fingerprints, dirt build-up and other environmental storytelling elements are still largely absent. These finishing touches remain much easier to create manually in dedicated texturing software.
Is AI Faster?
Without question.
The AI-generated version of our test model took less than 15 minutes to produce. Creating an equivalent model entirely by hand would likely require around four hours of modelling before texturing even begins. That’s an enormous productivity gain. However, speed doesn’t eliminate the need for skilled artists. Instead, it changes where their time is spent.
Our Current Workflow
Rather than replacing traditional modelling, we’ve found that AI works best as the first stage of the process.
We now favour a hybrid workflow:
- Generate a high-poly concept model with AI.
- Use that model as a reference for building an optimised low-poly asset.
- Bake the high-resolution detail into normal maps.
- Refine the textures and materials manually.
- Prepare the asset correctly for use inside a game engine.
This approach combines the speed of AI with the control and optimisation that experienced artists provide. There are still limitations that AI doesn’t fully understand. Scale, for example, is often inconsistent, and complex assets made up of multiple materials and repeated components still require human planning. Large industrial equipment, for instance, needs to be broken into logical texture sets and reusable assets, something current AI tools simply don’t account for.
Final Thoughts
AI modelling has already become an incredibly valuable production tool, but it’s not yet a replacement for experienced 3D artists.
What it excels at is accelerating the early stages of asset creation. It removes much of the repetitive work involved in creating an initial high-poly model, allowing artists to focus their expertise on optimisation, texturing and polish.
As these tools continue to improve, that balance will undoubtedly shift. For now, though, the biggest gains come from combining AI with traditional workflows rather than choosing one over the other.
The future of 3D modelling isn’t AI versus artists, it’s AI working alongside artists to create better assets in less time.