Qt 3D development has been coming along nicely and is scheduled to be released with Qt 5.7. One of the issues we face with offering a high-level 3D rendering API is facilitating the asset conditioning pipeline. When dealing with 3D assets, there is a need to process the assets created in digital content creation applications like Maya, 3ds Max, or Blender into a format that makes sense for the rendering engine. This is analogous to 2D asset conditioning pipeline where a designer creates an image in Adobe Illustrator (an .ai file), and before you use it in your application you convert it to PNG format.
What is an Asset Conditioning Pipeline?
Before diving into the implementation details of Qt 3D its worth explaining what an asset conditioning pipeline is. Assets created in digital content creation applications are typically stored in data formats that reflect the needs and capabilities of that tool, and not the application you intend to use them. The saved file will likely contain an abundance of extra data that is not relevant to your application leading to a larger storage need, as well as more time to load and put into use at runtime. As an example, if we look at the data stored in a Blender file (.blend) in addition to containing geometry data, there is information specific to the Blender editor and renderer. Information like the layouts of the editor, undo stack, modifiers, levels of detail etc. There is also other really expensive things like high polygon models that are sculpted and used for baking additional assets like normal maps. So even for a very simple model the file saved by the editor could have much unnecessary data that just takes up space in your deployment bundle, and in some cases the data it has is not in the most efficient format for how we will use it.
Many 3D content creation tools have the capability to export into file interchange formats like COLLADA. These exporters are useful in the sense that they can strip out much of the excess data that is tool specific. But while formats like COLLADA makes it easier to transfer 3D assets between content creation tools, they still don’t store data in a format facilitating efficient loading and usage. In addition, getting assets in the desired format is just not always possible. For instance, receiving a 40 MB car model in OBJ format from the design department is less than ideal, to put it mildly, but sometimes unavoidable in practice. It would be nice if developers could progress with the project regardless, without having to deal with asset loading times of several minutes.
Given those issues, the asset conditioning pipeline is responsible for transforming that asset file into a format that is useful for that intended task. So stripping the asset of unnecessary data, and converting what is left to a format that can be easily and efficiently used by the rendering engine.
How does Qt 3D facilitate the asset conditioning pipeline?
Qt 3D is quite flexible in what assets can be used at runtime. It is now possible to use any of the above formats directly in your application at runtime, despite the inefficiency. This is enabled by use of the Open Asset Import Library (libassimp). We bundle this as a 3rd party library with the Qt 3D module and provide a scene parser plugin that uses it to load a Qt 3D scene from an asset file. This approach can be seen as using non-conditioned assets, or runtime conditioned assets, and will not be ideal for resource sensitive applications, especially on mobile and embedded platforms.
In addition to the Assimp scene parser plugin, we also have a plugin for loading GL Transmission Format (glTF) assets. The glTF format like the COLLADA format is a Khronos-defined (draft) standard, but unlike COLLADA, glTF is designed for the efficient transmission and loading of 3D scenes and models for applications. Assets bundled as glTF have geometry data, textures, and shaders all bundled in such a way that it is easy for Qt 3D to upload those assets to the GPU, and make use of them in the Qt 3D scene. Because of this we choose glTF as the format to use for our asset conditioning tool.
Introducing qgltf
To get a quick overview, you may want to watch the recording of the lightning talk from Qt World Summit 2015 in Berlin:
The Qt tool qgltf is an application that is bundled with the Qt 3D module. It is a host tool in the same way as qmake or moc, in that is built and run on the host machine. The purpose of qgltf is to take an arbitrary 3D asset created with a digital content creation tool, and convert to glTF which will be loaded into a Qt 3D scene by the above mentioned glTF scene parser plugin. By using the bundled Open Asset Importer Library in the qgltf tool, we are able to convert the same files Assimp scene parser can load into glTF for efficient loading. This also means that the number of dependencies for the Qt libraries on the target system can be reduced. This can be important for embedded systems in particular: there is no need to ship potentially large 3rd party libraries together with Qt when the assets are baked into a common, efficient format during build time.
There are existing converter tools (for example COLLADA2GLTF) available that can convert 3D assets to glTF, and in some cases the output of these will be usable as-is by the Qt 3D’s glTF scene parser plugin. However, besides the obvious issue of COLLADA not being the only commonly used format, glTF’s focus is on WebGL at the moment. In Qt 3D we have support for all current OpenGL and OpenGL ES versions, which means that shaders for a given GLSL version built into the assets can be problematic. More importantly, it will often be desirable to use Qt 3D’s built-in materials instead of providing custom shaders (generated by the converter tool) for every asset. This way models and scenes can participate in Qt 3D’s standard lighting system. Therefore, in addition to a set of basic built-in shaders, qgltf promotes the use of extensions like KHR_materials_common which allow models to use Qt 3D’s own standard materials instead of asset-provided shaders and technique descriptions.
By using the qgltf tool we also have the possibility to bring further optimizations to the generated files. Assets in glTF typically consist of several different standalone files. The main asset file is the .gltf file which is a JSON file can that actually describes the scene itself. With qgltf we can output this as a binary JSON file, or even a compressed version of that to get even more efficient loading times. Generating tangent vectors and scaling vertices during build time are also supported. In case Qt 3D’s built-in materials are not deemed suitable, qgltf can generate techniques and shaders both for GLSL 1.00 and version 150 (suitable for OpenGL 3.2+ core profile contexts). Another experimental feature qgltf can provide is automatic compression of texture assets. So if you have several texture files that are in PNG format, and you are deploying to a device that supports ETC1 compressed images, qgltf will make sure the asset it generates also contains and uses those compressed textures.
It is also important to point out that not every possible feature of glTF is in use: things that are better defined through Qt’s, QML’s and Qt 3D’s own powerful facilities are left out from the baked assets. This includes lights and animations (skeletal animations are on the roadmap for future releases, though). So in short, the tool’s focus is on converting and baking the parts of the assets that matter, nothing more, nothing less, into a format that is most efficient to deploy and load at runtime, even on I/O-bound embedded systems.
Usage of qgltf
Keeping with our tradition of flexibility, you also have a few options on how you can use the qgltf tool to best fit your workflow. First can use it as a standalone command line tool that you can include in your existing asset import pipeline. If you have the Qt 3D module installed alongside your Qt installation the qgltf tool exists at the same path as the other Qt tools (qmake, moc, uic). If you are using qmake as your build system you also have the possibility to have qgltf run automatically when your application builds. To do this you simply need to list your 3D model assets as followed in one of your qmake project files:
# 3D assets QT3D_MODELS += assets/3d/car.obj # Parameters for qgltf (use binary JSON and rely on Qt 3D's built-in materials) QGLTF_PARAMS = -b -S # load the qgltf qmake feature load(qgltf)
This will convert the OBJ file into a glTF format file, and add it to a Qt resource file (.qrc) together with all the geometry and image files. To make use of that model in your Qt 3D application you just need to load the generated file by setting it as the source in your SceneLoader. Everything else is managed automatically by the build system.
SceneLoader {
source: “qrc:/models/car.qgltf”
}
What does the future hold?
Qt 3D is still quite new and under heavy development so there are still many possibilities that can be explored with regards to asset conditioning and other tooling. We could for example have additional measures in qgltf that optimize the original mesh when possible. We have also been looking into the possibility of generating a QML file along with the glTF asset that can facilitate the loading and easy access of elements (sub-meshes, materials, etc.) within the models and the scene. And of course if you have any feedback or ideas we would love to hear them.
The post Qt 3D: The Asset Conditioning Pipeline appeared first on Qt Blog.