What I focused on
As we needed to build a 3D game engine during this project i focused a lot on the deeper functionalities. While I worked on more areas than I will showcase here, these are the areas that i spent the most time on / wrote on my own.
Serialization
One of the first areas I focused on was the workflow between Unity, our external level editor, and our own in-house engine. I was worried about this as I had heard that some of the previous year’s groups had massive pipeline issues where some of *the programmers had spent some 20% of their time in unity to create the necessary components, and subsequent export of them. This is an obvious issue as our groups are small and losing a fifth of even one programmer’s time would result in a lot less features in our games.
I started by experimenting with an JSON based export tool that uses C# built in JSON serializer to automatically export our custom components. While functional, the load speeds were nothing to be proud of as loading scenes with over 4000 objects would take a couple of seconds if un-threaded, additionally I wanted to minimize the amount of C++ code that we needed to write in the engine and this system would require manual serialization code, and I wanted it to be able to load a type from one line of code, as this minimizes potential bugs where someone forgets to add a loader for a variable.
After a discussion with one of my teachers, I started to experiment with a binary serialization method where a single memcpy would be able to load an array of components, resulting in extremely fast load times and no manual serialization needed if the types match the size of C#. However, binary serialization has some other issues, primarily that loading different versions of components, say if a new variable was added to a component then that component would not be able to load as it won’t match the version that It was exported to the file. This would stop our level designers from exporting levels with that component unless the C# version of that component in Unity was updated to match the main engine component.
I decided to solve this issue by creating a tool, that our Level designers would be able to run inside of Unity, which would scan the C++ components, convert the variable types into the C# equivalent, and create or update the existing components in Unity. This would, in-theory, completely remove the necessity for a programmer to manually create components in Unity.
An example of this conversion would be as follows:
With the binary loader and a functional automatic component import tool for Unity, the workflow that I created was nearly fully automized. One of the few areas that could not be automated using this tool was the conversion of Unity’s built-in components, such as the Transform or the Light component where we needed them to work as normal in Unity so that our Level designers could preview how the level should look in unity, but still export to our engine. We ended up writing these manually as we only need to write them once per built-in component.
Pathfinding and navmesh generation
During this project I implemented RecastNavigation navmesh builder, pathfinding using A* and string pulling.
As we were using unity as an external level editor and unity has the capability of generating navmeshes, we started by simply exporting the generated navmesh as a triangulated mesh. This isn’t optimal, as A* can be optimized to use a lot more than triangles for its pathfinding. But it worked in the beginning as we simply needed very basic AI. As we moved into the beta sprint however we needed better AI’s that would, most importantly, look smarter. As we were on a limited schedule I decided against creating my own polygon generating navmesh and instead decided to implement RecastNavigation to re-generate a more precise polygon navmesh from data that we extract from unity. This allows for quite precise navmesh generation, with the level designers being able to preview how the navmesh was going to approximately look before they even export it.
After tweaking the generation values over the course of the game project I was able to get the generation quite accurate to what we wanted. That isn’t to say that we the implementation process was without fault. As it turned out, the unity navmesh exporter that I wrote tried to optimize the vertices creating weird results where the ground wasn’t near what we saw in unity. Here is an example of this, before we tweaked the generation values.
A bad navmesh
The final navmesh generator however, looked a lot better after we fixed the unity exported and tweaked the generation value:
A Better navmesh
In the final version, the main issue was with “slivers” in the navmesh, where there is a lot of polygons near each other. However, as our pathfinding system was able to overcome this without noticeable issues, I never looked at decreasing it more, as it was already better and nearly un-noticeable if you do not know where to look while playing.
Runtime navmesh compilation
One of the things that allowed us to end up with good navmeshes was the ability for our Level Designers to modify the navmesh generation values for the scene that they were building, while most scenes used the default settings, some were built with slight difference to the walkable height, etc. to make the navmesh entirely adjusted to the level that it was in.
Another feature I added, was the ability to dump the generated navmesh to a file. This was to reduce the loading times as generating a large map could take up to twenty seconds. As the file was dumped in binary its loaded and initialized nearly instantly allowing the final version of the game to have near zero loading times between levels.