Today, while I was coming back home from work, I had a discussion with a colleage about one of our most important game tools: our animation system.
Said system is very big and has many features, it's probably one of our greater efforts and I doubt there is something more advanced out there. Now, it's even becoming a sort of game rapid prototyping thing, and it supports a few scripting languages, plus an editor, written in another language.
To make all those components communicate properly takes quite a bit of code, and so we needed to create another component that somewhats facilitates connecting the others.While we were discussing about the merits of techniques such as code generation versus code parsing, to link together different languages, it was clear that what was needed was indeed some sort of reflection, and that having said reflection would remove the need of also other parts of code (i.e. serialization).
So I went back home, and started thinking about why we didn't have that. Well, surely the problem had to be historical. Right now looking at our design, the "right" solution was obvious, but I knew that system started really, really small and evolved over the years.I realized actually that we didn't have in general a standard reflection system...
That's rather odd, as in many companies when you start creating your code infrastructure, reflection ends being one of the core components, and everyone uses that, it's more like a language extension, one of the many things you have to code to make C++ look less broken. We didn't have anything like that. We really don't have a core, we don't have an infrastructure at all!
Lack of strategy. We do have a lot of code. A lot. Many different tools, you won't believe how many, and I think that noone really can even view them all. We keep all those modules and systems in different repositories, really in different ogranizational structures with different policies and owners... It's huge and it can look messy.
To overcome the lack of a real infrastructure, some studios have their own standards, maybe a common subset of this huge amount of code that has been approved and tested as the base of all the products that studio makes. Some other studios do not do that, some other again do it partially.
Are we stupid? It looks crazy. I started thinking about how we could do better. Maybe, instead of choosing a subset of technologies that make our core and gluing them together with some bridge code and some test code, we could make our own copies of what we needed, and actually modify the code to live together more nicely. Build our infrastructure by copying and pasting code, modifying it, and not caring about diverging from the original modules.
But then what? It would mean that everything we modify will live in its own world, we can't take updates made by others, and we can't take other modules that depend on one of the pieces that we modified. And every game, to leverage on this new core, had basically yo be rewritten! Even cleaning up the namespaces is impossible! No, it's not a way that could be practical, even if we had the resources to create a team working on that task for a couple of years.
What went wrong? Nothing really. As bad as it might look, we know that it's the product of years of decisions, all of which I'm sure (or most of them) were sane and done by the best experts in our fields. We are smart! But... in the end it doesn't look like it! I mean, if you start looking at the code it's obvious that there was no strategy, the different pieces of code were not made to live together.
Is it possible to do better? Not really, no. We know that in software development, designing is a joke. You can't gather requirements, or better, requirements are something that you have to continuosly monitor. They change, even during the lifetime of a single product. How could we design technology to be shared... it's impossible!
Your only hope is to do the best you can do in a product, and then in another, and start to observe. Maybe there is some functionality that is common across them, that can be ripped out and abstracted into something shareable. Instread of trying to solve a general problem, solve a specific one, and abstract when needed. Gather. That's sane, it's the only way to work.
But then you get to a point were something started in a project, got ripped out because it was a good idea to do so, and evolves on its own across projects. Then another studio in the opposite side of the world sees that component, thinks its cool and integrates it. Integrates it together with its own stuff, that followed a similar path. The paths of those two technologies were not made to work together, so for sure the won't be orthogonal, they won't play nice. There will be some bloat. And the more you make code, promote it to a shareable module, and integrate other modules, the more bloat you get. It's unavoidable, but it's the only thing that you could do.
So what? We're looking at a typical problem. Strong tactics, good local decision, that do not lead over time to strong strategy. It's like a weak computer chess player (or go, chess is too easy nowadays). What's the way out of this? Well... do as strong computer chess programs do! They evaluate tactics over time. They go very deep, and if they find that the results are crap, they prune that tree, they trash some of their tactical decisions and take others. Of course computer chess can go forward in time and then back, wasting only CPU time.
We can't go back, but we can still change our pieces on the chessboard. We can still see that a part of the picture is going wrong and delete it... at least if we took the only important design decision out there: making your code optional. That's the only thing you have to do, you have to be sure to work in an environment where decisions can be changed, code can be destroyed and replaced. Two paths, two different technologies after ten years intersect. Good. They intersect too much? They become bloated? You have to be able to start a new one, that leverages on the experience, on the exploration done. But that is possible only if everything else does not deeply depend on those two.
Tactics are good. Tactics are your only option, in general. If you're small, have little code, have a few programmers, then you might live in the illusion that you can have a strategy. You're not, it's only that strong tactics at that size, look like a strategy. It's like playing chess on a smaller board, the same computer player that seemed weak, becomes stronger (even more clear again, with go) *. And of course that's not bad.
Some design is not bad, drawing the overall idea of where you could be going... Like implementing some smarter heuristics for chess. It's useful, but you don't live with the idea that it's going to be the solution. It can improve your situation by a small factor, but overall you will still need to bruteforce, to have iterations, to let things evolve. Eventually, over the years, relying on smart design decision is not what is going to make a difference. They will turn bad. You have to rely on the idea that tactics can become strategy. And to do that, you have to be prepared to replace them, without feeling guilty. You've explored the space, you've gathered information (Metropolis sampling is smart).
---
* Note: that's also why a lot of people, smart people, do not believe me when I say that stuff like iteration, fast iteration, refactoring, dependency elimination, languages and infrastructures that support those concepts, are better than a-priori design, UML and such. They have experience of too little worlds (or times). I really used to think in the same way, and even now it's very hard for me to just go and prototype, to ignore the (useless and not achievable) beauty of a perfect design drawn on a small piece of paper. We go to a company, or get involved into a project, or have experience of a piece of code. We see that there is a lot of crap. And that we could easily have done better! Bad decisions everywhere, those people must be stupid (well... sometimes they are, I mean, some bad decisions were just bad of course). Then we make our new system, we trash the old shit, and live happily. If the system is small enough, and the period of time we worked on it is small enough, we will actually feel we won... We didn't, we maybe took the right next move, a smart tactical decision. I hope that it didn't took too long to make it... because anyway, that's far from winning the match! But it's enough to make us care way too much about how to take that decision, how to make that next move, and not see that the real match does not care much about that, that they are not even fighting the big problem. It's really hard to understand all that, I've been lucky in my career, as I got the opportunity to see the problems at many different scales.
