Next-gen and realism.

First of all, "real" in next-gen games is not usually real as in real world. Is more real as in film versions of the reality. And it's obvious, we want to express our view, not the bare truth. Also it's very fortunate, because we can get creative.

Physically based or not? Is Blinn still right? What looks good, is good in Computer Graphics?
Of course he is. Graphics is about look. But we should care about physics not because we are not able to achieve the correct look by fakes (and there are ps2 games that can prove that), but because fakes are usually hard.

Cheats will always be done. We don't have enough power to not cheat, even considering the simple models of light that are used in nowdays offline renderers. But now doing "the right thing" is a great tool that helps a lot. Ease of use, was the main reason behind Global Illumination in the first place, and now it's the same for the next generation unbiased renderers. It's just more convenient to work in the most accurate light model that you can simulate, because it will usually look right without much tuning.

But always remember also that you should empower artists, don't take it to the extremes. Artists are very good at tweaking stuff, so keep usability always in mind. A fully automated GI solution can be a nightmare for artists if they can't bend the lighting model to their needs, when they need to.

So start with physics, then add hack-ability on that.

In the end, Blinn's motto nowdays has also different meaning to me. It tells me to care about perception and phsicology (uncanny valley? Crysis had to address that problem for example, we are there), as we're getting close to the limit where those components are really important.

You CAN'T afford doing things in the wrong way.

Game development is hard. Design is, by its nature, continously subject to changes. Competition is very high. The required investments also. You work on the bleeding edge of the technology. You need to have skills in many different fields. Many times missing deadlines is not an option, or it's very expensive and you have to deliver an high quality product as well in order to be able to compete.

It's really hard enough. You don't want to add on top of that a bad development process. You have to do the things in the best possible way, to manage to get everything working. And sometimes the temptation of avoiding good practices is high, due to the pressure you have expecially towards the end of a project. Some other times your company/game/product simply does not start in the right way. And then everything will be messy and a waste of time/money.

Change is not avoidable. You have to be prepared for it. You really want to identify probles as soon as possilbe.

• Plan every major feature. Gather requirements. Gather references. Plan possible implementations. Try to identify risks. Make fuzzy time estimates. Refine those estimate until you're confident enough about them.
  
• If you don't have enough information to be confident about your fetaure plan, stop and GATHER IT. Make early prototypes. Don't blindly try to do something and hope that everything turns out right. It WON'T.
• AGAIN: Don't do things blindly. Gather information. Make prototypes. This is not related to programming only. In example, you want to strive for a given lighting in your game. You're not sure about some quantities like the number of light sources, light types, shadow casting direction, shadow casting object, shadow receiving objects, etc... Then do a prototype with some test renderings directly in your 3d authoring application. Prototype possible solutions, see if it's reasonable to expect that your assumptions are right.
  
• Investigate about how many areas a feature will impact.
  
• Allocate your technical budgets for each major feature early (i.e. CPU time, GPU time, memory etc).
• Don't enter production if the planning phase is not done. You will have enough changes to deal with anyway. Don't start the real product if you've not investigated and planned all the major new features. Prototype everything that has an high risk.
  
• Be incremental. Work in small iterations. THINK. Develop. Test. Refine/refactor.
• Always test. Test as soon as it's possible. Run batch tests. Make a working game as soon as it's possible. Do early QA tests on every newly implemented game feature. Gather information.
• First priority is to have something working. You can't break the work flow. Spend extra time to make sure that your feature is working. Don't rush.

And, more day-to-day coding related:

• Strive for code quality. Do reviews. Use linting tools. Adhere to good coding standards. Refactor.
• Design for change. Design for flexibility. Try to have as few interdependencies as possible. Try to be always able to trash a given code and rewrite it. It will happen. It SHOULD happen, as code WILL rot, no piece of code is immortal. This is expecially true in games, with fast moving technology and always changing requirements/scenarios.
  
• Keep the documentation updated. Run a internal WIKI. It has to be EASY to write the documentation. Documentation tasks should be allocated for any new major technological component. Comment your code. Comment your HACKS, you will do them at the end of the project, but at least you should comment them so you can remove them later.
• Before starting a new project on an old code base, clean the code base. KILL any dead code. Kill last minute hacks. Search for old "todo" items.
  
• Try to do the right thing. Don't cheat too much. Cheats are hard to maintain.
  
• Think about your tools. Tools are hard. Tools are the key of productivity and are the main mean of interaction between the coding and graphics department. If you can't affod to do many in house tools, find a way to leverage on existing technologies and be minimal. Don't build huge tools that you can't affod to refine and maintain. A bad tool WILL kill a good feature.

If there are some items on that list that you were NOT doing during game development, then probably you're doing the wrong thing (or I wrote something stupid). If you do the wrong thing you will end up loosing money. If you loose too much money, your project will fail.

Faking nextgen

Another old article from the web, the making of Shadow of the Colossus (alt. link). Very intereting tricks, another example of how oldgen knowledge is always useful (and also think about WII and PSP...)

I missed that...

...the first time I've read about Capcom's MT engine, so it's not exactly new, but as I was reading again MT documents around the web for my previous article, I've found the following nice trick (here):

MSAA trickery. Another thing the Capcom presentation talks about is their use of MSAA trickery, for increased speed. On the consoles, where you have lower-level access to frame buffer formats, etc. you can do wacky stuff like pretending a 1280×720 (720p) no-AA screen is a 640×360 (ordered grid) 4xMSAA screen. This works, because these would have the same pixel dimensions. By drawing e.g. particles to the 640×360 4xMSAA screen instead of the 720p screen you would reduce your fragment shader computation to 1/4 (as the fragment shader is only executed once per pixel for multisampling), while still rendering to the same pixels as you would have if drawing into the 720p buffer. This is a way of trading fidelity for speed (or vice versa) and it is a very nifty trick.

Concurrency part 2

When you have sorted out the concurrency problems for CPU threads, and established a way to safely generate data in CPU for the GPU, set up your object pipelines, you will end up hitting the problem of multithreaded draw calls.

At the moment, such a thing is not possible in any mainstream platform, you have to issue all the draw calls from a single thread that owns the rendering device. The usual solution to this problem is to bake command buffers (display lists in openGl terminology) on the non-rendering threads and then pass them to the rendering one that draws them.

The problem with this approach is that you can't sort/optimize/wathever primitives between threads, all the draw calls are baked and just copied in the main ring buffer. Of course you can always organize your rendering objects in lists that are bound to a given renderbuffer/pass and process those lists in parallel, doing so you're pretty sure that you can't do more optimization than the ones you can perform on a single list. A problem arises when you have a few big lists and all the other ones are much smaller, in that case, processing one list per thread does not give you an optimal load balancing. So other solutions could be more employed, depending on the context.

A very common one is to have some higher level rendering data, usually meshes with materials and all the context needed to do draw that data. Those primitives/contextes are added by the various threads in a render queue that is then sorted (by rendertarget, passes, materials etc...) and generates state changes and draw calls. The state API is hidden and used only by the rendering thread. Using a lockfree stack helps.

Another interesting solution is the one employed by Capcom's MT engine (Lost Planet). It's like a cross-platform command buffer API, where the commands have hints on their ordering (rendertarget, pass, etc) and are issued in parallel by multiple threads, then sorted in each thread, gathered and merge-sorted togheter in the rendering thread and then converted in actual draw calls. This is somewhat an hybrid approach between an high level submission API and a native commandbuffer API, when you can still do every kind of inter-thread optimization, but in a very fast way, without hiding the state API and doing only a simple translation of the commands to the native API ones in the main thread.

Read more about this solution here. Notes and links to the MT engine here.