Google reader share

So, some of you have noticed that my google reader share is dead (not updated anymore). That's not me being lazy, but google being a bit evil and trying to shove google+ down our throats by killing the old facilities instead of integrating plus into the existing stuff. So, there is no reader share anymore, and my iPad newsreader (reeder), which was responsible of most of the share posts, does not support google+ yet. Stay tuned.

Photoshop scripting - Cleartype for images

Left: bilinear, Right: bilinear with "cleartype"
note- the effect is configured for a "landscape" RGB pattern LCD

I always wanted to learn how to script Photoshop (what I learned is that it's a pain and the documentation sucks...), so yesterday I started googling and created a little script to emulate cleartype on images. Here is the source (it assumes that a rgb image is open in PS):

// 3x "cleartype" shrink script

var doc = app.activeDocument;

var docWidth = doc.width.as("px");

var docHeight = doc.height.as("px");

doc.flatten();

// let's go linear RGB

doc.bitsPerChannel = BitsPerChannelType.SIXTEEN;

doc.changeMode(ChangeMode.RGB);

// now that's a bit tricky... we have to go through an action, which has binary data... which I'm not sure it will be cross-platform

// it works on Photoshop CS3 on Win7...

function cTID(s) { return app.charIDToTypeID(s); };

function sTID(s) { return app.stringIDToTypeID(s); }; 

var desc6 = new ActionDescriptor();

var ref5 = new ActionReference();

ref5.putEnumerated( cTID('Dcmn'), cTID('Ordn'), cTID('Trgt') );

desc6.putReference( cTID('null'), ref5 );

desc6.putData( cTID('T   '), String.fromCharCode( 0, 0, 1, 236, 65, 68, 66, 69, 2, 16, 0, 0, 109, 110, 116, 114, 82, 71, 66, 32, 88, 89, 90, 32, 7, 219, 0, 10, 0, 22, 0, 19, 

0, 25, 0, 58, 97, 99, 115, 112, 65, 80, 80, 76, 0, 0, 0, 0, 110, 111, 110, 101, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 1, 0, 0, 246, 214, 0, 1, 0, 0, 0, 0, 211, 44, 65, 68, 66, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 9, 99, 112, 114, 116, 0, 0, 0, 240, 0, 0, 0, 50, 100, 101, 115, 99, 0, 0, 1, 36, 0, 0, 0, 101, 119, 116, 112, 116, 

0, 0, 1, 140, 0, 0, 0, 20, 114, 88, 89, 90, 0, 0, 1, 160, 0, 0, 0, 20, 103, 88, 89, 90, 0, 0, 1, 180, 0, 0, 0, 20, 

98, 88, 89, 90, 0, 0, 1, 200, 0, 0, 0, 20, 114, 84, 82, 67, 0, 0, 1, 220, 0, 0, 0, 14, 103, 84, 82, 67, 0, 0, 1, 220, 

0, 0, 0, 14, 98, 84, 82, 67, 0, 0, 1, 220, 0, 0, 0, 14, 116, 101, 120, 116, 0, 0, 0, 0, 67, 111, 112, 121, 114, 105, 103, 104, 

116, 32, 50, 48, 49, 49, 32, 65, 100, 111, 98, 101, 32, 83, 121, 115, 116, 101, 109, 115, 32, 73, 110, 99, 111, 114, 112, 111, 114, 97, 116, 101, 

100, 0, 0, 0, 100, 101, 115, 99, 0, 0, 0, 0, 0, 0, 0, 11, 67, 117, 115, 116, 111, 109, 32, 82, 71, 66, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 88, 89, 90, 32, 0, 0, 0, 0, 0, 0, 235, 194, 0, 1, 0, 0, 0, 1, 65, 50, 

88, 89, 90, 32, 0, 0, 0, 0, 0, 0, 97, 15, 0, 0, 36, 77, 255, 255, 255, 232, 88, 89, 90, 32, 0, 0, 0, 0, 0, 0, 103, 37, 

0, 0, 220, 208, 0, 0, 5, 29, 88, 89, 90, 32, 0, 0, 0, 0, 0, 0, 46, 162, 255, 255, 254, 227, 0, 0, 206, 39, 99, 117, 114, 118, 

0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 0, 0 ) );

desc6.putEnumerated( cTID('Inte'), cTID('Inte'), cTID('Clrm') );

desc6.putBoolean( cTID('MpBl'), true );

desc6.putBoolean( cTID('Dthr'), false );

desc6.putInteger( cTID('sdwM'), 2 );

executeAction( sTID('convertToProfile'), desc6, DialogModes.NO );

doc.backgroundLayer.applyGaussianBlur(0.75); // limit the frequency a bit to avoid too many fringes

doc.resizeImage(

UnitValue(docWidth, "px"),

UnitValue(docHeight / 3,"px"), 

null, 

ResampleMethod.BILINEAR // To-do: box filter (mosaic + nearest)

);

var unitValue = UnitValue(1, "px");

// RGB pattern, note that the nearest resize will take the center pixel, that's why red shifts by one and not zero

var redLayer = doc.backgroundLayer.duplicate();

redLayer.applyOffset(unitValue, 0, OffsetUndefinedAreas.WRAPAROUND);

var greenLayer = doc.backgroundLayer.duplicate();

greenLayer.applyOffset(-unitValue, 0, OffsetUndefinedAreas.WRAPAROUND);

var blueLayer = doc.backgroundLayer.duplicate();

blueLayer.applyOffset(-unitValue*2, 0, OffsetUndefinedAreas.WRAPAROUND);

doc.resizeImage( // Resize to "select" the RGB columns in the various layers

UnitValue(docWidth / 3, "px"), 

UnitValue(docHeight / 3,"px"), 

null, 

ResampleMethod.NEARESTNEIGHBOR

);

//var col = new SolidColor(); col.rgb.hexValue = "FF0000"; redLayer.photoFilter(col, 100, false);

redLayer.mixChannels ([[100,0,0,0],[0,0,0,0],[0,0,0,0]], false);

greenLayer.mixChannels ([[0,0,0,0],[0,100,0,0],[0,0,0,0]], false);

blueLayer.mixChannels ([[0,0,0,0],[0,0,0,0],[0,0,100,0]], false);

redLayer.blendMode = BlendMode.LINEARDODGE; // add!

greenLayer.blendMode = BlendMode.LINEARDODGE; // add!

// blue is the base layer

doc.flatten();

// let's go to 8bit sRGB

doc.convertProfile ("sRGB IEC61966-2.1", Intent.PERCEPTUAL, true, true);

doc.bitsPerChannel = BitsPerChannelType.EIGHT;

Silvio Berlusconi

This is obviously going to be off-topic with the rest of the blog... If you landed here for the first time, this is a rendering related blog and this article is an exception to the rule.

As the Europe's and Italy's financial crisis deepens, news crossed the wire today that prime minister Berlusconi vowed to resign. I see many people asking around the world how it was possible that this happens only now, how did Berlusconi manage to be in power for seventeen years even after countless scandals and accusations. While in general I think it's not surprising, and the eight years of Bush administration could be served as an example, I'd like to try to explain what's peculiar about Italy's situation (of course, in my point of view). 

Also, of course, I will make generalizations in the following. In no way I want to express that this is applies to everyone and everything, that should be pretty clear.

Survivalists

One way or another, we keep going on. This is by far what I believe to be the deepest of our problems. We don't care much about our society, we avert our eyes and keep going on, everyone trying to find a hole in which to live their lives. 

We are masters in bending (if there is a profit to be made) or ignoring laws. Even our image outside the country is that of creative, chaotic individuals (at best), known for being obsessed about family and our own small individualities. 

We are not socialists not liberals, we are just driven towards what can get us a gain tomorrow morning. Mind you, to a degree this happens everywhere, but it's not a defining quality of a population quite as it happens in Italy, where is deeply buried everywhere, from how people live their lives to how companies make business.

Even our economy, made mostly of small or family owned companies, with our comparatively large private savings and our huge public debt, is a testimony of this mentality.

I can't tell why this is the case, we are a young republic and unification was not a smooth deal, but we don't believe in society. Berlusconi is the embodiment of all this, and I don't know how much he was just "born this way" or how much he knowingly acts to please, to be popular, but he is certainly great in leveraging such sentiments. His political message, either explicitly or implicitly has always been "vote for me and I'll let you live your lives without control", "you don't need to be responsible for your actions", "I was successful, don't ask me how, you want to be like me, I won't ask you how"...
Berlusconi was never a left or right wing politician, he is obsessed about communism and certainly sees Italy's leftists as pure evil, but his actions are not the ones of a liberal. Among other things, he's remembered for saying openly at an entrepreneurs' convention  that companies without off-shore operations were not led smartly, and that evading taxes (one of Italy's chief problems) was morally sound in a country like ours (in which taxes are too high). He didn't them proceed to lower the taxes and impose strict controls to have everyone paying the right amount, or to incentive competition and freedom of enterprise. 

In the first months of his government he proceeded in the opposite direction, abrogating liberalization laws that were passed by the previous government, loosening controls over financial transactions and not reducing a single tax. Not touching established interests, not reducing bureaucracy, but just allowing people to just screw each other more freely.

Shameless

Berlusconi is probably not the worst individual in Italy's history. Corruption and misgovernment were always there and can be even tracked to the same underlying sentiment. The first republic created a massive debt because political parties were quite literally buying votes by flushing enormous amounts of public money into all kinds of public ventures, creating hundreds of thousands of "fake" jobs, public employers who were pretty much useless. But it came down onto his knees when it was found that politicians also used public money to fund their own parties. There was corruption, but there is was still a sense of shame. 

Berlusconi took inspiration from this and pushed it one step further, he was proud of his tricks, every trial he escaped by passing laws in his own favor, every lie and joke he said made him look "smarter", more successful. It's not that the vast majority of Italians do not know he was a thief, that's also why many of his voters were even shy of saying so, especially if singled out. 

It's that deep down, they knew, but they admired his skill, they wanted to live the same dream, to take the easy way and just not have to care. That's also why for years even after all the sex scandals he managed to keep a mostly Catholic country under control. That's why he still now has a huge following...

Media and Opposition

These two aspects were also important and I'm sure there is a lot more to be said, but I think they don't contribute to explain the Berlusconi phenomenon quite as understanding how much deeply he connects with some Italian sentiments.

Berlusconi owns most of Italy's media and he is renown to be a great communicator. He lies, but his lies are so constant and so fiercely defended by so many, that they slowly become truths. Words slowly lose their meaning and the public becomes divided into factions who do not reason but just mindlessly cheer for one or the other party.

Again, that's partially a "quality" of Italians, being more emotional than rational, being hot headed and profoundly divided. But he managed to exploit that incredibly well.

His power does not extend over only media of course, most if not everyone in his party is strongly tied to him, he choose men with little political past and respectability of their own, people who depended on him to be elected. Berlusconi IS his party, and everyone sings the song he sings. And he made pretty clear that was the way from the start, his party always had direct references to him in the logo, in the hymns, everywhere. Everyone laughs at his jokes. Everyone follows the same rules, tells the sames words, uses the same dialectic tricks. I'm not sure if it's imitation or doctrine but it's powerful.

On the other hand the opposition is fragmented and largely seen as made of intellectuals  and professional politicians who do not have any connection with the people. (which to a degree can be even very true). They were always bad communicators so it was easy for Berlusconi to play them, routinely saying that there was no better alternative than him, that the left-wing was made of communists that had no real plan other than raising the taxes (even if financial pressure increases or decreases have not really been strongly linked to any particular government). Furthermore they showed no cohesion, being unable to claim even the huge victories they sometimes achieved (Italy was able to enter the Euro as one of the founding parters due to the work of a left-wing government for example) and not being able to look past their divisions.

Open questions - my two rules

As I wrote here, there are some fundamental questions in realtime rendering that I wish I knew more about. I do have though two  rules I apply when thinking about rendering techniques

1. Reduce variance: It's better to be consistent at a lower quality than have glitches/flickering artifacts at a higher quality.
• Postulate: all graphical effects should be reviewed in motion, crossing quality boundaries
2. Less is more: It's better to not have a given effect than have it at a too low quality level.

As an example we can analyze shadows. The first rule tells us that it's better to have stable cascades at a lower resolution than having perspective shadowmaps at high resolution. 

The second rule tells us that it's better to have lower filtering and cull shadows from some objects or limit the shadowing maximum distance, than having bad shadows everywhere.


Unrelated, I just saw this as a job opening at Valve... Smart guys!

Psychologist

We believe that the more we know about human behavior, about how and why people do what they do, the better our products will be. All game designers are, in a sense, experimental psychologists. That’s why we’re looking for a experimental psychologist to apply knowledge and methodologies from psychology to game design and all aspects of Valve’s operations. We want to exploit your experience with experimental design, research methods, statistics, and human behavior to help craft even more compelling gameplay experiences for future Valve titles. We’d also expect you to research and weigh in on any and all topics that are relevant to improving the experiences of our customers, partners, and employees.

Duties:

• Provide relevant insight into human behavior in order to shape gameplay and customer experience.
• Perform statistical analyses on all aspects of Valve’s operations: gameplay, financial, and company data.
• Research compelling new hardware technologies.
• Design experiments to evaluate various gameplay hypotheses and design choices.
• Improve existing playtesting methodologies while incorporating novel techniques to improve best practices.
• Develop innovative ways of acquiring relevant data to answer open questions about all aspects of Valve’s products and business practices.

Requirements:

• Graduate degree in Psychology (or equivalent) field
• Advanced knowledge of statistics
• Familiarity with one or more of the following pieces of data analysis software: SPSS, Systat, Matlab, R, (or equivalent)
• Four years experience with:
  • Experimental design/research methods
  • Relevant research in cognitive, social, human factors, and related disciplines in psychology

Recommended:

• Proficiency in one or more of the following programming languages: C++, SQL, PHP, (or equivalent)

Open questions

From the series "End of the virtual world" by Robert Overweg

Battlefield, Mass Effect, Modern Warfare, Red Dead, Crysis, Rage, Forza, GT, Alan Wake...

What makes the graphics work? I've played the last three call of duty games on 360, with a projector and a 5.1 system. I found them to be amazing. Then I downloaded MW2 on steam, and the graphics looked mediocre.

Conversely, Mass Effect 2 seemed decent on 360 but way more awesome on PC.

Red Dead pushes less polygons and has more visual defects than Crysis but awed me in a way no other game of this generation did.

Battlefield 3 is a technical jewel but MW seemed to me to have a better atmosphere.

Why? Is my subjective judgment shared by others? Is my memory failing me and have my expectations shifted? Or there is also something technical behind these impressions?

MW textures on PC have the low-res quality of a bad port. Is this, coupled with the increase in resolution, the reason the game looks worse to me on PC, makes my brain "see" the polygons more? Or is it because its atmosphere is better suited for a projector and a couch? Or maybe it's because playing the game I was more immersed into it than when I just looked at the graphics on the PC.

Why was it the opposite for ME2? Is its art style, less reliant on gritty texture detail, making better use of the high resolution and antialiasing of the PC?

Why I enjoyed more ME2 single player graphics than BF3 ones? Are the graphics enhancing the gameplay, or is also the opposite true, game and story do affect the perception of the graphics?

Is it the aliasing that is killing BF? Or is the inability of deferred lighting to express the subtleness of light transport and materials the issue? Is our industry jumping on the deferred lighting approach too fast, without really understanding what it's losing form precomputed lighting?

What about the heavy bloom and flares that BF3 and Crysis2 use? How are they working? How do they alter the perception of the image?

From my experience I observed some patterns, but I don't really know much, I've also found very little research...

Aliasing and other high frequency artifacts quickly tell your brain that it's looking at CG, they are very disturbing. Motionblur at 30fps looks more cinematic and packs more punch than 60fps without blur. We tolerate framerate problems way more if the game looks busy (i.e. A huge explosion) than if they are not connected with game actions. Colour is hard to get right, and ambient lighting and proper occlusion of lighting terms are important to represent volume, rendering the air (haze, fog, scattering, desaturation tricks etc) helps with scale. Crowd variety is achieved more with colour and behavior variation than texture and model. Specular lobes are everywhere, have always fresnel, and we can't recognize errors in the light directions in the specular but we use high-gloss highlights to evaluate shape. Bleeding dark edges (i.e. when dealing with subsampled effects) looks less questionable than having bright halos.

There, I think I didn't miss anything, that's pretty much all I know, it fills a few lines of text. I think that's a big challenge for us, there are more studios that know how to do deferred lighting right and fast than there are studios that know what's important to create an immersive, beautiful game. We don't know what to focus on where, what devices are used for what, which artifacts are tolerable and which are disturbing.

We know the (sometimes) the physics but we have very little math to model the psychology. Yes, KSK fits skin specular well. But what parts of it are important? When does it break and breaks the perception of skin being skin? We need to make hacks, and physically reasonable hacks are fundamental in our line of business, physics are fundamental, but physiologically motivated hacks would be way better!
This also affects all the "tuning" decisions, i.e. more LOD switching but with better detail near the camera, or vice-versa? Geometry or textures? Bigger SSAO radius and more noise or the opposite and so on.
Moreover linking perception (vision) and psychology with rendering would give us more objective tools for art-direction, like what device is the best to convey in the general audience a given intent, what makes a sense of "scale" or of "fear" and so on...

Rendering without knowing about perception is crazy, it's like if musicians knew more about sound waves and instruments than harmony and melody. And yet we often chose rendering techniques based on really faint leads about what is needed to look good. There is little research, and the little there is focuses on issues that seldom are directly applicable to modern videogame rendering techniques.
Even worse, we are just not starting to understand the basics, like color and normals, and not only in the industry but even in most publications you will find little regard towards even basic visual perception metrics.


Videogame rendering today at its best it's a work of iteration, the more you can try and the better feedback you get, the more you inch towards this ill-defined target of visual splendor. But even artists and art-directors with a great eye for light and colour seldom have much experience about realtime CG artifacts and their impact on perception.


Shifting from art to science has to happen in order for our profession to evolve, we can't rely on art direction for technical problems, it's not only too error-prone but also very inefficient. Scientific studies can be shared and described exactly, while art direction remains subjective and does not result in a shared progress.


It of course not something that happens only in rendering (or presentation in general, animation, audio), we even make games with very faint leads about what is needed to make fun. And that's why often you have studios investing big amounts of money, and staffed with great talents, producing results that are impressive but still fail, while only very few games really know how to be fun, and really know how to immerse players in their art...

More to come... Meanwhile you might want to read something (other than the links I've already scattered in the post): Some interesting reads here. Also Holly Rushmeier's work (some is linked in the post - from the website the EG2001 and EG2003 presentations are very neat)
Please comment if you know more resources on the topic. A question has been posted on Quora here...