Search this blog

14 June, 2008

Pet projects

Many, many programmers that have real jobs also manage to find free time for pursuing personal coding projects.

That's not my case, as I do have a lot of free time, but manage to waste it in numerous other ways (right now they are: playing rockband, playing lego indiana jones, watching montalbano or or southpark, reading books and papers, writing this blog, sleeping, doing photography).
Among my longest standing projects that weren't I can find: a SIMD raytracer, better if intersecting isosurfaces or something equally exothic and implementing Metropolis light transport (so I don't have to compare it to all the other, more useful, triangle-mesh based raytracers), a shader prototyping framework/DSL (to replace FX Composer) and many other ones, mostly computer art related. Sometime I do some shader programming at home, but anything more complex than that seems not to be able to overcome my laziness.

But assuming you're smarter than me, you might be interested in starting one. But hey, personal projects are about having fun, and learning new things, so why doing it in a boring language-that-you-already-know if you can do it in a new, exciting language? That's an incredible deal: you'll learn a language, you'll finish your project faster (if you choose the language wisely) and you'll probably learn new programming paradigms that will extend your "design pattern" (!!!) library.

What if you don't have any idea about an interesting project to do? Well, fear not my friend. You can achieve the same goal by trying to solve programming challenges. There are a lot of good ones, most of them are language agnostic (like hacker's delight, project euler, code kata), some very good ones do target a single language (and are perfect if you want to learn if), but
usually can also be solved in other languages without any problem (like python challenge, rubyquiz or challenges), others are geared towards a given programming paradigm (like ICPF).

The following are my personal suggestions (after that I'll try not to write about languages for a while, back to rendering stuff...):
  • HLSL/CG/Cuda/Rapidmind: Perfectly suited for data parallel algorithms, like image processing or eulerian physics. And of course they're kind good for shaders and realtime graphic stuff too. Shaders are fun, you should really try to get a work that includes coding them. You get all the glam, no crashing bugs, no pointers, fast iteration times, advanced math, lots of research while still caring a lot about low level optimizations too! Also, GPU forces you to code in a data parallel way, but data parallelism is very good also for modern multicore/multithreaded CPUs (Rapidmind compiles for Cell SPU as well, Muda is another interesting project), so it's a good skill to have. For CG/HLSL programming I would reccomend to use FX Composer (not great, but the best we have) or a .Net language like C# via the Tao framework, SlimDX or MD3D10
  • OcaML/F#/AliceML: Those are quite general programming languages, OcaML has a compiler that is as optimizing as the most refined C++ ones. But ML languages, even if they are not purely functional, are strongly oriented towards functional programming. I wouldn't write a server in them (but that's only my opinion). They seem to be the perfect environment to write a raytracer into (hint hint! well, nothing new really), even if they miss SIMD intrinsics (as far as I know). They also have a strong reputation for writing program transformers (i.e. compilers). And of course like most functional programming languages they support incremental development a LOT better than C++. I would start with F# as it has a lot of nice features, it's a .Net language that works with VisualStudio, and it's going to be included in the next iteration of the aforementioned IDE. OcaML on the other side is a much older ML dialect, and it also comes with one of the most optimized compilers. I've included also AliceML as I like its implementation of futures, serialization and distributed computing, but I don't know it well enough to reccomend it. Just have a look at it, as it's a good implementation of many nice ideas...
  • Scheme: Scheme is a lisp dialect. Even if I've only done projects in Common Lisp, I find it to be to huge, confusing and outdated, so I suggest Scheme instead. Hey! MIT guys are smarter than normal people, and they learn coding with Scheme! Surely there has to be something good in it... You can see that I've not linked "scheme" to, as the language standard is so small that it's commonly extended in non-standard ways by every implementation of it. Also the standard itself has a lot of "optinal features" that can be implemented or not by a given compiler. That's why I've directly linked to a specific scheme implementation, drScheme/pltScheme. Lisp should be known for plenty of reasons: it's one of the first programming languages, it's a symbolic language, it's a functional language, it's elegant, it has a strong mathematical foundation, it's a metaprogramming language, many other languages have features that are heavily inspired by it (ruby...). But what about an actual, real project using it? Well of course it's perfectly suited for metaprogramming and writing domain specific embedded languages. Programs that transform themselves. Add to this the ability of modifying code while it runs, and you have a natural platform for doing computational art. There are already a couple of projects doing that, but they don't seem to be very refined to me, there's still plenty of space...
  • Python/Ruby: As a scripting language, I prefer Ruby way more than Python. Ruby is really great, it's almost perfect. And it's easy to learn too. But Ruby is slower, way slower, I don't think it's really suited as of now for rendering stuff. There are a lot of numerical and graphical libraries for python, API bindings, it's used as a scripting language in FX Composer (via IronPython) and in other DCC applications. Overall it could be a good choice. It's even used to run SIMD code for on the Cell processor...
  • Other languages: There are plenty of them, JavaScript (well, ECMAScript as it's called now) for example is way more "advanced" (it's not Java AT ALL, it's Scheme with a Java-ish syntax applied over it to make marketing dept. happier) than you might think (and commonly used as a scripting language for many DCC applications). Smalltalk is the incarnation of live coding, reflection and object oriented programming is its purest form (and probably had the same influence Lisp had on the design of subsequent languages). Proce55ing is a simple and kinda well-known graphic-oriented java dialect. Io is nice, incredibly simple yet powerful, and comes with OpenGL bindings. Vector languages like APL and J also are worth knowing. Haskell is lazy, and I love that. Forth, oh shit, I even like TCL... And of course you might actually want to base your project on the implementation of a new programming language too (in that case I would start with LLVM and AnTLR)!
Of course, after all that, you might start to appreciate more C#, the features added in the third version of the language, LINQ, Parallel.FX and the huge effort that's going on to evolve that in order to include features that are really nothing new (lisp and more lisp!), but still did not ever find a way into mainstream languages before...
I've already posted plenty of links to cool C# stuff, so I won't repeat them again here.

P.S. all the above does not reflect my personal tastes in programming languages. It's simply stuff that's very useful to learn and to play with! In fact even if I think it's a broken language, I still can on a good day enjoy programming in C++. And even if I love Lisp power and elegance, I still don't think I should implement a language before starting to use it, Lisp is the ultimate metaprogramming tool, but in order to achieve it has almost no syntax, and I love having a syntax when I'm programming, because I'm noob enough to actually enjoy having tools for checking it (that's also why I generally prefer static typing, but I'm also more of a "system" programmer than of a "scripting" one).


Anonymous said...

There's also this Java Framework being developed by a company called Pervasive out of Austin. It also looks to be well suited for data parallel algorithms. I'm not sure about shading or real time graphics but maybe it's something you would be interested in if you can get away from rockband or lego indiana jones :)

DEADC0DE said...

Sun's fortress is very nice too:

there's plenty of other interesting things, I've only cited the ones I like most/think that are interesting to experiment with

Anonymous said...

you forgot to cite TopCoder when talking about "programming challenges". (

top coder is a professional "challenging arena" where you get paid if you win the challenges and you can get hired if they like you :)