My JS1K Demo - The Making Of

If you haven't seen it yet, check out the JS1K demo contest. The goal is to do something neat in 1 kilobyte of JavaScript code.

I couldn't resist making one myself, so I pulled out my bag of tricks from my Winamp music visualization days and started coding. I'm really happy with how it turned out. And no, it won't work in Internet Explorer 8 or less.

Edit: OH SNAP! I just rewrote the demo to include volumetric light beams and still fit in 1K:

Making Worlds 4 - The Devil's in the Details

Last time I'd reached a pretty neat milestone: being able to render a somewhat realistic rocky surface from space. The next step is to add more detail, so it still looks good up close.

Adding detail is, at its core, quite straightforward. I need to increase the resolution of the surface textures, and further subdivide the geometry. Unfortunately I can't just crank both up, because the resulting data is too big to fit in graphics memory. Getting around this will require several changes.

Strategy

Until now, the level-of-detail selection code has only been there to decide which portions of the planet should be drawn on screen. But the geometry and textures to choose from are all prepared up front, at various scales, before the first frame is started. The surface is generated as one high-res planet-wide map, using typical cube map rendering:

This map is then divided into a quad-tree structure of surface tiles. It allows me to adaptively draw the surface at several pre-defined levels of detail, in chunks of various sizes.

Source

This strategy won't suffice, because each new level of detail doubles the work up-front, resulting in exponentially increasing time and memory cost. Instead, I need to write an adaptive system to generate and represent the surface on the fly. This process is driven by the Level-of-Detail algorithm deciding if it needs more detail in a certain area. Unlike before, it will no longer be able to make snap decisions and instant transitions between pre-loaded data: it will need to wait several frames before higher detail data is available.

Uncontrolled growth of increasingly detailed tiles is not acceptable either: I only wish to maintain tiles useful for rendering views from the current camera position. So if a specific detailed portion of the planet is no longer being used—because the camera has moved away from it—it will be discarded to make room for other data.

Generating Individual Tiles

The first step is to be able to generate small portions of the surface on demand. Thankfully, I don't need to change all that much. Until now, I've been generating the cube map one cube face at a time, using a virtual camera at the middle of the cube. To generate only a portion of the surface, I have to narrow the virtual camera's viewing cone and skew it towards a specific point, like so:

This is easy using a mathematical trick called homogeneous coordinates, which are commonly used in 3D engines. This turns 2D and 3D vectors into respectively 3D and 4D. Through this dimensional redundancy, we can then represent most geometrical transforms as a 4x4 matrix multiplication. This covers all transforms that translate, scale, rotate, shear and project, in any combination. The right sequence (i.e. multiplication) of transforms will map regular 3D space onto the skewed camera viewing cone.

Given the usual centered-axis projection matrix, the off-axis projection matrix is found by multiplying with a scale and translate matrix in so-called "screen space", i.e. at the very end. The thing with homogeneous coordinates is that it seems like absolute crazy talk until you get it. I can only recommend you read a good introduction to the concept.

With this in place, I can generate a zoomed height map tile anywhere on the surface. As long as the underlying brushes are detailed enough, I get arbitrarily detailed height textures for the surface. The normal map requires a bit more work however.

Normals and Edges

As I described in my last entry, normals are generated by comparing neighbouring samples in the height map. At the edges of the height map texture, there are no neighbouring samples to use. This wasn't an issue before, because the height map was a seamless planet-wide cube map, and samples were fetched automatically from adjacent cube faces. In an adaptive system however, the map resolution varies across the surface, and there's no guarantee that those neighbouring tiles will be available at the desired resolution.

The easy way out is to make sure the process of generating any single tile is entirely self-sufficient. To do this, I expand each tile with a 1 pixel border when generating it. Each such tile is a perfectly dilated version of its footprint and overlaps with its neighbours in the border area:

This way all the pixels in the undilated area have easily accessible neighbour pixels to sample from. This border is only used during tile generation, and cropped out at the end. Luckily I did something similar when I played with dilated cube maps before, so I already had the technique down. When done correctly, the tiles match up seamlessly without any additional correction.

Adaptive Tree

Now I need to change the data structure holding the mesh. To make it adaptive, I've rewritten it in terms of real-time 'split' and 'merge' operations.

Just like before, the Level-of-Detail algorithm traverses the tree to determine which tiles to render. But if the detail available is not sufficient, the algorithm can decide that a certain tile in the tree needs a more detailed surface texture, or that its geometry should be split up further. Starting with only a single root tile for each cube face, the algorithm divides up the planet surface recursively, quickly converging to a stable configuration around the camera.

As the camera moves around, new tiles are generated, increasing memory usage. To counter this steady stream of new data, the code identifies tiles that fall into disuse and merges them back into their parent. The overall effect is that the tree grows and shrinks depending on the camera position and angle.

Queuing and scheduling

To do all this real-time, I need to queue up the various operations that modify the tree, such as 'split', 'merge' and 'generate new tile'. They need to be executed in between rendering regular frames on screen. Whenever the renderer decides a certain tile is not detailed enough, a request is placed in a job queue to address this.

While continuing to render regular frames, these requests need to be processed. This is harder than it sounds, because both planet rendering and planet generation have to share the GPU, preferably without causing major stutters in rendering speed.

The solution is to spread this process over enough visible frames so that the overal rendering speed is not significantly affected. For example, if a new surface texture is requested, several passes are made. First the height map is rendered, the next frame the normal map is derived from it, then the height/normal maps are analyzed and put into the tree, after which they will finally appear on screen:

I took some inspiration from id Tech 5, the next engine coming from technology powerhouse id Software. They describe a queued job system that covers any frame-to-frame computation in a game engine (from texture management to collision detection), and which schedules tasks intelligently.

Do the Google Earth

With all the above in place, the engine can now progressively increase the detail of the planet across several orders of magnitude. Here's a video that highlights it:

And some shots that show off the detail:

Kindle Faux PDF Zoom

Through the miracle of xmas, I acquired a Kindle. A sleek e-reader, but also a shameless vehicle for Amazon's digital book store. With the latest firmware installed, they do make for great PDF readers... in theory.

The good news is that the e-ink display on the Kindle is indeed pretty sweet. It works so well that the screen looks positively fake when it's not changing, as if it was just a display item in a shop somewhere. But the bad news is that the software needs a lot of love.

The included PDF reader for example has no zoom option. All you can do is toggle between portrait and landscape. Either way, normal sized text ends up tiny and barely readable.

Thankfully, we can still do it ourselves. Armed with PyPDF I wrote a simple script that takes a regular A4/Letter PDF and chops each page into four parts. You can pan through the document just by hitting next. Most of the stuff I read these days is academic, in the classic two column paper format, so this orders the sub-pages to match that.

The script is available for download. It requires Python and pyPdf. Usage:

python rekindle.py file.pdf

It will produce file.kindle.pdf. The code doesn't actually look at the contents, it just cuts blindy, so it might need adjustment for certain docs.

Making Worlds - Intermission

Today at BazCamp YVR I gave a short presentation and demo of my "Making Worlds" project, as well as an overview of procedural content generation in general.

The slides are available for download.

Making Worlds 3 - That's no Moon...

It's been over two months since the last installment in this series. Oops. Unfortunately, while trying to get to the next stage of this project, I ran into some walls. My main problem is that I'm not just creating worlds, but also learning to work with the Ogre engine and modern graphics hardware in particular.

This presents some interesting challenges: between my own code and the pixels on the screen, there are no less than three levels of indirection. First, there's Ogre, a complex piece of C++ code that provides me with high-level graphics tools (i.e. objects in space). Ogre talks to OpenGL, which abstracts away low-level graphics operations (i.e. commands necessary to draw a single frame). The OpenGL calls are handed off to the graphics driver, which translates them into operations on the actual hardware (processing vertices and pixels in GPU memory). Given this long dependency chain, it's no surprise that when something goes wrong, it can be hard to pinpoint exactly where the problem lies. In my case, an oversight and misunderstanding of an Ogre feature lead to several days of wasted time and a lot of frustration that made me put aside the project for a while.

With that said, back to the planets...

Normal mapping

Last time, I ended with a bumpy surface, carved by applying brushes to the surface. The geometry was there, but the surface was still just solid white. To make it more visually interesting, I'm going to apply light shading.

The most basic information you need for shading a surface is the surface normal. This is the vector that points straight away from the surface at a particular point. For flat surfaces, the normal is the same everywhere. For curved surfaces, the normal varies continuously across the surface. Typical materials reflect the most light when the surface normal points straight at the light source. By comparing the surface normal with the direction of incoming light (using the vector dot product), you can get a good measure of how bright the surface should be under illumination:

Lighting a surface using its normals.

To use normals for lighting, I have two options. The first is to do this on a geometry basis, assigning a normal to every triangle in the planet mesh. This is straightforward, but ties the quality of the shading to the level of detail in the geometry. A second, better way is to use a normal map. You stretch an image over the surface, as you would for applying textures, but instead of color, each pixel in the image represents a normal vector in 3D. Each pixel's channels (red, green, blue) are used to describe the vector's X, Y and Z values. When lighting the surface, the normal for a particular point is found by looking it up in the normal map.

The benefit of this approach is that you can stretch a high resolution normal map over low resolution geometry, often with almost no visual difference.

Lighting a low-resolution surface using high-resolution normals.

Here's the technique applied to a real model:

(Source - Creative Commons Share-alike Attribution)

Normal mapping helps keep performance up and memory usage down.

Finding Normals

So how do you generate such a normal map, or even a single normal at a single point? There are many ways, but the basic principle is usually the same. First you calculate two different vectors which are tangent to the surface at the point in question. Then you use the cross product to find a vector perpendicular to the two. This third vector is unique and will be the surface normal.

For triangles, you can pick any two triangle edges as vectors. In my case, the surface is described by a heightmap on a sphere, which makes things a bit trickier and requires some math.

I asked my friend Djun Kim, Ph.D. and teacher of mathematics at UBC for help and he recommended Calculus on Manifolds by Michael Spivak. This deceptively small and thin book covers all the basics of calculus in a dense and compact way, and quickly became my new favorite reading material.

Differential Geometry

In this section, I'll describe the formulas needed to calculate the normals of a spherical heightmap. Unlike what I've written before, this will dive shamelessly into specifics and not eschew math. The reason I'm writing it down is because I couldn't find a complete reference online. If math scares you, this section might not be for you. Scroll down until you reach the crater, or take a detour by reading A Mathematician's Lament by Paul Lockhart, which will enlighten you.

First, we're going to derive normals for a regular flat terrain heightmap. To start, we need to define the terrain surface. Starting with a 2D heightmap, i.e. a function f(u,v) of two coordinates that returns a height value, we can create a 3 dimensional surface g:

We can use this formal description to find tangent and normal vectors. A vector is tangent when its direction matches the slope of the surface in a particular direction. Differential math tells us that slope is found by taking the derivative. For our function of 2 variables, that means we can find tangent vectors along curves of constant v or constant u. These curves are the thin grid lines in the diagram. Actually, we can find tangents along any line, in any direction. But along u and v lines, the other variable acts like a constant, which simplifies things.

To do this, we take partial derivatives with respect to u (with v constant) and with respect to v (with u constant). The set of all partial derivatives is called the Jacobian matrix J, whose rows form the tangent vectors t_u and t_v, indicated in red and purple:

The cross product of t_u and t_v gives us n, the surface normal.

When applied to a discrete heightmap, the function f(u,v) is a 2D array map[u][v], and the partial derivatives at the end have to be replaced with something else. We can use finite differences to approximate the slope of the surface by differencing neighbouring samples:

This result and the formula for n are usually provided as-is in terrain mapping guides, without going through the full process of finding tangents first. However, it's important to use the Jacobian matrix formulation once you switch to spherical terrain.

To make a sphere, we add an additional function k which warps the flat terrain into a spherical shell. Each shell is the result of warping a single face of the cubemap and covers exactly 1/6th of the sphere. In what follows, We'll only consider a single face and its shell.

We designate the intermediate pre-warp coordinates (s,t,h), and the final post-warp coordinates as (x,y,z):

The principle behind the spherical mapping is this: first we take the vector (s, t, 1), which lies in the base plane of the flat terrain. We normalize this vector by dividing it by its length w, which has the effect of projecting it onto the sphere: (s/w, t/w, 1/w) will be at unit distance from (0, 0, 0). Then we multiply the resulting vector by the terrain height h to create the terrain on the sphere's surface, relative to its center: (h·s/w, h·t/w, h/w)

Just like with the function g(u,v) and J(u,v), we can find the Jacobian matrix J(s,t,h) of k(s,t,h). Because there are 3 input values for the function k, there are 3 tangents, along curves of varying s (with constant t and h), varying t (constant s and h) and varying h (constant s and t). The three tangents are named t_s, t_t, t_h.

PS: If your skills at derivation are a bit rusty, remember that Wolfram Alpha can do it for you.

How does this help? The three vectors describe a local frame of reference at each point in space. Near the edges of the grid, they get more skewed and angular. We use these vectors to transform the flat frame of reference into the right shape, so we can construct a new 90 degree angle here.

In mathematical terms, we multiply the 'flat' partial derivatives by the Jacobian matrix. This is similar to the chain rule for regular derivatives, only for multiple variables.

That is, to find the partial derivatives (i.e. tangent vectors) of the final spherical terrain with respect to the original terrain coordinates u and v, we can take the flat terrain's tangents t_u and t_v and multiply them by J(s,t,h). Once we have the two post-warp tangents, we take their cross product, and find the normal of the spherical terrain:

It's imporant to note that this is not the same as simply multiplying the flat terrain normal with J(s,t,h). J(s,t,h)'s rows do not form a set of perpendicular vectors (it is not an orthogonal matrix), which means it does not preserve angles between vectors when you multiply by it. In other words, J(s,t,h) * n, with n the flat terrain normal, would not be perpendicular to the spherical terrain. This is why it's important to return to the basic calculus underneath, so we can get the correct, complete formula.

Thus ends the magical math adventure. If you read it all the way through, cheers!

No Wait, It is a Moon.

With the normal map in place, I can now render the planet's surface and get a realistic idea of what it looks like. To show this off, I tweaked the brush system a bit: instead of using the literal brush image (e.g. a smooth, round crater), the brush is distorted with fractal noise. It makes every application of the brush subtly different from the next, and saves me from manually drawing e.g. a hundred different craters.

Here's a side by side comparison of the original brush and a distorted version.

Currently I've only implemented one type of distortion, which lends a rocky appearance to the surface. With that in place, my engine can now generate somewhat realistic looking moon surfaces. Here's the demo:

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

• Creating Spherical Worlds, Maxis/Electronic Arts. (source).
• Calculus on Manifolds, Michael Spivak.

Making Worlds 2 - Scaling Heights

Last time, I had a working, smooth sphere mesh. The next step is to create terrain.

Scale

Though my goal is to render at a huge range of scales, I'm going to focus on views from space first. That strongly limits how much detail I need to store and render. Aside from being a good initial sandbox in terms of content generation, it also means I can comfortably keep using my current code, which doesn't do any sophisticated memory or resource management yet. I'd much rather work on getting something interesting up first rather than work on invisible infrastructure.

That said, this is not necessarily a limitation. The interesting thing about procedural content is that every generator you build can be combined with many others, including a copy of itself. In the case of terrain, there are definite fractal properties, like self-similarity at different levels of scale. This means that once I've generated the lowest resolution terrain, I can generate smaller scale variations and combine them with the larger levels for more detail. This can be repeated indefinitely and is only limited by the amount of memory available.

Perlin Noise is a celebrated classic procedural algorithm,
often used as a fractal generator.

Height

To build terrain, I need to create heightmaps for all 6 cube faces. Shamelessly stealing more ideas from Spore, I'm doing this on the GPU instead of the CPU, for speed. The GPU normally processes colored pixels, but there's no reason why you can't bind a heightmap's contents as a grayscale (one channel) image and 'draw' into it. As long I build my terrain using simple, repeated drawing operations, this will run incredibly fast.

In this case, I'm stamping various brushes onto the sphere's surface to create bumps and pits. Each brush is a regular PNG image which is projected onto the surface around a particular point. The luminance of the brush's pixels determines whether to raise or lower terrain and by how much.

Three example brushes from Spore. (source)

However, while the brushes need to appear seamless on the final sphere, the drawing area consists only of the straight, square set of cube map faces. It might seem tricky to make this work so that the terrain appears undistorted on the curved sphere grid, but in fact, this distortion is neatly compensated for by good old perspective. All I need to do is set up a virtual scene in 3D, where the brushes are actual shapes hovering around the origin and facing the center. Then, I place a camera in the middle and take a snapshot both ways along each of the main X, Y and Z directions with a perfect 90 degree field of view. The resulting 6 images can then be tiled to form a distortion-free cube map.

Rendering two different cube map faces. The red area is the camera's viewing cone/pyramid, which extends out to infinity.

To get started I built a very simple prototype, using Ogre's scene manager facilities. I'm starting with just a simple, smooth crater/dent brush. I generate all 6 faces in sequence on the GPU, pull the images back to the CPU to create the actual mesh, and push the resulting chunks of geometry into GPU memory. This is only done once at the beginning, although the possibility is there to implement live updates as well.

Here's a demo showing a planet and the brushes that created it, hovering over the surface. I haven't implemented any shading yet, so I have to toggle back and forth to wireframe mode so you can see the dents made by the brushes:

The cubemap for this 'planet' looks like this when flattened. You can see that I haven't actually implemented real terrain carving, because brushes cause sharp edges when they overlap:

The narrow dent on the left gets distorted and angular where it crosses the cube edge. This is a normal consequence of the cubemapping, as it looks perfectly normal when mapped onto the sphere in the video.

Engine Tweaks

The demo above also incorporates a couple of engine improvements. With a real heightmap in place, I can implement real level-of-detail selection. That means the resolution of any terrain tile is decided based on how much detail would be lost if a simpler tile was used. The flatter a tile, the less detail is necessary. This ensures complex geometry is used only on those sections that really need it. This is great for visual fidelity, but causes a lot of geometry to pop up if sharp ridges are present in the terrain. In this case, my rendering engine was happily trying to push 700k triangles through the GPU per frame. While even my laptop GPU can actually do that at pretty smooth frame rates nowadays, some optimizations are in order to give me some breathing room.

The culprit was that I wasn't really doing any early removal of geometry that was hidden or otherwise out of frame. To fix that, I now do visibility checks together with the level-of-detail selection. First it checks if a chunk is over the horizon or not before considering it for selection. This is easy to calculate and eliminates a lot of unnecessary drawing, especially when looking straight down. If that first visibility check passes, I perform a tighter check using the camera's viewing cone. With these two measures in place, I'm only averaging about 50,000-100,000 triangles visible per frame, with room for more optimization. These optimizations only remove geometry that's already off screen, so there is no visual difference.

Cubemap Seams and Dilation

When rendering into cube maps, each side is rendered independently. In theory each face should match perfectly with adjacent ones due to the way they've been created. In practice however, slight mismatches can occur due to rounding errors at the edges, creating seams. This can be fixed by explicitly copying one pixel-wide edges from one face into the adjacent ones, until they all match up.

The next big step is to start shading the surface, but in order to do that I need to be able to run filters on the cube map. Specifically, I need to be able to compare neighbouring height samples anywhere on the surface. In the straight forward cubemap scenario this is non-trivial, because neighbouring samples at the edges need to be fetched from different cube faces at different orientations in space.

I decided to implement something I call 'dilated cubemaps'. I've never really heard this described formally, though I doubt it's never been thought of before:

Instead of every face neatly matching with the next, I dilate the cube faces so they stick through eachother. At the same time, I use a larger texture size to compensate, and I adjust the field-of-view of the rendering camera to match. If done right, the resulting cubemap is a pixel-perfect expanded version of the undilated map.

The dilated cubemap provides reliable neighbouring samples for all samples in the original cube map up to a distance as wide as the new border. Unlike regular cubemap wrapping, the dilated regions are distorted to conform to the current face's grid. This matches the real change in grid direction that occurs on the final sphere mesh and lets you sample exactly across cube map edges.

I played with the cubemap dilation because I was thinking of some complicated filters to run that require regular grids (like CFD). But in retrospect, I probably don't need the exact spacing of sample points at the edges for this, so regular undilated cubemapping will probably do. Still, it's good to have around, and certainly was an interesting exercise in pixel-exact rendering.

What Next?

With basic heightmap generation in place, I can now start putting in some 'tech artist' time to play with various brushes and drawing behaviour. Lighting and shading is another big one and should provide a massive improvement to the visuals.

Right now I've taken a week between postings, though it remains to be seen whether I can maintain that. Creating these blog entries is turning into a pretty time consuming endeavour, especially as I get into territory where I have to make my own diagrams and illustrations.

References

The techniques I used were pioneered by people smarter and older than me, I'm just building my own little digital machine with them.

• Creating Spherical Worlds, Maxis/Electronic Arts. (source).
• Ken Perlin, who invented a lot of this stuff.

Making Worlds 1 - Of Spheres and Cubes

Let's start making some planets! Now, while this started as a random idea kind of project, it was clear from the start that I'd actually need to do a lot of homework for this. Before I could get anywhere, I needed to define exactly what I was aiming for.

The first step in this was to shop around for some inspirational art and reference pictures. While there is plenty space art to be found online, in this case, nothing can substitute for the real thing. So I focused my search on real pictures, both of landscapes (terran or otherwise) as well as from space. I found classy shots like these:

Making Worlds: Introduction

For the past year or so I've been reacquainting myself with an old friend: C++.

More specifically, I've been exploring graphics programming again, this time with the luxurious flexibility of the modern GPU at my fingertips. To get me started, I shopped around for an open source engine to play with. After trying Irrlicht and finding its promises to be a bit lacking, Ogre turned out to be a really good choice. Though its architecture is a bit intimidating at first, it is all the more sound. More importantly, it seems to have a relatively healthy open-source community around it.

So with Ogre as my weapon of choice, I've started a new project: Making Worlds. More specifically, I want to procedurally generate a 3D planet, viewable from outer space as well as the ground (at flight-sim levels of detail), which can be rendered real-time on recent graphics hardware.

Why? Because I really like procedural content generation. It's an odd discipline where anything goes, and techniques from across mathematics, engineering and physics are applied. Then, you add a good dose of creativity and artistic sense, and perhaps mix in some real-world data too, until you find something that looks right.

Plus, far from being an exercise in pointlessness, procedural content is gaining in popularity, especially for video games.

So, in the style of Shamus Young's excellent Procedural city series, I'm going to start blogging about Making Planets. Unlike him however, I'm not going to adhere to a strict schedule.

Here's a teaser for the first installment.

JavaScript audio synthesis with HTML 5

HTML5 gives us a couple new toys to play with, such as <AUDIO> and <VIDEO> tags. On the visual side, we've already seen live green-screening with Canvas and JS, and in terms of audio there's been several JS drum machines already. But the question I was interested in was: can you use JavaScript to stream live data into these media tags?

Enter the JavaScript audio synth. It generates a handful of samples using very basic time-domain synthesis, wraps them up in a WAVE file header and embeds them in <AUDIO> tags using base64-encoded data URIs. Each sample is then triggered using timers to play the drum pattern. It's quite simple to do and runs fast enough in HTML5 capable browsers to be unnoticeable. Yes, it sounds tinny, but that's just because I'm too lazy to design proper filters for toys like this. Unfortunately, while the synthesis is fast enough to run real-time, you can't actually use it for a full live audio stream, as there is no way to queue up chunks of synthesized audio for seamless playback. I tried triggering multiple <AUDIO> tags in parallel to address this, but that didn't work either.

My final attempt was to generate tons of periodic audio loops only a couple of ms long, and to play them back with looping turned on while altering each tag's volume in real time, hence doing a sort of additive wavetable synthesis. Unfortunately, looping is not a fully supported feature, and the only browser I found that does it (Safari) doesn't loop seamlessly at all.

All in all, my first brush with the <AUDIO> tag was a major disappointment. The <VIDEO> tag's high-level approach leads to similar limitations, but they are offset by the flexibility and power of the <CANVAS> tag. Unfortunately, there is no 'audio canvas' to solve similar problems with audio.

Rick Falkvinge on the Swedish Pirate Party

Last month, I attended the Open Web Vancouver conference. Without a doubt, I thought the most interesting talk at the whole conference was Rick Falkvinge's keynote session about the Swedish Pirate Party.

You may have heard of the Pirate Party. Founded in 2006, the popular image is that of an anarchist movement that grew out of the sense of entitlement of media pirates on the internet. It is said that these people want to abolish modern copyright for purely selfish reasons. Unfortunately this is not just a tired old stereotype, but completely wrong.

Thankfully, Rick's entire session is available online. In one hour, he calmly and intelligently explains his vision. He shows that the Pirate Party's agenda is about civil liberties, and part of a discussion that has been going on for centuries. By tracing back the history of copyright, he shows a clear pattern: when new technology threatens established powers and businesses, those powers try to use legislation to protect themselves and maintain control. It started with the printing press, and continues today with the internet and file sharing technologies. He shows how we've already allowed the establishment to create legislation that tries to control its true potential, and how we need a political countermovement that represents all the interests of a free, digitally liberated society.

You can watch the entire presentation below. The slides with illustrations and stats are available as well.

Part 1:

Part 2:

After the talk, I got the chance to ask Rick something I'd been wondering for quite a while: whether he thought that the "pirate" label had helped or hurt their cause?

His answer was very clever: while it is obviously a controversial name, it gave their movement a lot of much needed exposure very early on, which they wouldn't have gotten otherwise. But more than that, by adopting the "pirate" moniker, they clearly state that they intend to change the meaning and perception of that word. Hence it prevents the copyright lobby (and anyone else) from using it as a slur to delegitimize their movement.

So please don't let the "pirate" label scare you off or assume that the issues they are championing are not serious, because they really are.

In fact, the entire world got a demonstration of this last month when in Iran, post-election protests erupted after accusations of election fraud. The Iranian government denied the accusations and tried to snuff out the dissent, by any means necessary. The suppression of free communication was an essential part of this, and was mostly successful. However, thanks to modern, secure, free networking technology, vital information and media still managed to leak out and show the world what was really going on. If the Iranians hadn't been able to freely record media and communicate, the picture would've been quite different.

This alone should be reason enough to consider access to digital communication an essential human right, and yet, countries such as Sweden, France and New Zealand have introduced laws (or have tried to) to allow the government to wiretap all international digital traffic; to ban certain people permanently from the internet; to force all internet service providers to deny access to sites based purely on government-controlled blacklists. Plus, let's not forget the continued reign of the Grand Firewall of China which keeps China's populace from seeing anything 'disruptive', as well as similar measures in other countries.

Such ideas are Wrong and Unjust and should be countered at every step. They are merely the result of an old guard trying to frantically hold on to what they know, in a world that has already irreversibly changed. I urge you to support the Pirate Party, and to support similar organisations in your own country.

Our New Art Wall

We finished our new office decorations today, with chalk on blackboard paint. It covers the hideous wallpaper very nicely.

Check the YouTube video below, or go to Vimeo for a high-def version.

Yes, we cheated, shamelessly, by using a projector, but it looks rad!

Based on:

• Girl with Hair Ribbon by Roy Lichtenstein
• A Darth Vader photo/capture found on Google Images
• An iconic frame from the Sin City movie based on the Frank Miller comic

Think Culture, not Race

The Edge Foundation has this event called the World Question Center where every year, they ask some of the world's brightest thinkers one Big Question. Last year it was What have you changed your mind about and why?

I loved pouring over the answers, mostly because many of them described a shift in the writer's thinking, rather than just changing their opinion on a particular fact.

However thinking about what my answer would be, I couldn't really come up with something I felt strongly about. That was until recently, when I saw a very poignant speech on racism. The crystallization of 'race' as an artificial descriptor of culture finally put concrete words onto something that had been in the back of my head for years now. Here goes.

What did I change my mind about and why?

Growing up liberal and secular, I was imbued with all the virtues of humanistic thinking: freedom of thought, freedom of speech, equality and justice. Part of that was the often repeated mantra that racism was bad. All of us kids accepted it without question, and as we grew up, we applied it in our own life. Meet someone who's darker skinned than you? Whose eyes are different? They are equal to you and you should treat them with respect. So would it echo in my head. But in the homogeneity of ex-catholic Belgium, this was still the exception rather than commonplace. Which meant that every time I had that thought running through my head, it felt like an uneasy reminder rather than something that was just part of my own values and ethics.

It wasn't until I moved to Vancouver that I really figured out why, because I was suddenly transplanted into the melting pot of this American-Asian city. It didn't take much for me to lose that guilty reminder about racism and to learn to take people truly at face value. People here come in all sorts of colors and sizes, living in one community, and everyone acts like they are part of the same culture. It's a very diverse and populous culture, but one culture nevertheless.

Whereas back in pastoral Belgium, it was a pretty good rule of thumb that someone would only think and talk like you if they also looked like you. Physical traits, grouped by 'race', became a predictor of culture. Which meant that the tension that comes with differences between cultures became associated with people's physical appearance.

But what most people forget is that our modern, western lines of 'race' (Caucasian, Latino, African-American, Asian, ...) are just the latest version of an age-old concept. These qualifiers used to be much more numerous and specific. Today's caucasians used to be Slavs, Celts, Normans, Gauls, Moors, etc. But as populations grew, and political divisions shifted, cultures got mixed, and some notions of 'race' became obsolete. Racial divisions evolved, and as such are not arbitrary lines that someone has drawn somewhere just based on looks. They are almost always drawn on top of existing cultural borders.

And this is the lesson that I was never really taught. Nobody ever really said that the feeling of unease that I would have around what I perceived to be 'other races' was just cultural tension, and that lumping that in with this person looks different from me was the mistake. Not the fact that I was feeling uneasy. The giant guilt trip that I had been saddled with about inequality and racism left me so uptight, that I was handicapped in relating to the diversity in my own culture as well as other cultures. For a very long time, I was the awkward guy who could say he 'hangs out with plenty of ______ people' and still felt bad about it. But not anymore.

It's only when you get over that that you can truly work on the real problem: how to get all our different cultures to live together on the same planet. Which is still a real problem, and one that needs to be tackled, not ignored, and certainly not to be shushed with overzealous accusations of racism.

The Reality of Illegal TV Downloads

As you may know, I'm a sci-fi nerd, hence I've been pretty excited about the reimagined Battlestar Galactica series coming to a close. So, me and my fellow connoisseur of the awesome, Greg, put together a quick survey on Google Docs to get predictions about the end of the show. The internets filled it in.

The Battlestar nerdery was all in good fun, but more interestingly, I also asked a question about how people watch the show: via live broadcast, recorded or downloaded? Legally or illegally? Depending on your point of view, these results are either entirely obvious, or quite surprising. So far, 313 people filled in the survey, which was advertised only through blogs and Twitter for two days:

Given the circumstances, the people who answered this fit two descriptions. One, they are fan enough to actually fill out a survey about a show on TV. Two, they read blogs, talk on Twitter, hang out in forums, i.e. they know and use the web intimately. So, SciFi channel SyFy, NBC Universal, all big name media: do you see that big green chunk of people who download your shows illegally? These are merely potential customers that you haven't reached yet.

When you look at your ratings and bemoan the dwindling numbers, think of that 30%. Sure, these people are probably not getting you any ad money, but you can profit off them indirectly. Tech savvy people are the backbone of your nerdy fandom, and they add value to your precious intellectual property. Who do you think helped all those girlfriends, husbands, parents or siblings get over the silly name 'Battlestar Galactica' and actually watch the show? Who wrote all that stuff on the Battlestar wiki in their spare time, providing an anchor for online discussions and activity, keeping your brand active? Yup that's right, the nerds with their computers.

And seriously, those 30% aren't all anarchistic hacker types who despise copyright. A lot of them are just people who want to enjoy the show they love in the way that is convenient for them. An illegal high-definition torrent released a couple hours after the TV broadcast is indeed pretty darn convenient. Lucky for you, you are in the unique position of offering something even better.

Just stop treating the live broadcast as being sacred: it is merely one showing after the content has been made available. Instead, provide your own episode downloads at the same time as the TV broadcast. Make it attractive with additional extras for your hungry audience, like director commentary or deleted scenes. By all means offer a free ad-supported plan like Hulu for those who don't mind having their shows and brains invaded by rabid commercialism.

But please, open up the modestly priced option of high quality, ad-free, DRM-free downloads. The technology is there. If you do it right, you will go from making no money off of these people, to making some money off of them. Trust me, this group is only getting bigger by the day. Of course, it won't be easy: your inaction has caused an entire ecosystem of illegal distribution to spring up across IRC, Usenet, private trackers and the web. These people are organised and very good at what they do. Your competition is tough.

In this light, it's a bit silly to try and push region-restricted, delayed and limited online releases onto people. You're only providing a product worse than what's already available. You're clinging to your old ways, and only succeeding because a lot of this stuff is darn new and only the kids are doing it anyway.

Except, even the grown up folks around me are starting to figure it out. Some run Boxee on their cracked Apple TV (a plug and play process). Some have a dedicated torrent box at home that they log into (screen sharing built into their Mac). They've got phones in their pockets that can network literally anywhere, and look, someone can sell them an app to torrent their shows for a buck or two. See how this whole digital economy stuff works?

Do you honestly think you can control all that with increasingly restrictive DRM backed by increasingly restrictive law? Time's arrow points to our point of view becoming the dominant one. Either reinvent yourself, or continue losing. It's your move, TV.

Using Web APIs for Research

Recently we launched our new product at Strutta, a 'create your own contest site' web service. In each contest, users submit and vote on each other's videos, pictures, songs or writings.

As part of the research we did for the development, we wanted to examine our competition. So, I dove into YouTube to try and figure out some of their ideas and algorithms. For me, this wasn't entirely new: when I posted my Line Rider videos to YouTube, I followed up each video with manual statistics tracking and gained some insight into how a video becomes popular on YouTube. However, that only gave me a very narrow view of the community and its dynamics.

Since then though, things have changed a lot. YouTube now has a public API as well as pre-made libraries to use. With these, it becomes very easy to collect statistics and perform your own analysis. So, armed with Python, I set out to investigate YouTube's ubiquitous 'related videos' feature.

I found it interesting to analyse a big site through their own API rather than screen scraping. Traditionally, one first tries to collect as much data as possible, but the resulting data set can become very unwieldy. In this case, I already had full access and I could focus on exactly which queries I wanted to run, how to aggregate my data, and which measures to focus on.

The results revealed some interesting conclusions. My big write-up can be found on the Strutta Blog, aptly titled Six Degrees of YouTube.