"I think a lot of mathematics is really about how you understand things in your head.It's people that did mathematics, we're not just general purpose machines, we're people. We see things, we feel things, we think of things. A lot of what I have done in my mathematical career has had to do with finding new ways to build models, to see things, do computations. Really get a feel for stuff.

It may seem unimportant, but when I started out people drew pictures of 3-manifolds one way and I started drawing them a different way. People drew pictures of surfaces one way and I started drawing them a different way.There's something significant about how the representation in your head profoundly changes how you think.

It's very hard to do a brain dump. Very hard to do that. But I'm still going to try to do something to give a feel for 3-manifolds. Words are one thing, we can talk about geometric structures. There are many precise mathematical words that could be used, but they don't automatically convey a feeling for it.I probably can't convey a feeling for it either, but I want to try."– William Thurston, The Mystery of 3-Manifolds (Video)

How do you convince web developers—heck, people in general—to care about math? This was the challenge underlying Making Things With Maths, a talk I gave three years ago. I didn't know either, I just knew why I liked this stuff: demoscene, games, simulation, physics, VR, … It had little to do with what passed for mathematics in my own engineering education. There we were served only eyesore PowerPoints or handwritten overhead transparencies, with simplified graphs, abstract flowcharts and rote formulas, available on black and white photocopies.

Smart people who were supposed to teach us about technology seemed unable to teach us *with* technology. Fixing this felt like a huge challenge where I'd have to start from scratch. This is why the focus was entirely on showing rather than telling, and why MathBox 1 was born. It's how this stuff looks and feels in my head, and how I got my degree: by translating formulas into mental pictures, which I could replay and reason about on demand.

Initially I used MathBox like an embedded image or video: compact diagrams, each a point or two in a presentation. My style quickly shifted though. I kept on finding ways to transform from one visualization to another. Not for show, but to reveal the similarities and relationships underneath. MathBox encouraged me to animate things correctly, leveraging the actual models themselves, instead of doing a visual morph from A to B. Each animation became a continuous stream of valid examples, a quality both captivating and revealing.

For instance, How to Fold a Julia Fractal is filled with animations of complex exponentials, right from the get go. This way I avoid the scare that ($ e^{i\pi} $) is a meaningful expression; symbology and tau-tology never have a chance to obscure geometrical workings. Instead a web page that casually demonstrates conformal mapping and complex differential equations got 340,000 visits. Despite spotty web browser support and excluding all mobile phones for years.

The next talk, Making WebGL Dance, contained elaborate *long takes* worthy of an Alfonso Cuarón film, with only 3 separate shots for the bulk of a 30 minute talk. The lesson seemed obvious: the slides shouldn't have graphics in them, rather, *the graphics should have slides in them*. The diagnosis of PowerPoint syndrome is then the constant trashing of context from one slide to the next. A traditional blackboard doesn't have this problem: you build up diagrams slowly, by hand, across a large surface, erasing selectively and only when you run out of space.

It's not just about permanence and progression though, it's also about leveraging our natural understanding of shape, scale, color and motion. Think of how a toddler learns to interact with the world: poke, grab, chew, spit, smash. Which evolves into run, jump, fall, get back up again. Humans are naturals at taking multiple cases of "If I do this, that will happen" and turning it into a consistent, functional model of how things work. We learn language by bootstrapping random jibberish into situational meaning, converging on a shared protocol.

That said, I find the usual descriptions of how people experience language and thought foreign. Instead, when Temple Grandin speaks about visual thinking, I nod vigorously. Thought to me is analog concepts and sensory memories, remixed with visual and other simulations. It builds off the quantities and qualities present in spatial and temporal notions, which appear built-in to us.

Speech and writing is then a program designed to reconstruct particular thoughts in a compatible brain. There are a multitude of evolving languages, they can be used elegantly, bluntly, incomprehensibly, but the desired output remains the same. In my talks, armed with weapons-grade C2-continuous animations, it is easy to transcode my film reel into words, because the slides run themselves. The string of concepts already hangs in the air, I only add the missing grammar that links them up. This is a puzzle our brains are so good at solving, we usually do it *without* thinking.

Language is the ability of thoughts to compute their own source code.

(It's not proof, I just supply pudding.)

I don't say all this to up my Rain Man cred, but to lay to rest the recurring question of where my work comes from. I translate the pictures in my head to HD, in order to learn from and refine the view. As I did with quaternions: I struggled to grok the hypersphere, it wouldn't fit together right. So I wrote the code to trace out geodesics in color and fly around in it, and suddenly the twisting made sense. Hence my entire tutorial was built to replicate the same discovery process I went through myself.

There was one big problem: scenes now consisted of *diagrams of diagrams*, which meant working around MathBox more than with it. Performance issues arose as complexity grew. Above all there was a total lack of composability in the components. None of this could be fixed without ripping out significant pieces, so doing it incrementally seemed futile. I started from scratch and set off to reinvent all the wheels.

$$ \text{MathBox}^2 = \int_1^2 \text{code}(v) dv $$

MathBox 2 was inevitably going to suffer *second-system syndrome*, parts would be overengineered. Rather than fight it, I embraced it and effectively wrote a strange vector GPU driver in CoffeeScript. (Such is life, this is a blueprint meant to be simplified and made obsolete over time, not expanded upon.) It's a freight train straight to the heart of a graphics card, combining low-level and high-level in a way that feels novel 🐴 when you use it, squeezing 🐴 through a very small opening.

What was tedious before, now falls out naturally. If I format the scene above as XML/JSX, it becomes:

```
<root>
<!-- Place the camera -->
<camera />
<!-- Change clock speed -->
<clock>
<!-- 4D Stereographic projection -->
<stereographic4>
<!-- Custom 4D rotation shader -->
<shader />
<!-- Move vertices -->
<vertex>
<!-- Sample an area -->
<!-- Draw a set of lines -->
<area />
<line />
<!-- Sample an area -->
<!-- Draw a set of lines -->
<area />
<line />
<!-- Sample an area -->
<!-- Draw a set of lines -->
<area />
<line />
</vertex>
</stereographic4>
</clock>
</root>
```

In order to make these pieces behave, a bunch of additional attributes are applied, most of which are strings or values, some of which are functions/code, either JavaScript or GLSL:

```
<root id="1" scale={300}>
<camera id="2" proxy={true} position={[0, 0, 3]} />
<clock id="3" speed={1/4}>
<stereographic4 id="4" bend={1}>
<shader id="5" code="
uniform float cos1;
uniform float sin1;
uniform float cos2;
uniform float sin2;
uniform float cos3;
uniform float sin3;
uniform float cos4;
uniform float sin4;
vec4 getRotate4D(vec4 xyzw, inout vec4 stpq) {
xyzw.xy = xyzw.xy * mat2(cos1, sin1, -sin1, cos1);
xyzw.zw = xyzw.zw * mat2(cos2, sin2, -sin2, cos2);
xyzw.xz = xyzw.xz * mat2(cos3, sin3, -sin3, cos3);
xyzw.yw = xyzw.yw * mat2(cos4, sin4, -sin4, cos4);
return xyzw;
}"
cos1=>{(t) => Math.cos(t * .111)} sin1=>{(t) => Math.sin(t * .111)} cos2=>{(t) => Math.cos(t * .151 + 1)} sin2=>{(t) => Math.sin(t * .151 + 1)} cos3=>{(t) => Math.cos(t * .071 + Math.sin(t * .081))} sin3=>{(t) => Math.sin(t * .071 + Math.sin(t * .081))} cos4=>{(t) => Math.cos(t * .053 + Math.sin(t * .066) + 1)} sin4=>{(t) => Math.sin(t * .053 + Math.sin(t * .066) + 1)} />
<vertex id="6">
<area id="7" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
q1.set(0, 0, Math.sin(θ), Math.cos(θ));
q2.set(0, Math.sin(ϕ), 0, Math.cos(ϕ));
q1.multiply(q2);
emit(q1.x, q1.y, q1.z, q1.w);
}} live={false} channels={4} />
<line id="8" color="#3090FF" />
<area id="9" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
q1.set(0, Math.sin(θ), 0, Math.cos(θ));
q2.set(Math.sin(ϕ), 0, 0, Math.cos(ϕ));
q1.multiply(q2);
emit(q1.x, q1.y, q1.z, q1.w);
}} live={false} channels={4} />
<line id="10" color="#20A000" />
<area id="11" rangeX={[-π/2, π/2]} rangeY={[0, τ]} width={129} height={65} expr={(emit, θ, ϕ, i, j) => {
q1.set(Math.sin(θ), 0, 0, Math.cos(θ));
q2.set(0, 0, Math.sin(ϕ), Math.cos(ϕ));
q1.multiply(q2);
emit(q1.x, q1.y, q1.z, q1.w);
}} live={false} channels={4} />
<line id="12" color="#DF2000" />
</vertex>
</stereographic4>
</clock>
</root>
```

Phew. That's how you make a 4D diagram with Hopf fibration as far as the eye can see. Except it's not actually JSX, that's just me and my pretty-printer pretending.

The key is the data itself. It's an array of points mostly, but how that data is laid out and interpreted determines how useful it can be.

Most basic primitives come in fixed size chunks. Particles are single points, lines have two points, triangles have three points. Polygons and polylines have N points. So it made sense to have a *tuple of N points* be the basic logical unit. You can think in logical pieces of geometry, rather than raw points or individual triangles, unlike GL.

Each primitive maps over data in a standard way. Feed an `array`

of points to a `line`

, you get a polyline. Feed a `matrix`

of points to a `surface`

and you get a grid mesh. Simple. But feed a `voxel`

to a `vector`

, and you get a *3D vector field*. The general idea is that drawing 1 of something should be as easy as drawing 100×100×100.

This is particularly useful for custom data expressions, which stream in live or procedural data. They now receive an `emit(x, y, z, w)`

function, for emitting a 4-vector like XYZW or RGBA. This is little more than an inlineable call to fill a `floatArray[i++] = x`

, quite a lot faster than returning an array or object.

```
mathbox
.interval({
expr: function (emit, x, i, t) {
y = Math.sin(x + t);
emit(x, y);
emit(x, -y);
},
width: 64,
items: 2,
channels: 2,
})
.vector({
color: 0x3090FF,
width: 3,
start: true,
});
```

*Emitting 64 2D vectors on an interval, 2 points each.*

More importantly it lets you emit N points in one iteration, which makes the JS expressions themselves feel like geometry shaders. The result feeds into one or more styled drawing ops. The number of emit calls has to be constant, but you can always knock out or mask the excess geometry.

```
emit = switch channels
when 1 then (x) ->
array[i++] = x
++j
return
when 2 then (x, y) ->
array[i++] = x
array[i++] = y
++j
return
when 3 then (x, y, z) ->
array[i++] = x
array[i++] = y
array[i++] = z
++j
return
when 4 then (x, y, z, w) ->
array[i++] = x
array[i++] = y
array[i++] = z
array[i++] = w
++j
return
```

*Both the expression and emitter will be inlined into the stream's iteration loop.*

```
consume = switch channels
when 1 then (emit) ->
emit array[i++]
++j
return
when 2 then (emit) ->
emit array[i++], array[i++]
++j
return
when 3 then (emit) ->
emit array[i++], array[i++], array[i++]
++j
return
when 4 then (emit) ->
emit array[i++], array[i++], array[i++], array[i++]
++j
return
```

*Closures of Hanoi*

GPUs can operate on 4×1 vectors and 4×4 matrices, so working with 4D values is natural. Values can also be referenced by 4D indices. With one dimension reserved for the tuples, that leaves us 3 dimensions XYZ. Hence MathBox arrays are 3+1D. This is for *width*, *height*, *depth*, while the tuple dimension is called *items*. It does what it says on the tin, creating 1D W, 2D W×H and 3D W×H×D arrays of tuples. Each tuple is made of N vectors of up to 4 channels each.

Thanks to cyclic buffers and partial updates, history also comes baked in. You can use a spare dimension as a free time axis, retaining samples on the go. You can `.set('history', N)`

to record a short log of a whole array over time, indefinitely.

All of this is modular: a data source is something that can be sampled by a 4D pointer from GLSL. Underneath, arrays end up packed into a regular 2D float texture, with "items × width" horizontally and "height × depth" vertically. Each 'pixel' holds a 1/2/3/4D point.

Mapping a 4D 'pointer' to the real 2D UV coordinates is just arithmetic, and so are operators like `transpose`

and `repeat`

. You just swap the XY indices and tell everyone downstream that it's now *this* big instead. They can't tell the difference.

You can create giant procedural arrays this way, including across rectangular texture size limits, as none of them actually exist except as transient values deep inside a GPU core. Until you materialize them by rendering to a texture using the `memo`

primitive. Add in operators like interpolation and convolution and it's a pretty neat real-time finishing kit for data.

I want to render live 3D graphics based on a declarative data model. That means a choice of shapes and transforms, as well as data sources and formats. I also want to combine them and make live changes. Which sounds kind of DOMmy.

3D engines don't have *Document Object Models* though, they have *scene graphs* and *render trees*: minimal data structures optimized for rendering output. In Three.js, each tree node is a JS object with properties and children like itself. Composition only exists in a limited form, with a parent's matrix and visibility combining with that of its children. There is no fancy data binding: the renderer loops over the visible tree leaves every frame, passing in values directly to GL calls. Any geometry is uploaded once to GPU memory and cached. If you put in new parameters or data, it will be used to produce the next frame automatically, aside from a *needsUpdate* bit here and there for performance reasons.

So Three.js is a thin retained mode layer on top of an immediate mode API. It makes it trivial to draw the same thing over and over again in various configurations. That won't do, I want to draw *dynamic* things with the same ease. I need a richer model, which means wrapping another retained mode layer around it. That could mean observables, data binding, tree diffing, immutable data, and all the other fun stuff nobody can agree on.

However I mostly feed data *in* and many parameters will end up as shader properties. These are passed to Three as a dictionary of `{ type: '…', value: x }`

objects, each holding a single parameter. Any code that holds a reference to the dictionary will see the same value, as such it acts as a *register*: you can share it, transparently binding one value to N shaders. This way a single `.set('color', 'blue')`

call on the fringes can instantly affect data structures deep inside the `WebGLRenderer`

, without actually cascading through.

I applied this to build a view tree which retains this property, storing all attributes as shareable registers. The Three.js scene graph is reduced to a single layer of `THREE.Mesh`

objects, flattening the hierarchy. Rather than clumsy CSS3D divs which encode matrices as strings, there's binary arrays, GLSL shaders, and highly optimizable JS lambdas.

As long as you don't go overboard with the numbers, it runs fine even on mobile.

```
<root id="1" scale={600} focus={3}>
<camera id="2" proxy={true} position={[0, 0, 3]} />
<shader id="3" code="
uniform float time;
uniform float intensity;
vec4 warpVertex(vec4 xyzw, inout vec4 stpq) {
xyzw += 0.2 * intensity * (sin(xyzw.yzwx * 1.91 + time + sin(xyzw.wxyz * 1.74 + time)));
xyzw += 0.1 * intensity * (sin(xyzw.yzwx * 4.03 + time + sin(xyzw.wxyz * 2.74 + time)));
xyzw += 0.05 * intensity * (sin(xyzw.yzwx * 8.39 + time + sin(xyzw.wxyz * 4.18 + time)));
xyzw += 0.025 * intensity * (sin(xyzw.yzwx * 15.1 + time + sin(xyzw.wxyz * 9.18 + time)));
return xyzw;
}"
time=>{(t) => t / 4} intensity=>{(t) => {
t = t / 4;
intensity = .5 + .5 * Math.cos(t / 3);
intensity = 1.0 - Math.pow(intensity, 4);
return intensity * 2.5;
}} />
<reveal id="4" stagger={[10, 0, 0, 0]} enter=>{(t) => 1.0 - Math.pow(1.0 - Math.min(1, (1 + pingpong(t))*2), 2)} exit=>{(t) => 1.0 - Math.pow(1.0 - Math.min(1, (1 - pingpong(t))*2), 2)}>
<vertex id="5" pass="view">
<polar id="6" bend={1/4} range={[[-π, π], [0, 1], [-1, 1]]} scale={[2, 1, 1]}>
<transform id="7" position={[0, 1/2, 0]}>
<axis id="8" detail={512} />
<scale id="9" divide={10} unit={π} base={2} />
<ticks id="10" width={3} classes=["foo", "bar"] />
<scale id="11" divide={5} unit={π} base={2} />
<format id="12" expr={(x) => {
return x ? (x / π).toPrecision(2) + 'π' : 0
}} />
<label id="13" depth={1/2} zIndex={1} />
</transform>
<axis id="14" axis={2} detail={128} crossed={true} />
<transform id="15" position={[π/2, 0, 0]}>
<axis id="16" axis={2} detail={128} crossed={true} />
</transform>
<transform id="17" position={[-π/2, 0, 0]}>
<axis id="18" axis={2} detail={128} crossed={true} />
</transform>
<grid id="19" divideX={40} detailX={512} divideY={20} detailY={128} width={1} opacity={1/2} unitX={π} baseX={2} zBias={-5} />
<interval id="20" width={512} expr={(emit, x, i, t) => {
emit(x, .5 + .25 * Math.sin(x + t) + .25 * Math.sin(x * 1.91 + t * 1.81));
}} channels={2} />
<line id="21" width={5} />
<play id="22" pace={10} loop={true} to={3} script=[[{color: "rgb(48, 144, 255)"}], [{color: "rgb(100, 180, 60)"}], [{color: "rgb(240, 20, 40)"}], [{color: "rgb(48, 144, 255)"}]] />
</polar>
</vertex>
</reveal>
</root>
```

*Note: The JSX is a lie, you define nodes in pure JS.*

From afar there's a tree of nodes, similar to SVG tags. This is the MathBox library of vector primitives. The basic shapes are all there: points, lines, faces, vectors, surfaces, etc. These nodes are placed inside a shallow hierarchy of views and transforms.

However none of the shapes draw anything by themselves. They only know how to draw *data* supplied by a linked source. Data can be an array (static or live), a procedural source, custom JS / GLSL code, etc. This is further augmented by data operators which can be sandwiched between source and shape, forming automatic pipelines between siblings.

The current set of components looks like this:

- Group
- Inherit
- Root
- Unit

- Camera

- Axis
- Face
- Grid
- Line
- Point
- Strip
- Surface
- Ticks
- Vector

- Area
- Array
- Interval
- Matrix
- Scale
- Volume
- Voxel

- Grow
- Join
- Lerp
- Memo
- Resample
- Repeat
- Slice
- Split
- Spread
- Swizzle
- Transpose

- DOM
- HTML

- Move
- Play
- Present
- Reveal
- Slide
- Step

- Compose
- RTT

- Shader

- Format
- Label
- Text
- Retext

- Clock
- Now

- Fragment
- Layer
- Transform
- Transform4
- Vertex

- Cartesian
- Cartesian4
- Polar
- Spherical
- Stereographic
- Stereographic4
- View

To make you feel at home, nodes have an `id`

and `classes`

, and you can use CSS selectors to identify them. Nodes link up with preceding siblings and parents by default, but you can select any node in the tree. This allows for arbitrary graphs, including feedback loops. However all of this is optional: you can also pass in direct node objects or MathBox's own jQuery-like selections. What it doesn't have is a notion of detached document fragments: nodes are immediately inserted on creation.

A node's attributes can be `.get()`

and `.set()`

, though there is also a read-only `.props`

dictionary for fashionable reasons. The values are strongly typed as Three.js colors, vectors, matrices, … but accept e.g. CSS colors and ordinary arrays too. The values are normalized for immediate use, the original values are preserved on the side for printing and serialization.

What's unique is the emphasis on time. First, properties can be directly bound to time-dependent expressions, on creation or afterwards. Second, clocks are primitives on their own. This allows for nested timelines, on-demand bullet time, fast forwards and more. It even supports limited time travel, evaluating an expression several frames in the past. This can be used to ensure consistent 60 fps data logging through janky updates, useful for all sorts of things. It's exposed publicly as `.bind(key, expr)`

and `.evaluate(key, time)`

per node. It's also dogfood for declarative animation tracks. The primitives `clock`

/`now`

provide timing, while `step`

and `play`

handle keyframes on tracks.

This is definitely *a* DOM, but it has only basic features in common with *the* HTML DOM and does much less. Most of the magic comes from the components themselves. There's no cascade of styles to inherit. Children compose *with* a parent, they do not inherit *from* it, only caring about their own attributes. The namespace is clean, with no weird combo styles à la CSS. As much as possible, attributes are unique orthogonal knobs you can turn freely.

On the inside I separate the generic data model from the type-specific View Controller attached to it. The controller's job is to create and manage Three.js objects to display the node (if any). Because a data source and a visible shape have very little in common, the nodes and their controllers are blank slates built and organized around named *traits*. Each trait is a data mix-in, with associated attributes and helpers for common behavior. Primitives with the same traits can be expected to work the same, as their public facing models are identical.

Controllers can traverse the graph to find each other by matching traits, listening for events and making calls in response. This way only specific *events* will cascade through cause and effect, often skipping large parts of the hierarchy. The only way to do a "global style recalculation" would be to send a forced change event to every single controller, and there's never a reason to do so.

The controller lifecycle is deliberately kept simple: `make()`

, `made()`

, `change(…)`

, `unmake()`

, `unmade()`

. When a model changes, its controller either updates in place, or rebuilds itself, doing an unmake/make cycle. The change handler is invoked on creation as well, to encourage stateless updates. It affords live editing of anything, without having to micro-optimize every possible change scenario. Controllers can also watch bound selectors, retargeting if their matched set changes. This lets primitives link up with elements that have yet to be inserted.

Unlike HTML, the DOM is not forced to contain a render tree as well. Only some of the leaf nodes have styles and create renderables. Siblings and parents are called upon to help, but the effects don't have to be strictly hierarchical. For example, a visual effect can wrap a single leaf but still be applied after all its parents, as transformations are collected and composed in passes.

The result is not so much a *document model* as it is a *computational model* inside a *presentational model*. You can feed it finalized data and draw it directly… or you can build new models within it and reproject them live. Memoization enables feedback and meta-visualization. The line between data viz and demo scene is rarely this blurry.

Here, the notion of a *computed style* has little meaning. Any value will end up being transformed and processed in arbitrary ways down the pipe. As I've tried to explain before, the kinds of things people do with `getComputedStyle()`

and `getClientBoundingRect()`

are better achieved by having an extensible layout model, one that affords custom constraints and composition on an equal footing. To do otherwise is to admit defeat and embrace a leaky abstraction by design.

The shallow hierarchy with composition between siblings is particularly appealing to me, even if I realize it introduces non-traditional semantics more reminiscent of a command-line. It acts as both a jQuery-style chainable API, and a minimal document model. If it offends your sensibilities, you could always defuse the magic by explicitly wiring up every relationship. In case of confusion, `.inspect()`

will log syntax highlighted JSX, while `.debug()`

will draw the underlying shader graphs.

I've defined a good set of basic primitives and iterated on them a few times. But how to implement it, when WebGL doesn't even fully cover OpenGL ES 2?

- MathBox² - PowerPoint Must Die
*A DOM for Robots - Modelling Live Data*- Yak Shading - Data Driven Geometry
- ShaderGraph 2 - Functional GLSL

“The last time that I stood here was seven years ago.”

“Seven years ago! How little do you mortals understand time.

Must you be so linear, Jean-Luc?”– Picard and Q, All Good Things, Star Trek: The Next Generation

*Note: This article is significantly longer than previous instalments. It features 4 interactive slideshows, each introducing a new tool as well as related concepts around it. In one way, it's just another math guide, but going much deeper. In another, it's a thesis on everything I know about animating. Their intersection is a handbook for anyone who wants to make things move with code, but I hope it's an interesting read even if that's not your goal.*

Developers have a tough job these days. A seamless experience is mandatory, polish is expected. On touch devices, they are expected to become magicians. The trick is to make an electronic screen look and feel like something you can physically manipulate. Animation is the key to all of this.

Not just any animation though. Flash intros were hated for a reason. The `<blink>`

tag is not your friend, and flashing banner ads only annoy rather than invite. If elaborately designed effects distract from the content, or worse, ruin smoothness and performance, it'll turn people off rather than endear. Animation can only add value when its fast and fluid enough to be responsive.

It's not mere polish either, a finishing touch. Animation–and UI in general—should always be an additional conversation with the user, not a representation of internal software or hardware state. When we press Play in a streaming music app, the app should respond immediately by showing a Pause control, even if the music won't actually start playing for another second. When we enable Airplane Mode on our phones, we don't care that it'll take a few seconds to say good bye to the cell tower and turn off the radio. The UI is there to respond to our wishes: it should act like a personal assistant, not a reluctant helper, or worse, a demanding master.

Hence animation is visual language and communicates both explicitly and implicitly. It establishes an unspoken trust and confidence between designer and user: we promise nothing will appear, change or disappear without explanation. It can show where to find things, like an application that minimizes into place in the dock, or a picture sliding into a thumbnail strip. It can tell miniature stories, like a Download button turning into a progress bar turning into a checkmark. More simply just the act of scrolling around a live document, creating the illusion of viewing an infinite canvas, persisting in space and time. Here, page layout is the use of placement and style to denote hierarchy and meaning in a 2D space.

As with any conversation, tone matters, in this case expressed through choreography. Items can fade into the background or pop to demand our attention, expressing calm or assertiveness. Elements can complement or oppose, creating harmony or dissonance. Animations can be minimalist or baroque, ordered or parallel, independent or layered. The proper term for this is *staging*, and research shows that it can significantly increase our understanding of diagrams and graphs when applied carefully. Whenever elements transition, preferably one at a time, it is easier to gauge changes in color, size, shape and position than when we are only shown a before and after shot.

This is important everywhere, but especially so for abstract topics like data visualization and mathematics. When we have no natural mental model of something, we build our understanding based on the interface we use to examine it. The more those interfaces act like real objects, the less surprising they are.

In doing so, we replace explicit explanations with implicit metaphors from the natural world: distance, direction, scale, shadow, color, contrast. These are the cues our brains evolved to be excellent at interpreting. By imbuing virtual objects with these properties, we make them more realistic and thus more understandable. Mind you, this is not a call for *skeuomorphism*, far from it. The properties we are seeking to mimic are far more basic, far more important, than some faux leather and stitching.

The clearest example of this has to be inertial scrolling. Compared to an ordinary mouse wheel, scrolling on a tablet is actually much more complicated. We can flick and grab, go as fast or slow as we want. When skimming through a list, often we never wait for the page to stop moving, in theory requiring more effort to read. Yet everyone who's seen a toddler with an iPad can attest to its uncanny ease of use and efficiency, offering improved control and comprehension. Our brains are very good at tracking and manipulating objects in motion, particularly when they obey the laws of physics: moving with *consistent inertia and force*.

Which brings me to the actual topic of this post: how animation works on a fundamental level. I'd like to teach a mental model based on physics and math, and how to precisely control it. Along the way, we'll come to understand why Apple built a physics engine into iOS 7's UI, reveal some secrets of the demoscene, compose fluid keyframe animations, and defeat the final boss: seamless rotation in 3D. In doing so, we'll also go beyond just visual animation. The techniques described here work equally well for manipulating audio, processing data or driving meatspace devices. In a world of data, animation is just a different word for *precise control*.

An animation is something that *changes over time*. As it so happens, these three humble words are a veritable Pandora's box of mathematics. They open up to the strange world of the continuously and infinitely dividable, also known as *calculus*.

In a previous article, I covered the origins of calculus and how to approach the concept of infinity. In what follows, we won't be needing it much though. We'll be working with finite steps throughout, with *discrete* time. This makes it vastly easier to understand, and is an eminently useful stepping stone to the true theory of continuous motion, which you can find in any good physics textbook.

Math class hates it when we just punch numbers into our calculator instead of deducing the exact result: a decimal number is meaningless on its own. On that, I can agree. But when we punch in a couple thousand numbers and look at them in aggregate, it can tell us just as much. This page will be your calculator.

We've seen how to examine an animation at multiple levels of change: position, velocity, acceleration. Differences approximate *derivatives* and let us to dissect our way down the chain. Accumulators approximate *integration* and let us construct higher levels from lower ones. Thus we can manipulate an animation at any level. By plugging in correct physical laws or arbitrary formulas, we can produce behavior that is as physical or unphysical as we like.

Everything we've done so far has been independent animation, without interaction. Even inertial scrolling has this luxury: whenever the user is touching, there is no inertia and the animation system is inactive. It's only when you let go that the surface coasts.

In many cases, this is not enough: animations need to be scheduled and executed while retaining full interactivity. Often the animation needs to continue despite its target changing midway. In order to handle such situations, we need to build adaptive models that remain continuous and smooth, no matter what.

We'll also need to drop the assumption that the frame rate—the time step—is constant. In the real world, the frame rate might drop here or there, or be variable altogether. In either case, we'd prefer it if the effect of this was minimal. If we're adding music to an animation, this is essential to prevent desynchronization. It will also have some nasty consequences for our physics engine, and we need to level it up significantly.

By manipulating time, we've managed to eliminate frame rate issues altogether, even make it work to our advantage. We've discovered more accurate physics engines, so we don't have to waste time simulating tiny steps. We've also created interruptible animations and turned them into filters. We can choose their easing curves and use feedback systems to remove the need for any manual interruptions altogether.

Here, *linear time-invariant* systems are very useful building blocks: they are simple to implement, but eminently customizable. Both IIR and FIR filters are simple in their basic form. We can also combine feedback systems with other physical or unphysical forces: we can move the target any way we like, perhaps superimposing variation onto existing curves. If we broaden our horizons a bit, we can find applications outside of animation: data analysis, audio manipulation, image processing, and much, much more.

Of course, there are plenty of non-linear and/or non-time-invariant systems too, too many to cover. When dealing with animation though, we'll prefer systems based on physics. They're just the trick to turn a bunch of artificial data into something that feels slick and natural. That said, physics itself is sometimes non-linear: fluids like water, smoke or fire are perfect examples. Solving those particular boondoggles requires the kind of calculus that frightens most adults and large children, so we won't go into that here. It's the same thing though: you simulate it finitely with a couple of clever tricks and the awesome power of raw number crunching.

*Continued in part two.*

“It is known that there are an infinite number of worlds, simply because there is an infinite amount of space for them to be in. However, not every one of them is inhabited. Therefore, there must be a finite number of inhabited worlds.

Any finite number divided by infinity is as near to nothing as makes no odds, so the average population of all the planets in the universe can be said to be zero. From this it follows that the population of the whole universe is also zero, and that any people you may meet from time to time are merely the products of a deranged imagination.”– The Restaurant at the End of the Universe, Douglas Adams

If there's one thing mathematicians have a love-hate relationship with, it has to be *infinity*. It's the ultimate tease: it beckons us to come closer, but never allows us anywhere near it. No matter how far we travel to impress it, infinity remains disinterested, equally distant from everything: infinitely far!

$$ 0 < 1 < 2 < 3 < … < \infty $$

Yet infinity is not just desirable, it is absolutely necessary. All over mathematics, we find problems for which no finite amount of steps will help resolve them. Without infinity, we wouldn't have real numbers, for starters. That's a problem: our circles aren't round anymore (no $ π $ and $ \tau $) and our exponentials stop growing right (no $ e $). We can throw out all of our triangles too: most of their sides have exploded.

A steel railroad bridge with a 1200 ton counter-weight.

Completed in 1910. Source: Library of Congress.

We like infinity because it helps avoid all that. In fact even when things are not infinite, we often prefer to pretend they are—we do geometry in infinitely big planes, because then we don't have to care about where the edges are.

Now, suppose we want to analyze a steel beam, because we're trying to figure out if our proposed bridge will stay up. If we want to model reality accurately, that means simulating each individual particle, every atom in the beam. Each has its own place and pushes and pulls on others nearby.

But even just $ 40 $ grams of pure iron contains $ 4.31 \cdot 10^{23} $ atoms. That's an inordinate amount of things to keep track of for just 1 teaspoon of iron.

Instead, we pretend the steel is solid throughout. Rather than being composed of atoms with gaps in between, it's made of some unknown, filled in material with a certain density, expressed e.g. as *grams per cubic centimetre*. Given any shape, we can determine its volume, and hence its total mass, and go from there. That's much simpler than counting and keeping track of individual atoms, right?

Unfortunately, that's not quite true.

Like all choices in mathematics, this one has consequences we cannot avoid. Our beam's density is *mass per volume*. Individual points in space have zero volume. That would mean that at any given point inside the beam, the amount of mass there is $ 0 $. How can a beam that is entirely composed of nothing be solid and have a non-zero mass?

*Bam! No more iron anywhere.*

While Douglas Adams was being deliberately obtuse, there's a kernel of truth there, which is a genuine paradox: what exactly is the mass of every atom in our situation?

To make our beam solid and continuous, we had to shrink every atom down to an infinitely small point. To compensate, we had to create infinitely many of them. Dividing the finite mass of the beam between an infinite amount of atoms should result in $ 0 $ mass per atom. Yet all these masses still have to add up to the total mass of the beam. This suggests $ 0 + 0 + 0 + … > 0 $, which seems impossible.

If the mass of every atom were not $ 0 $, and we have infinitely many points inside the beam, then the total mass is infinity times the atomic mass $ m $. Yet the total mass is finite. This suggests $ m + m + m + … < \infty $, which also doesn't seem right.

It seems whatever this number $ m $ is, it can't be $ 0 $ and can't be non-zero. It's definitely not infinite, we only had a finite mass to begin with. It's starting to sound like we'll have to invent a whole new set of numbers again to even find it.

That's effectively what Isaac Newton and Gottfried Leibniz set in motion at the end of the 17th century, when they both discovered calculus independently. It was without a doubt the most important discovery in mathematics and resulted in formal solutions to many problems that were previously unsolvable— our entire understanding of physics has relied on it since. Yet it took until the late 19th century for the works of Augustin Cauchy and Karl Weierstrass to pop up, which formalized the required theory of *convergence*. This allows us to describe exactly how differences can shrink down to nothing as you approach infinity. Even that wasn't enough: it was only in the 1960s when the idea of infinitesimals as fully functioning numbers—the hyperreal numbers—was finally proven to be consistent enough by Abraham Robinson.

But it goes back much further. Ancient mathematicians were aware of problems of infinity, and used many ingenious ways to approach it. For example, $ π $ was found by considering circles to be infinite-sided polygons. Archimedes' work is likely the earliest use of *indivisibles*, using them to imagine tiny mechanical levers and find a shape's center of mass. He's better known for running naked through the streets shouting Eureka! though.

That it took so long shows that this is not an easy problem. The proofs involved are elaborate and meticulous, all the way back. They have to be, in order to nail down something as tricky as infinity. As a result, students generally learn calculus through the simplified methods of Newton and Leibniz, rather than the most mathematically correct interpretation. We're taught to mix notations from 4 different centuries together, and everyone's just supposed to connect the dots on their own. Except the trail of important questions along the way is now overgrown with jungle.

Still, it shows that even if we don't understand the whole picture, we can get a lot done. This article is in no way a formal introduction to infinitesimals. Rather, it's a demonstration of why we might need them.

What is happening when we shrink atoms down to points? Why does it make shapes solid yet seemingly hollow? Is it ever meaningful to write $ x = \infty $? Is there only one infinity, or are there many different kinds?

To answer that, we first have to go back to even simpler times, to Ancient Greece, and start with the works of Zeno.

Zeno of Elea was one of the first mathematicians to pose these sorts of questions, effectively trolling mathematics for the next two millennia. He lived in the 5th century BC in southern Italy, although only second-hand references survive. In his series of paradoxes, he examines the nature of equality, distance, continuity, of time itself.

Because it's the ancient times, our mathematical knowledge is limited. We know about zero, but we're still struggling with the idea of nothing. We've run into negative numbers, but they're clearly absurd and imaginary, unlike the positive numbers we find in geometry. We also know about fractions and ratios, but square roots still confuse us, even though our temples stay up.

Limits are the first tool in our belt for tackling infinity. Given a sequence described by countable steps, we can attempt to extend it not just to the end of the world, but literally forever. If this works we end up with a finite value. If not, the limit is undefined. A limit can be equal to $ \infty $, but that's just shorthand for *the sequence has no upper bound*. Negative infinity means no lower bound.

Until now we've only encountered fractions, that is, *rational numbers*. Each of our sums was made of fractions. The limit, if it existed, was also a rational number. We don't know whether this was just a coincidence.

It might seem implausible that a sequence of numbers that is 100% rational and converges, can approach a limit that isn't rational at all. Yet we've already seen similar discrepancies. In our first sequence, every partial sum was *less than* $ 1 $. Meanwhile the limit of the sum was *equal to* $ 1 $. Clearly, the limit does not have to share all the properties of its originating sequence.

We also haven't solved our original problem: we've only chopped things up into infinitely many *finite pieces*. How do we get to *infinitely small pieces*? To answer that, we need to go looking for *continuity*.

Generally, continuity is defined by what it is and what its properties are: a noticeable lack of holes, and no paradoxical values. But that's putting the cart before the horse. First, we have to show which holes we're trying to plug.

What we just did was a careful exercise in hiding the obvious, namely the digit-based number systems we are all familiar with. By viewing them not as digits, but as paths on a directed graph, we get a new perspective on just what it means to use them. We've also seen how this means we can construct the rationals and reals using the least possible ingredients required: division by two, and limits.

In school, we generally work with the decimal representation of numbers. As a result, the popular image of mathematics is that it's the science of *digits*, not the underlying structures they represent. This permanently skews our perception of what numbers really are, and is easy to demonstrate. You can google to find countless arguments of why $ 0.999… $ is or isn't equal to $ 1 $. Yet nobody's wondering why $ 0.000… = 0 $, though it's practically the same problem: $ 0.1, 0.01, 0.001, 0.0001, … $

Furthermore, in decimal notation, rational numbers and real numbers look incredibly alike: $ 3.3333… $ vs $ 3.1415…\, $ The question of what it actually means to have infinitely many non-repeating digits, and why this results in continuous numbers, is hidden away in those 3 dots at the end. By imagining $ π $ as $ 3.1415…0000… $ or $ 3.1415…1111… $ we can intuitively bridge the gap to the infinitely small. We see how the distance between two neighbouring real numbers must be so small, that it really is equivalent to $ 0 $.

That's not as crazy as it sounds. In the field of *hyperreal numbers*, every number actually has additional digits 'past infinity': that's its infinitesimal part. You can imagine this to be a multiple of $ \frac{1}{\infty} $, an infinitely small unit greater than $ 0 $, which I'll call $ ε $. You can add $ ε $ to a real number to take an infinitely small step. It represents a difference that can only be revealed with an infinitely strong microscope. Equality is replaced with *adequality*: being equal aside from an infinitely small difference.

You can explore this hyperreal number line below.

As $ ε $ is a fully functioning hyperreal number, $ ε^2 $ is also infinitesimal. In fact, it's even infinitely smaller than $ ε $, and we can keep doing this for $ ε^3, ε^4, …\,$ To make matters worse, if $ ε $ is infinitesimal, then $ \frac{1}{ε} $ must be infinitely big, and $ \frac{1}{ε^2} $ infinitely bigger than that. So hyperreal numbers don't just have inwardly nested infinitesimal levels, but outward levels of increasing infinity too. They have infinitely many dimensions of infinity both ways.

So it's perfectly possible to say that $ 0.999… $ does not equal $ 1 $, if you mean they differ by an infinitely small amount. The only problem is that in doing so, you get much, *much* more than you bargained for.

That means we can finally answer the question we started out with: why did our continuous atoms seemingly all have $ 0 $ mass, when the total mass was not $ 0 $? The answer is that the mass per atom was *infinitesimal*. So was each atom's volume. The density, *mass per volume*, was the result of dividing one infinitesimal amount by another, to get a normal sized number again. To create a finite mass in a finite volume, we have to add up infinitely many of these atoms.

These are the underlying principles of calculus, and the final puzzle piece to cover. The funny thing about calculus is, it's conceptually easy, especially if you start with a good example. What is hard is actually working with the formulas, because they can get hairy very quickly. Luckily, your computer will do them for you:

That was differential and integral calculus in a nutshell. We saw how many people actually spend hours every day sitting in front of an *integrator*: the odometers in their cars, which integrate speed into distance. And the derivative of speed is acceleration—i.e. how hard you're pushing on the gas pedal or brake, combined with forces like drag and friction.

By using these tools in equations, we can describe laws that relate quantities to their *rates of change*. Drag, also known as air resistance, is a force which gets stronger the faster you go. This is a relationship between the first and second derivatives of position.

In fact, the relaxation procedure we applied to our track is equivalent to another physical phenomenon. If the curve of the coaster represented the temperature along a thin metal rod, then the heat would start to equalize itself in exactly that fashion. Temperature wants to be smooth, eventually averaging out completely into a flat curve.

Whether it's heat distribution, fluid dynamics, wave propagation or a head bobbing in a roller coaster, all of these problems can be naturally expressed as so called *differential equations*. Solving them is a skill learned over many years, and some solutions come in the form of infinite series. Again, infinity shows up, ever the uninvited guest at the dinner table.

Infinity is a many splendored thing but it does not lift us up where we belong. It boggles our mind with its implications, yet is absolutely essential in math, engineering and science. It grants us the ability to see the impossible and build new ideas within it. That way, we can solve intractable problems and understand the world better.

What a shame then that in pop culture, it only lives as a caricature. Conversations about infinity occupy a certain sphere of it—Pink Floyd has been playing on repeat, and there's usually someone peddling crystals and incense nearby.

*"Man, have you ever, like, tried to imagine infinity…?"* they mumble, staring off into the distance.

"Funny story, actually. We just came from there…"

*Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.*

*More like this: How to Fold a Julia Fractal.*

"Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy. And yet," Death waved a hand, "And yet you act as if there is some ideal order in the world, as if there is some… some rightness in the universe by which it may be judged."– The Hogfather, Discworld, Terry Pratchett

Mathematics has a dirty little secret. Okay, so maybe it's not so dirty. But neither is it little. It goes as follows:

*Everything in mathematics is a choice.*

You'd think otherwise, going through the modern day mathematics curriculum. Each theorem and proof is provided, each formula bundled with convenient exercises to apply it to. A long ladder of subjects is set out before you, and you're told to climb, climb, climb, with the promise of a payoff at the end. "You'll need this stuff in real life!", they say, oblivious to the enormity of this lie, to the fact that most of the educated population walks around with *"vague memories of math class and clear memories of hating it."*

Rarely is it made obvious that all of these things are entirely optional—that mathematics is the art of making choices so you can discover what the consequences are. That algebra, calculus, geometry are just words we invented to group the most interesting choices together, to identify the most useful tools that came out of them. The act of mathematics is to play around, to put together ideas and see whether they go well together. Unfortunately that exploration is mostly absent from math class and we are fed pre-packaged, pre-digested math pulp instead.

And so it also goes with the numbers. We learn about the natural numbers, the integers, the fractions and eventually the real numbers. At each step, we feel hoodwinked: we were only shown a part of the puzzle! As it turned out, there was a 'better' set of numbers waiting to be discovered, more comprehensive than the last.

Along the way, we feel like our intuition is mostly preserved. Negative numbers help us settle debts, fractions help us divide pies fairly, and real numbers help us measure diagonals and draw circles. But then there's a break. If you manage to get far enough, you'll learn about something called the *imaginary numbers*, where it seems sanity is thrown out the window in a variety of ways. Negative numbers can have square roots, you can no longer say whether one number is bigger than the other, and the whole thing starts to look like a pointless exercise for people with far too much time on their hands.

I blame it on the name. It's misleading for one very simple reason: all numbers are imaginary. You cannot point to anything in the world and say, "This is a 3, and that is a 5." You can point to three apples, five trees, or chalk symbols that represent 3 and 5, but the concepts of 3 and 5, the numbers themselves, exist only in our heads. It's only because we are taught them at such a young age that we rarely notice.

So when mathematicians finally encountered numbers that acted just a little bit different, they couldn't help but call them *fictitious* and *imaginary*, setting the wrong tone for generations to follow. Expectations got in the way of seeing what was truly there, and it took decades before the results were properly understood.

Now, this is not some esoteric point about a mathematical curiosity. These imaginary numbers—called *complex numbers* when combined with our ordinary real numbers—are essential to quantum physics, electromagnetism, and many more fields. They are naturally suited to describe anything that turns, waves, ripples, combines or interferes, with itself or with others. But it was also their unique structure that allowed Benoit Mandelbrot to create his stunning fractals in the late 70s, dazzling every math enthusiast that saw them.

Yet for the most part, complex numbers are treated as an inconvenience. Because they are inherently multi-dimensional, they defy our attempts to visualize them easily. Graphs describing complex math are usually simplified schematics that only hint at what's going on underneath. Because our brains don't do more than 3D natively, we can glimpse only slices of the hyperspaces necessary to put them on full display. But it's not impossible to peek behind the curtain, and we can gain some unique insights in doing so. All it takes is a willingness to imagine something different.

So that's what this is about. And a lesson to be remembered: complex numbers are typically the first kind of numbers we see that are undeniably strange. Rather than seeing a sign that says *Here Be Dragons, Abandon All Hope*, we should explore and enjoy the fascinating result that comes from one very simple choice: *letting our numbers turn*. That said, there *are* dragons. Very pretty ones in fact.

What does it mean to let numbers turn? Well, when making mathematical choices, we have to be careful. You could declare that $ 1 + 1 $ should equal $ 3 $, but that only opens up more questions. Does $ 1 + 1 + 1 $ equal $ 4 $ or $ 5 $ or $ 6 $? Can you even do meaningful arithmetic this way? If not, what good are these modified numbers? The most important thing is that our rules need to be consistent for them to work. But if all we do is swap out the *symbols* for $ 2 $ and $ 3 $, we didn't actually change anything in the underlying mathematics at all.

So we're looking for choices that don't interfere with what already works, but add something new. Just like the negative numbers complemented the positives, and the fractions snugly filled the space between them—and the reals somehow fit in between *that*—we need to go look for new numbers where there currently aren't any.

We've seen how complex numbers are arrows that like to turn, which can be made to behave like numbers: we can add and multiply them, because we can come up with a consistent rule for doing so. We've also seen what powers of complex numbers look like: we fold or unfold the entire plane by multiplying or dividing angles, while simultaneously applying a power to the lengths.

With a basic grasp of what complex numbers are and how they move, we can start making Julia fractals.

At their heart lies the following function:

$$ f(z) = z^2 + c $$

This says: map the complex number $ z $ onto its square, and then add a constant number to it. To generate a Julia fractal, we have to apply this formula repeatedly, feeding the result back into $ f $ every time.

$$ z_{n+1} = (z_n)^2 + c $$

We want to examine how $ z_n $ changes when we plug in different starting values for $ z_1 $ and iterate $ n $ times. So let's try that and see what happens.

Making fractals is probably the least useful application of complex math, but it's an undeniably fascinating one. It also reveals the unique properties of complex operations, like conformal mapping, which provide a certain rigidity to the result.

However, in order to make complex math practical, we have to figure out how to tie it back to the real world.

It's a good thing we don't have to look far to do so. Whenever we're describing wavelike phenomena, whether it's sound, electricity or subatomic particles, we're also interested in how the wave evolves and changes. Complex operations are eminently suited for this, because they naturally take place on circles. Numbers that oppose can cancel out, numbers in the same direction will amplify each other, just like two waves do when they meet. And by folding or unfolding, we can alter the frequency of a pattern, doubling it, halving it, or anything in between.

More complicated operations are used for example to model electromagnetic waves, whether they are FM radio, wifi packets or ADSL streams. This requires precise control of the frequencies you're generating and receiving. Doing it without complex numbers, well, it just sucks. So why use boring real numbers, when complex numbers can do the work for you?

In visualizing complex waves, we've seen functions that map real numbers to complex numbers, and back again. These can be graphed easily in 3D diagrams, from $ \mathbb{R} $ to $ \mathbb{C} $ or vice-versa. You cross 1 real dimension with the 2 dimensions of the complex plane.

But complex operations in general work from $ \mathbb{C} $ to $ \mathbb{C} $. To view these, unfortunately you need four-dimensional eyes, which nature has yet to provide. There are ways to project these graphs down to 3D that still somewhat make sense, but it never stops being a challenge to interpret them.

For every mathematical concept that we have a built-in intuition for, there are countless more we can't picture easily. That's the curse of mathematics, yet at the same time, also its charm.

Hence, I tried to stick to the stuff that is (somewhat!) easy to picture. If there's interest, a future post could cover topics like: the nature of $ e^{ix} $, Fourier transforms, some actual quantum mechanics, etc.

For now, this story is over. I hope I managed to spark some light bulbs here and there, and that you enjoyed reading it as much as I did making it.

Comments, feedback and corrections are welcome on Google Plus. Diagrams powered by MathBox.

*More like this: To Infinity… And Beyond!.*

*For extra credit: check out these great stirring visualizations of Julia and Mandelbrot sets. I incorporated a similar graphic above. Hat tip to Tim Hutton for pointing these out. And for some actual paper mathematical origami, check out Vihart's latest video on Snowflakes, Starflakes and Swirlflakes.*

For most of my life, I've found math to be a visual experience. My math scores went from crap to great once I started playing with graphics code, found some demoscene tutorials, and realized I could reason about formulas by picturing the graphs they create. I could apply operators by learning how they morph, shift, turn and fold those graphs and create symmetries. I could remember equations and formulas more easily when I could layer on top the visual relationships they embody. I was less likely to make mistakes when I could augment the boring symbolic manipulation with a mental set of visual cross-checks.

So, when tasked with holding a conference talk on how to make things out of math at Full Frontal—later redone at Web Directions Code—I knew the resulting presentation would have to consist of intricate visualizations as the main draw, with whatever I had to say as mere glue to hold it together.

The problem was, I didn't know of a good tool to do so, and creating animations by hand would probably be too time consuming. With the writings of Paul Lockhart and Bret Victor firmly in mind, I also knew I wanted to start blogging more about mathematical concepts in a non-traditional way, showing the principles of calculus, analysis and algebra the way I learnt to see them in my head, rather than through the obscure symbols served up in engineering school.

So I set out to create that tool, keeping in mind the most important lesson I've picked up as a web developer: one cannot overstate the value in being able to send someone a link and have it just work, right there. It was obvious it would have to be browser-based.

*2015 Update: MathBox has evolved, version 2 is now available*.

Now, when people think of graphs in a browser, the natural thought is vector graphics and SVG, which quickly leads to visualization powerhouse d3.js. It really is an amazing piece of tech with a vast library of useful code to accompany it. When I wrapped my head around how d3's enter/exit selections are implemented and how little it actually does to achieve so much, I was blown away. It's just so elegant and simple.

Unfortunately, d3's core is intricately tied to the DOM through SVG and CSS. And that means ironically that d3 is not really capable of 3D. Additionally, d3 is a power tool that makes no assumptions: it is up to you to choose which visual elements and techniques to use to make your diagrams, and as such it is more like assembly language for graphs than a drop-in tool. These two were show stoppers.

For one, manually designing layouts, grids, axes, etc. every time is tedious. You should be able to drop in a mathematical expression with as little fanfare as possible and have it come out looking right. This includes sane defaults for transitions and animations.

For another, I've found that, when in doubt, adding an extra dimension always helps. The moment I finally realized that every implicit graph in N dimensions is really just a slice of an explicit one in N+1 dimensions, a ridiculous amount of things clicked together. And it took until years after studying signal processing to at long last discover the 4D picture of complex exponentiation that tied the entire thing together (projected into 3D below): it revealed the famous "magic formula" involving e, i and π to be a meaningless symbological distraction, a pinhole view of a much larger, much more beautiful structure, underpinning every Fourier and Z transform I'd ever encountered.

So, WebGL it was, because I needed 3D. Unfortunately that meant the promise of having it just work everywhere was tempered by a lack of browser support, but I would certainly hope that's something we can overcome sooner than later. Dear Apple and Microsoft: get your shit together already. Dear Firefox and Opera: your WebGL performance could be a lot better.

These days I don't really touch WebGL without going through Three.js first. Three.js is a wonderful, mature engine that contains tons of useful high-level components. At the same time, it also does a great job in just handling the boilerplate of WebGL while not getting in the way of doing some heavy lifting yourself.

Rendering vector-style graphics with WebGL is not hard, certainly easier than photorealistic 3D. Primitives like lines and points are sized in absolute pixels by default, and with hardware multisampling for anti-aliasing, you get somewhat decent image quality out of it. Though, as is typical for a Web API, we're treated like children and can only cross our fingers and *request* anti-aliasing politely, hoping it will be available. Meanwhile native developers have full control over speed and quality and can adjust their strategy to the specific hardware's capabilities. The more things change... And then Chrome decided to disable anti-aliasing altogether due to esoteric security issues with buggy drivers. Bah.

Now, when rendering with WebGL, you really have two options. One is to just treat it as a dumb output layer, loading or generating all your geometry in JavaScript and rendering it directly in 3D. With the speed of JS engines today, this can get you pretty far.

The second option is to leverage the GPU's own capabilities as much as possible, doing computations in GLSL through so-called vertex and fragment shader programs. These are run for every vertex in a mesh, every pixel being drawn, and have been the main force driving innovation in real-time graphics for the past decade. With the goal of butter-smooth 60fps graphical goodness, this seemed like the better choice.

Unfortunately, GLSL shaders are rather monolithic things. While you do have the ability to create subroutines, every shader still has to be a stand-alone program with its own main() function. This means you either need to include a shader for every possible combination of operations, or generate shader code dynamically by concatenating pre-made snippets or using #ifdef switches to knock them out. This is the approach taken by Three.js, which results in some very hairy code that is neither easy to read nor easy to maintain.

Having made a prototype, I knew I wanted to show continuous transitions between various coordinate systems (e.g. polar and spherical), knew I needed to render shaded and unshaded geometry, and knew I would need to slot in specific snippets for things like point sprites, bezier curves/surfaces, dynamic tick marks, and more. Sorting this all out Three.js-style would be a nightmare.

So I wrote a library to solve that problem, called ShaderGraph.js. It is best described as a smart code-concatenator, a few steps short of writing a full blown compiler. You feed it snippets of GLSL code, each with one or more inputs and outputs, and these get parsed and turned into lego-like building blocks. Each input/output becomes an outlet, and outlets are wired up in a typical dataflow style. Given a graph of connected snippets, it can be compiled back into a program by assembling the subroutines, assigning intermediate variables and constructing an appropriate main() function to invoke them. It also exports a list of all external variables, i.e. GLSL uniforms and attributes, so you can control the program's behavior easily.

If I'd stopped there however, I'd have just replaced the act of manual code writing with that of manually wiring graphs. So I applied the principle of convention-over-configuration instead: you tell ShaderGraph to connect two snippets, and it will automatically match up outlets by name and type. This is augmented by a chainable factory API, which allows you to pass a partially built graph around. It allows different classes to work together to build shaders, each inserting their own snippets into the processing chain.

For example, to render a Bezier surface, the vertex shader is composed of: cubic interpolation, viewport transform (position + tangents), normal calculation and lighting. When transforming to e.g. a polar viewport, the surface normals are seamlessly recalculated. It really works like magic and I can't wait to use this in my next WebGL projects.

At its core, Three.js matches pretty directly with WebGL. You can insert objects such as a Mesh, Line or ParticleSystem into your scene, which invokes a specific GL drawing command with high efficiency. As such, I certainly didn't want to reinvent the wheel.

Hence, MathBox is set up as a sort of scene-manager-within-a-scene-manager. It's a little sandbox that speaks the language of math, allowing you to insert various *primitives* like curves, vectors, axes and grids. Each of these primitives then instantiates one or more *renderables*, which simply wrap a native Three.js object and its associated ShaderGraph material. Thus, once instantiated, MathBox gets out of the way and Three.js does the heavy lifting as normal. You can even insert multiple mathboxen into a Three.js scene if you like, mixed in with other objects.

For example, a vector primitive is rendered as an arrow: it consists of a shaft and an arrowhead, realized as a line segment and a cone. An axis primitive is an arrow as well, but it also has tick marks (specially transformed line segments), and is positioned implicitly just by specifying the axis' direction rather than a start and end point.

To render curves and surfaces, you can either specify an array of data points or a live expression to be evaluated at every point. This turned out to be essential for the kinds of intricate visualizations I wanted to show, my slides being driven by timed clocks, shared arrays of data points, and live formulas and interpolations. I even fed in data from a physics engine, and it worked perfectly.

This is all tied together through Viewport objects, which define a specific mapping from a mathematical coordinate space into the 3D world space of Three.js. For example, the default cartesian viewport has the range [–1, 1] in the X, Y and Z directions. Altering the viewport's extents will shift and scale anything rendered within, as well as reflow grids and tick marks on each axis.

There are two more sophisticated viewport types, polar and spherical, which each apply the relevant coordinate transform, and can transition smoothly to and from cartesian. More viewport types can be added, all that is required is to define an appropriate transformation in JavaScript and GLSL. That said, defining a seamless transition to and from cartesian space is not always easy, particularly if you want to preserve the aspect-ratio through the entire process.

Finally, I had to tackle the problem of animation, keeping in mind a tip I learnt from the ever so mindbending Vihart: "If I can draw the point of a sentence, I don't actually need to say the sentence." This applies doubly so for animation: every time you replace a "before" and "after" with a smooth transition, your audience implicitly understands the change rather than having to go look for it.

Hence, each primitive can be fully animated. Each has a set of options (controlling behavior) and styles (controlling GLSL shaders), and there is a universal animator that can interpolate between arbitrary data types in a smart fashion.

For example, given a viewport with the XYZ range [[–1, 1], [–1, 1], [–1, 1]], you can tell it to animate to [[0, 2], [0, 1], [–3, 3]], and it just works. The animator will recursively animate each subarray's elements, and any dependent objects like grids and axes will reflow to match the intermediate values. This works for colors, vectors and matrices too. In case of live curves with custom expressions, the animator will invoke both the old and the new, and interpolate between the results.

However, executing animations manually in code is tedious, particularly in a presentation, where you want to be able to step forward and backward. So I added a Director class whose job it is to coordinate things. All you do is feed it a script of steps (add this object, animate that object). Then, as it applies them, it remembers the previous state of each object and generates an automatic rollback script. It also contains logic to detect rapid navigation, and will hurry up animations appropriately. This avoids that agonizing situation of watching someone skip through their slide deck, playing the same cheesy PowerPoint transitions over and over again.

With MathBox's core working, it was time to build my slides for the conference. After a quick survey, I quickly settled on deck.js as an HTML5 slidedeck solution that was clean and flexible enough for my purposes. However, while MathBox can be spawned inside any DOM element, it wouldn't work to insert a dozen live WebGL canvases into the presentation. The entire thing would grind to a halt or at least become very choppy.

So instead, I integrated each MathBox graphic as an IFRAME, and added some logic that only loads each IFRAME one slide before it's needed, and unloads it one slide after it's gone off screen. To sync up with the main presentation, all deck.js navigation events were forwarded into each active IFRAME using *window.postMessage*. With the MathBox Director running inside, this was very easy to do, and meant that I could skip around freely during the talk, without any worries of desynchronization between MathBox and the associated HTML5 overlays.

In fact, I applied a similar principle to this post. To avoid rendering all diagrams simultaneously and spinning up laptop fans more than necessary, each MathBox IFRAME is started as it scrolls into view and stopped once it's gone.

I've also found that having a handheld clicker makes a huge difference while speaking—as it allows you to gesture freely and move around. So, I grabbed the infrared remote code from VLC and built a simple bridge from to Cocoa to Node.js to WebSocket to allow the remote to work in a browser. It's a shame Apple's decided to discontinue IR ports on their laptops. I guess I'll have to come up with a BlueTooth-based solution when I upgrade my hardware.

In its current state, MathBox is still a bit rough. The selection of primitives and viewports is limited, and only includes the ones I needed for my presentation. That said, it is obvious you can already do quite a lot with it, and I couldn't have been happier to hear that all this effort had the desired response at the conference. I wasn't 100% sure whether other people would have the same a-ha moments that I've had, but I'm convinced more than ever that seeing math in motion is essential for honing our intuition about it. MathBox not only makes animated diagrams much easier to make and share, but it also opens the door to making them interactive in the future.

I plan to continue to evolve MathBox as needed by using it on this site and addressing gaps that come up, though I've already identified a couple of sore points:

- I used tQuery as a boilerplate and because I liked the idea of having a chainable API for this. However, this also means it's currently running off an outdated version of Three.js. I need to look into updating and/or dropping tQuery.

MathBox has been updated to Three.js r53. - Numeric or text labels are completely unsupported. It should be possible to use my CSS3D renderer for Three.js to layer on beautifully typeset MathJax formulas, positioning them correctly in 3D on top of the WebGL render.

I've added labeling for axes. I've integrated MathJax, but it's tricky because the typesetting is painfully slow in the middle of a 60fps render. But it's automatically used if MathJax is present. - All styles have to be specified on a per-object basis. Some form of stylesheet, default styles or class mechanism to allow re-use seems like an obvious next step.
- There are undoubtedly memory leaks, as I was focused first and foremost on getting it to work.
- Expressions that don't change frame-to-frame are still continuously re-evaluated, which is wasteful. There is a
`live: false`

flag you can set on objects, but it triggers a few bugs here and there. - There needs to be a predictable, built-in way of running a clock per slide to sync custom expressions off of. In my presentation I used a hack of clocks that start once first invoked, but this lacks repeatability.

I added a`director.clock()`

method that gives you a clock per slide.

Finally, it doesn't take much imagination to imagine a MathBox Editor that would allow you to build diagrams visually rather than having to use code like I did. However, that's a can of worms I'm not going to open by myself, especially because the API is already quite straightforward to use, and the library itself is still a bit in flux. Perhaps this could be done as an extension of the Three.js editor.

You can see what MathBox is really capable of in the conference video. I invite you to play around with MathBox and see what you can make it do. Contributions are welcome, and the architecture is modular enough to allow its functionality to grow for quite some time.