Sound from motion

Drop the tile below, then make it bigger and drop it again. The small one ticks high and quick. The big one lands lower and rings a little longer. Nobody picked those sounds from a library: the size of the tile decided them.

When a sound feels wrong next to an animation, it is almost always because the sound and the picture disagree. A tiny chip lands with a heavy thud. A card on the right sounds from the middle. A whoosh peaks where nothing is moving. The fix is not a better sound pack. Let the motion decide the sound.

Sound and motion are one event

Apple calls this harmony: things should feel the way they look and sound the way they move. In the physical world, physics gives you that for free. In software, you build it by hand.

The rule that produces it: the transient (the sharp start of the sound) sits on the contact frame, the frame where the visual lands. The card stops, the toggle seats, the logo appears. Not the frame where the motion begins.

Late is forgiven, early is not

Broadcast engineers measured how far apart sound and picture can drift before people notice. The numbers are lopsided:

We forgive sound that comes a little after the picture, because that is how the world works: light travels faster than sound. Sound that comes before the picture reads as broken almost at once.

So: never let the sound lead. Aim for 0 to 30 ms late, about one frame.

Try all three. "Early" sounds wrong even though it is only 90 ms off. "Late" at 200 ms becomes two events: a landing and then an echo.

Each kind of sound has its own sync point

• Clicks, keys, pops, impacts put their attack on the contact frame.
• A whoosh puts its peak on the fastest frame of the move, starting 0.1 to 0.45 s before.
• A riser ends on the hit, or just before it to leave a breath of air.
• A slow entrance gets two sounds: the travel early, then a small landing on the settle.

Finding the contact frame in code

For a strong ease-out, the motion has visibly stopped long before the math says it has. A good rule is about 85% of the way through the duration. A 480 ms landing makes contact around 408 ms.

Fire the sound from the same event that starts the change, timed to the landing:

// Confirming a landing: play when
// the transform transition ends.
card.addEventListener("transitionend",
  (e) => {
    if (e.propertyName === "transform") {
      play("land");
    }
  },
  { once: true });
 
// A tick that goes with a toggle
// fires with the change itself.
toggle.addEventListener("change",
  () => play("tick"));

Firing on a spring's animationend lands the sound late by its whole wobbly tail. And every sound file must start at its first audible sample, or the file's own silence makes you late.

The mapping: what the motion tells the sound

Once timing is right, let the motion set the rest.

The tile at the top uses the first row directly: pitch is 5200 × (60 / size)^0.75 Hz, and decay grows with size.

Fewer sounds than tweens

Deriving is not the same as sonifying everything. A scene with forty moving parts does not get forty sounds. It gets the few the eye actually follows. Two things landing on the same frame are one sound: voice the bigger one.

For films, write the sound as data first: a cue sheet, one row per visual event with its time, size and position. Lock the picture before you place anything, because a moved frame moves every cue after it.

Wire the "land" sound to the card's enter animation so it confirms the landing.
Find the contact frame: for this ease-out, about 85% of the duration (or the transform's transitionend). The sound's attack goes there, 0-30 ms late at most, never early.
For springs, use the first crossing of the rest position, not animationend.
Trim leading silence from the file to zero.
Derive pitch from size: small element = higher and shorter. One sound per event the eye tracks, not one per tween.