Plantasia Sound Engine

[Independent · 2026]

A generative sound engine shipped as a TypeScript library. Four botanical species, five shared ecological controls, a thirty four method public API, a modulation matrix that reaches every control from an LFO or a mod wheel, snapshots that carry the whole state as one JSON object and morph between two, and timed events plus per frame audio analysis that let visuals follow the music. Hosts own the UI; the engine owns the sound.

The Plantasia Sound Engine stage with the Desert preset playing: a green waveform over a dark field, level meters below

Product brief

The Plantasia Sound Engine makes sound that lives. A traditional preset captures one moment: oscillator, filter, effect levels. A Sound World describes how a sound is born, blooms, decays, and moves, and the engine plays it as a generative organism rather than recalling a snapshot.

It is a library for instrument apps. A host initializes it on a user gesture, loads a species or a preset, and drives it with five ecological controls, notes, or MIDI. Every musical event comes back out on a typed bus stamped with audio clock time, and a per frame analysis of the master bus gives the host bands, centroid, and onsets, so visuals can follow the sound without their own FFT.

It is part of the Plantasonic ecosystem and stands on its own. Each Sound World carries its own visual identity, a theme, palette, motion style, intensity, and tempo that host apps render however they like, and the control surface ships its own waveform and level visualizers. No other engine is involved.

The solution

01  /  05 1

Four species, one ecology

Seed (birth: intimate, damp, the first breath), Flowers (bloom: chorus rich harmonic opening, Juno inspired), Mold (decay: tape wear, spectral drift, erosion), Bacteria (microscopic motion: particle swarms, cellular jitter). Each is a plugin with its own synthesis graph, registered in a species registry. All four answer the same five controls: growth, bloom, roots, mold, bacteria. The knobs describe ecological forces; each species interprets them through its own architecture.

The control surface: waveform stage, transport buttons, meters, live event chips, and the Sound Worlds section open with Seed as the active species
02  /  05 2

Generative composition

A shared generative engine handles phrases, harmony, rhythm, probability, and memory, so a species does not have to compose. Ecology shapes it: growth raises density and voice activity, bacteria raises mutation and event frequency, roots grounds the low end. A central scheduler and transport replace ad hoc timers, so every species keeps the same clock and MIDI clock can drive it.

The Generative Engine section with Canopy loaded: phrase evolution, mutation, and event frequency mapped to ecology
03  /  05 3

Expressive performance

Notes arrive from a host call, the computer keyboard, or Web MIDI, and pass through a velocity engine and an expression router that map velocity, density, and macros to synth targets per species. Twelve performance macros route differently for each species, so one gesture means one thing in Seed and another in Mold. Mold itself is a living degradation macro that ages any world over time.

The Performance Macros section with Roots loaded, above the live waveform
04  /  05 4

A facade and a lifecycle

Hosts import one function, createPlantasiaEngine, and get the public tier: thirty four methods on one documentation page, folded so the seven a player feels come first: init, loadSpecies, start, noteOn and noteOff, setControl, modulate, applySnapshot. Six lines from an empty file to the first sound, and a build gate keeps it at six. Under it the engine enforces a state machine, so calling noteOn before start throws a clear error instead of failing silently. A played instrument starts with generative off; presets resolve to species through an adapter; the v1 preset API stays on the root export for older hosts.

The Layers section with Mycelium loaded: grains to bacteria, impulses to growth, ticks to roots, each with a level slider
05  /  05 5

Events for visuals

Every musical moment is a semantic event: speciesChanged, controlChanged, notePlayed, noteReleased, generatorEvent, densityChanged, and onset, each stamped with the audio clock time it happened at. A host subscribes and drives whatever it draws, and polls getAudioFeatures for bass, mid, high, centroid, and onset strength each frame. Since 1.1 the same loop can read the modulation state: what every LFO, envelope follower, and MIDI source is doing, and where each control sits between its base and its modulated value. Alongside the events, every preset ships a visual contract: theme key, color palette, motion style, intensity, and animation tempo, validated at build like the audio fields. The engine does not render; it tells the host what the sound looks like and when it moves.

The Timbre section with Rainforest loaded: filter cutoff, resonance, attack, and release beside a dense waveform

Sound that is born,
blooms, decays,
and moves.

By the numbers

The stage in a square crop: a green waveform from the Bloom preset over a dark field

4

live species: Seed, Flowers, Mold, Bacteria

5

ecological controls shared by every species

19

validation gates on every build, plus the browser harness in CI

12 ms

from noteOn to audible, measured in Chromium and WebKit

My role

I designed and built the engine: the Sound World concept and the four archetypes, the species plugin architecture and registry, the ecological control layer, the generative and performance engines, the lifecycle contract, the event bus, the master bus and audio analysis, the scheduler and MIDI scaffold, the host facade and Plantasonic adapter, the demo control surface, the browser performance harness and its CI job, the modulation engine and its species hook, the snapshot and morph layer, the lab page for tuning by ear, and the eighteen validation gates and documentation that hold it together.

Role

Designer, Engineer, Sound Designer

Timeframe

2026

Disciplines

Sound Design, Systems Design, Library API Design

Current state

Version 1.2.1, released September 2026. 1.2.0 came three days after 1.0.0 froze the public tier and 1.1.0 added modulation; 1.2.1 is the simplicity release: no engine change, but the surface folded so what a host reads first is the seven methods a player feels, the demo opening on those seven with save and recall, and six lines to the first sound measured on every build. 1.2 is hardening and state: getSnapshot and applySnapshot carry the whole host facing state (species, controls, tempo, routes, preferences, polyphony cap) as one JSON object, with a timed morph between two that crosses a species switch; a polyphony cap for phones; MIDI input select; the Playwright harness running in GitHub Actions in Chromium and WebKit; a bundle size gate; Tone.js pinned; and the v1 preset path deprecated for removal at 2.0. Thirty four methods, eleven presets, four species, nineteen validation gates. Measured: zero dropouts through a five second morph across a species switch, a switch ready in 50 to 120 ms, 48 control extreme loads without a throw. A local lab page for tuning by ear found a Flowers range error on its first A/B and measured that half the controls are inaudible on one held note, which is the input to the next release, a sound pass. The live control surface runs at sound-engine.xyz.

Stack

TypeScript, Web Audio, Tone.js, Web MIDI, Vite

Surface

Public tier (plantasia-sound-engine/public): engine, presets, adapter, events, analysis. Root export adds the v1 preset API and internals

Platform

Any browser with Web Audio; audio unlocks on a user gesture

Problems & decisions

Give Plantasonic a sound layer that behaves like a living thing, that any instrument app can mount in a few lines, and that tells the visual layer what it is doing.

Plantasonic instruments needed generative, botanical sound with a consistent feel across apps. A preset library could not carry that: presets recall moments, and each app would end up re-implementing how sound evolves, ages, and answers performance. The engine had to own that behavior once.

Two users again. The app author needs a safe, small API that cannot be called in the wrong order and that reports what the sound is doing. The player needs a handful of controls that feel like forces, not parameters: turn up growth and the world gets busier, turn up mold and it decays, whichever species is playing.

The v1 engine was preset centric. v2 reorganized everything around species: a plugin with its own synthesis graph that interprets the five shared ecological controls its own way. The preset system stayed, but every preset now resolves to a species plus an ecology. The tradeoff: two audio paths coexist during the transition (a Tone.js chain for v1 and per species graphs for v2), and the demo's analyser only listens to one of them, which is the first fix on the engine's list.

The architecture landed under a premature v2.0.0 tag before the host contract was real. Rather than build on it, I deprecated the tag, listed the integration blockers (lifecycle contract, playable only registry, unified facade, event bus, scheduler, MIDI, migration guide, browser sonic test, CPU budget test, adapter and integration doc), shipped all of them as 1.0.0 beta 1, and told hosts to pin that. The demo went through a validation pass that removed every control not wired to a real engine call. 1.0.0 followed once the surface was measured, not just built.

Before calling the public API stable I wrote a Playwright harness that plays the built engine in Chromium and WebKit and measures what a player feels: the time from noteOn to the first audible sample, located to the sample on a tap of the master bus, and dropouts over a minute of generative playback while a fake visual loop burns eight milliseconds of every frame. The first run measured 102 ms of latency: Tone.js schedules a tenth of a second ahead by default. Setting the lookahead to ten milliseconds brought it to 12 ms with zero dropouts in both browsers. The harness also showed that on a single held note the five ecological controls move the spectrum less than the voice moves itself; they act on the population, which is the design, and now it is written down rather than assumed. In 1.2 the harness moved into CI and grew rows for control extremes, a morph across a species switch, and per control audibility; a lab page built for tuning by ear found, on its first A/B, that Flowers threw a range error above ninety percent bloom, a bug no gate had reached because no gate had pushed a control to its edge.

Layers

  • Public tier: createPlantasiaEngine and twenty three methods on it, createPlantasonicAdapter, the shipped presets. Root export carries the v1 preset path and internals; nothing removed.
  • Lifecycle state machine: idle, loaded, running, disposed; wrong order throws.
  • Species manager and registry; ecology controls normalized and clamped at the engine layer.
  • Generative engine (phrases, harmony, rhythm, probability, memory) shared by all species.
  • Performance engine: velocity engine, expression router, twelve macros.
  • Scheduler and transport; Web MIDI input scaffold.
  • Semantic event bus for hosts, every payload stamped with audio clock time.
  • Master bus: every output path, v1 chain, signature graphs, and all species, terminates in one node with the analyser, FFT, and meter behind getAudioFeatures.
  • Modulation engine (1.1): global, beside the species manager; descriptor sources, additive routes, a 30 Hz frame handed to the species through an optional hook, change gated so eight routes cost a fifth of a millisecond per tick.
  • Snapshots (1.2): one versioned JSON object for species, controls, tempo, routes, preferences, and polyphony cap; applySnapshot validates before it changes anything, replaces routes, switches species, and morphs controls and tempo at 30 Hz over a chosen time.

Validation gates on every build

  • Presets, species API, ecology controls, generative engine, performance engine, species registry, lifecycle, facade, events, analysis, modulation, preferences, snapshot, scheduler, MIDI, species audio readiness, performance budget, v2 release check.

Browser performance harness

  • Playwright in Chromium and WebKit, locally and in GitHub Actions: noteOn to audible located to the sample on a master bus tap, dropouts over sixty seconds of generative Seed with and without eight modulation routes under a mock visual loop burning eight milliseconds a frame, control extremes on every species, a five second morph across a species switch, control audibility per species, bundle size against a budget. Dropouts, extremes, the morph and the size budget block; the rest is recorded per release.

Shared controls with species specific mappings is the idea that made the engine feel like one instrument instead of four. The honest beta was the right call; the premature major tag cost more trust than the extra week would have. The master analyser that every path feeds should have come first; when it landed, the stage lit up for the whole engine, and the harness built on it found that the default scheduling lookahead put a hundred milliseconds between noteOn and sound. Ten lines fixed it; the point is that it was measured before the surface was called stable. Eight placeholder species were cut rather than kept as promises. Modulation followed in 1.1, designed the same way: a grill first, a decisions table, then the code, then the harness rows before the tag. 1.2 held to it and added a rule: build the measuring tool before the work it measures. The lab page came first so the sound pass could start whenever ears were free, and it paid for itself the day it was built. The grill after that asked what the engine is in one sentence, and the answer, an instrument that plays itself and answers you, reordered the docs and the demo without touching a line of engine code: simpler is what you read first, not what exists.