Spectral livecoding with loops

SpectralOli can manipulate local audio & freesound audio loops by treating the spectrogram as a canvas. It exposes sound design as a livecoding DSL, primarily interacting with audio loops in more ways than one. For example: We can chop the audio in the time domain, filter it in the frequency domain, and introduce a novel workflow that treats the spectrogram as a 2D canvas that can be manipulated as a matrix where we can apply several transformations across X and Y axes.

Expressions are written per-track and chained left-to-right. Evaluate code instantly using Cmd+Enter (macOS) or Ctrl+Enter (Windows/Linux).

1. Loop Slicing & Sequencing

Chop audio in the time domain using automatic onset detection or uniform grid slicing, then reorder, stutter, and sculpt slices rhythmically using the seq() sequencer.

Automatic Slicing Commands

These prefix commands chop the buffer and populate the slice editor automatically. Place them before seq().

Command	Behavior & Analysis
`slicep n`	Percussive onset detection based on high-frequency energy transients. `n` must be a power of 2 between `256` and `8192`.
`slicem n`	Melodic onset detection using spectral flux to detect pitch and harmonic transitions.
`slicee n`	Equal-width slicing. Divides the audio buffer into `n` chunks of uniform length.

Sequencing & Slice Specs

seq(spec?) arranges slices into a rhythmic pattern. A spec is a string containing slice indices or ranges (start:stop). Negative numbers index from the end.

Spec Example	Selection Result
`seq()`	Play all existing slices sequentially (`0..N-1`).
`seq("0, 2, 4")`	Play slice 0, then 2, then 4 in order.
`seq("0:4")`	Play slices 0, 1, 2, and 3 (stop index is exclusive).
`seq("-4:")`	Play the last 4 slices of the buffer.
`seq(":4, 6:")`	Concatenate slice ranges (slices 0–3 followed by slice 6 onwards).

Pattern Transformations

Target specific steps by position using .at(spec, op, prob?). Chain transformations directly onto sequence steps:

Operation	Description
`stutter(n)`	Repeat each slice `n` times.
`repeat(n)`	Repeat the entire block `n` times.
`fast(multiplier)`	Chained onto a targeted `seq()` step (e.g. `.at("4", fast(2))`) — speeds up that step's playback by a positive factor (e.g., `2` plays at double tempo).
`slow(divisor)`	Chained onto a targeted `seq()` step — slows down that step's playback by a positive factor (e.g., `2` plays at half tempo).
`reverse()`	Play the targeted step's audio window backwards without altering sequence order.
`shuffle()`	Randomly permute targeted steps while untargeted steps stay fixed in place.
`silence()`	Mute targeted step output, acting as a clean musical rest.
`euclid(hits, steps, offset)`	Distribute `hits` evenly as possible `steps` (Euclidean rhythm). Optional `offset` rotates the pattern.
`mirror()`	Append a reversed copy of the sequence.

A global clock <multiplier> directive sets the playback speed for the whole track (e.g. clock 0.5 plays at half speed). Write it as its own line, anywhere in the track's code.

// Some chops and transformations on a 512-sample percussive loop
slicep 512
seq()
  .at("4:6", reverse())
  .at("6:10", fast(1.2))
  .at("14", stutter(2))
  .at("30:33", shuffle())
  .at("33:", slow(1.5))

Periodic Operations (every)

You can apply transformations periodically using the every(n, operation) method. It runs the provided pattern transformation only once every n loop cycles, leaving the sequence unaffected during the other cycles.

// Play the first 8 slices. Every 4 cycles, reverse the entire pattern
slicep 512
seq(":8").every(4, reverse())

2. Frequency Operations

Filter sound in the frequency domain by specifying STFT resolution, carving frequency passbands, and dynamically combining regions using boolean algebra and mathematical formulas.

Analysis Window: fft n sets the STFT frame size (power of two 256–8192, default 1024). Written as its own bare directive line, anywhere in the track's code. Smaller FFT sizes provide crisp temporal resolution (ideal for drums); larger sizes provide precise frequency resolution (ideal for harmonic chords).

Filtering Regions

Method	Behavior
`low(hz)` / `high(hz)`	Pass frequencies below (low-pass) or above (high-pass) `hz`.
`band(min, max)`	Pass frequencies between `min` and `max` Hz.
`harmonic(f0, count, width)`	Pass fundamental frequency `f0` and its `count` integer harmonics.

Combining Regions with Mask Algebra

A track has at most one frequency mask, written as a single infix expression combining regions with real arithmetic-like operators — no method chaining:

Operator	Behavior
`a + b`	Union (saturating): combine two regions, e.g. `band(200, 4000) + high(8000)`.
`a - b`	Subtraction (clamped): carve/notch `b` out of `a`, e.g. `band(200, 4000) - band(800, 1000)`.
`!a`	Complement: flip the region (passbands become stopbands), e.g. `!low(1000)`. Use parens to negate a whole sub-expression: `!(a + b)`.

gain <expr> is a separate global directive (not part of the mask expression) that scales the overall output amplitude. Its value can be a dynamic per-bin expression.

Math & Playback Time Modulation

Arguments support JavaScript arithmetic (+, -, *, /, %), Math functions (Math.sin, Math.cos), and the dynamic variable time (current playback time in seconds):

// LFO-modulated bandpass filter
fft 2048
gain 1.5
band(400 + Math.sin(time * 2) * 200, 2000) + high(8000) - band(800, 1000)

// Sweeping low-pass filter inverted into a sweeping notch
!low(time * 1000 % 10000)

3. Spectral Operations

Transform spectral energy distribution across time and frequency frames without altering transport speed, and treat the rolling spectrogram history as a 2D canvas matrix where normalized time is the X axis (0..1) and normalized frequency is the Y axis (0..1). Apply affine matrix warps, spectral blurring, granulation, or custom coordinate transformations.

Spectral Blur & Granulation

Method	Description
`.blur(time_amt, freq_amt)`	2D spectral smoothing. `time_amt` (0.5) controls frame decay across time; `freq_amt` (0.5) smears energy across adjacent frequency bins.
`sgranulate(scatter, mix)`	Spectral granulation. Scatters and recombines spectral frames from past rolling buffer history. `scatter` 0 stutters rhythmically, 1 floats randomly.

2D Canvas & Affine Matrix Transformations

Method	Sonic Equivalent & Matrix Behavior
`.scale(scaleX, scaleY, mix)`	Zoom / Stretch: `scaleX` stretches time; `scaleY` stretches bin spacing (inharmonicity, metallic bell timbres).
`.rotate(degrees, mix)`	Rotation: Spins the matrix around its center point, shearing time into frequency and frequency into time.
`.skew(skewX, skewY, mix)`	Shear: Slants the matrix into a parallelogram. `skewX` tilts frequencies over time (spectral delay / dispersion); `skewY` shifts pitch continuously over time (glissando / riser / tape-stop).
`.transpose(mix)`	Reflection: Swaps time and frequency axes across the diagonal (`x' = y`, `y' = x`). A long evolving pad becomes a frequency sweep; sub-bass energy becomes an immediate transient burst.

// Heavy time decay with subtle frequency smear on a bandpass filter (mask, then pipeline)
band(100, 5000)
blur(0.85, 0.2)

// Modulated spectral granulator where scatter density shifts over time
sgranulate(Math.sin(time * 0.5) * 0.5 + 0.5, 0.9)

// Inharmonic frequency stretch with 2x time zoom
band(100, 6000)
scale(2, 1.5, 1)

// Rhythmic matrix rotation modulated by playback time
rotate(time * 45 % 360, 0.8)

// Spectral delay dispersion (X skew) and continuous pitch riser (Y skew)
band(200, 8000)
skew(0.5, 0.2, 1)

// Transpose: reflect matrix across diagonal (swap time and frequency dimensions)
transpose(1)