Interaction System

AudioUI provides a unified interaction system that handles user input across all control components. The system is designed to be performant, accessible, and consistent with professional audio software standards.

Overview

The interaction system supports multiple input methods:

• Drag interactions: Mouse and touch dragging with configurable sensitivity
• Wheel interactions: High-precision scrolling with accumulator support
• Keyboard navigation: Accessible control via arrow keys and specialized handlers
• Pointer events: Multi-touch support for polyphonic interactions (Keys component)

Each control type uses specialized interaction logic optimized for its use case:

• Continuous controls (Knob, Slider): Drag, wheel, and keyboard
• Boolean controls (Button): Click, drag-in/drag-out, keyboard
• Discrete controls (CycleButton): Click, keyboard stepping/cycling
• Note-based controls (Keys): Multi-touch, glissando detection

Architecture

The interaction system consists of four main hooks, each wrapping a framework-agnostic controller:

useContinuousInteraction

For continuous controls (Knob, Slider). Handles:

• Pointer dragging: Mouse and touch with adaptive sensitivity
• Mouse wheel: High-precision scrolling with configurable sensitivity
• Keyboard navigation: Arrow keys, Home/End, Space/Enter
• Double-click reset: Reset to default value
• Focus management: Consistent focus behavior

useBooleanInteraction

For boolean controls (Button). Provides:

• Momentary mode: Press to activate, release to deactivate
• Toggle mode: Click to toggle state
• Drag-in/drag-out: Hardware-like behavior with global pointer tracking
• Keyboard support: Enter/Space for activation

useDiscreteInteraction

For discrete/enum controls (CycleButton). Provides:

• Cycling: Click or Space/Enter to cycle to next value
• Stepping: Arrow keys to step up/down through options
• Value resolution: Automatically finds nearest valid option
• Keyboard support: Arrow keys for stepping, Space/Enter for cycling

useNoteInteraction

For note-based controls (Keys). Provides:

• Multi-touch support: Tracks multiple concurrent pointers
• Glissando detection: Automatic note off/on when sliding across keys
• Pointer capture: Continues receiving events outside element
• Touch action: Prevents default touch behaviors

Interaction Modes

Drag Interactions

Drag interactions allow users to click/touch and drag to adjust values.

Direction

The drag direction determines how pointer movement maps to value changes:

• Circular (Knobs): Rotational drag around center
  • Clockwise = Increase
  • Counter-clockwise = Decrease
• Vertical (Vertical Sliders): Linear vertical drag
  • Up = Increase
  • Down = Decrease
• Horizontal (Horizontal Sliders): Linear horizontal drag
  • Right = Increase
  • Left = Decrease
• Both: Bidirectional drag (both axes affect value)

Copyimport { Knob, Slider } from "@cutoff/audio-ui-react";
 
export default function DragDirectionExample() {
  return (
    <div style={{ display: "flex", gap: "20px" }}>
      {/* Circular drag (default for knobs) */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Circular"
        interactionDirection="circular"
      />
      {/* Vertical drag */}
      <Slider
        value={0.5}
        onChange={(e) => {}}
        label="Vertical"
        orientation="vertical"
        interactionDirection="vertical"
      />
      {/* Horizontal drag */}
      <Slider
        value={0.5}
        onChange={(e) => {}}
        label="Horizontal"
        orientation="horizontal"
        interactionDirection="horizontal"
      />
    </div>
  );
}

Sensitivity

Sensitivity controls how much the value changes per pixel of movement. Default is 0.005 (approximately 200px throw for full range).

Copyimport { Knob } from "@cutoff/audio-ui-react";
 
export default function SensitivityExample() {
  return (
    <div style={{ display: "flex", gap: "20px" }}>
      {/* Low sensitivity - fine control */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Fine"
        interactionSensitivity={0.001}
      />
      {/* Default sensitivity */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Normal"
        interactionSensitivity={0.005}
      />
      {/* High sensitivity - coarse control */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Coarse"
        interactionSensitivity={0.02}
      />
    </div>
  );
}

Adaptive Sensitivity

For low-resolution parameters (where step size is large), sensitivity is automatically adjusted to ensure a single step change requires no more than 30 pixels of movement. This prevents "dead zones" in discrete-like continuous controls.

The adaptive calculation uses:

effectiveSensitivity = Math.max(baseSensitivity, stepSize / 30)

Drag Accumulator

For stepped parameters, small drag movements accumulate until they exceed the step size. This prevents unresponsive behavior when dragging slowly on low-resolution parameters.

Wheel Interactions

Wheel interactions provide high-precision scrolling for fine adjustments.

Behavior

• Direction: Standard scrolling (Up/Push = Increase, Down/Pull = Decrease)
• Page scroll: Strictly prevented - wheel events stop propagation to avoid scrolling the page
• Sensitivity: Configurable, defaults to 0.005
• Decoupled: Wheel sensitivity is independent from drag sensitivity

Wheel Accumulator

For stepped parameters, wheel deltas are accumulated until they exceed the step size. This ensures reliable operation across different hardware (trackpads with small deltas vs. mice with large clicks) and prevents landing "between" steps.

Copyimport { Knob } from "@cutoff/audio-ui-react";
 
export default function WheelInteractionExample() {
  return (
    <div style={{ display: "flex", gap: "20px" }}>
      {/* Wheel only */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Wheel Only"
        interactionMode="wheel"
      />
      {/* Both drag and wheel */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Drag & Wheel"
        interactionMode="both"
      />
    </div>
  );
}

Keyboard Interactions

Keyboard interactions provide accessible control and fine-grained adjustments.

Focus Management

Controls are focusable via Tab navigation. Focused controls display a high-contrast highlight effect (brightness/contrast boost + shadow) instead of a browser-default focus ring.

Keyboard Shortcuts

Continuous Controls (Knob, Slider):

• Arrow Up / Arrow Right: Increment value
• Arrow Down / Arrow Left: Decrement value
• Home: Set to minimum
• End: Set to maximum
• Space / Enter: Specialized handlers (component-specific)

Discrete Controls (CycleButton):

• Arrow Up / Arrow Right: Step to next value (clamped at max)
• Arrow Down / Arrow Left: Step to previous value (clamped at min)
• Space / Enter: Cycle to next value (wraps around)

Boolean Controls (Button):

• Space / Enter: Toggle/Activate

Copyimport { Knob, CycleButton } from "@cutoff/audio-ui-react";
 
export default function KeyboardInteractionExample() {
  return (
    <div style={{ display: "flex", gap: "20px" }}>
      {/* Focus and use arrow keys */}
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Focus & Use Arrows"
      />
      {/* CycleButton with keyboard stepping */}
      <CycleButton
        value="option1"
        options={[
          { value: "option1", label: "Option 1" },
          { value: "option2", label: "Option 2" },
          { value: "option3", label: "Option 3" },
        ]}
        onChange={(e) => {}}
        label="Use Arrows to Step"
      />
    </div>
  );
}

Pointer Events (Keys)

The Keys component uses pointer events for multi-touch and glissando support.

Multi-Touch

Supports multiple concurrent touches for polyphonic playing. Each touch is tracked independently.

Glissando Detection

Sliding across keys automatically triggers:

• Note off for the previous key
• Note on for the new key

This enables smooth glissando effects when sliding across the keyboard.

Pointer Capture

Uses pointer capture to continue receiving events even when the pointer leaves the element, ensuring reliable glissando detection across the entire keyboard.

Interaction Mode Configuration

interactionMode

Controls which input methods are enabled:

• "drag": Only drag interactions enabled
• "wheel": Only wheel interactions enabled
• "both": Both drag and wheel enabled (default)

Copyimport { Knob } from "@cutoff/audio-ui-react";
 
export default function InteractionModeExample() {
  return (
    <div style={{ display: "flex", gap: "20px" }}>
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Drag Only"
        interactionMode="drag"
      />
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Wheel Only"
        interactionMode="wheel"
      />
      <Knob
        value={0.5}
        onChange={(e) => {}}
        label="Both"
        interactionMode="both"
      />
    </div>
  );
}

Component-Specific Behavior

Knob & Slider (Continuous)

• Drag: Standard continuous adjustment with circular (knobs) or linear (sliders) direction
• Wheel: Fine adjustments with accumulator support for stepped parameters
• Keyboard: Step-based increments with configurable step size
• Double-click: Reset to default value (when resetToDefault is provided)

Button (Boolean)

• Click: Toggle or momentary activation
• Drag-in/drag-out: Hardware-like behavior where buttons respond to pointer entering/leaving while pressed
• Global pointer tracking: Tracks pointer state globally to enable drag-in behavior from anywhere on the page
• Step sequencer pattern: Enables classic step sequencer interactions where multiple buttons can be activated with a single drag gesture

Copyimport { Button } from "@cutoff/audio-ui-react";
 
export default function ButtonDragInExample() {
  return (
    <div style={{ display: "flex", gap: "10px" }}>
      {/* Drag across buttons to activate them */}
      {[0, 1, 2, 3, 4, 5, 6, 7].map((step) => (
        <Button
          key={step}
          value={false}
          onChange={(e) => {}}
          label={`Step ${step + 1}`}
          latch={true}
        />
      ))}
    </div>
  );
}

CycleButton (Discrete)

• Click: Cycles to next value (wraps around to start)
• Keyboard stepping: Arrow keys step through options (clamped at min/max)
• Keyboard cycling: Space/Enter cycles to next value (wraps around)
• No drag: Discrete-only control - does not support continuous interaction

Keys (Note-Based)

• Multi-touch: Multiple concurrent touches for polyphonic playing
• Glissando: Sliding across keys automatically triggers note off/on
• Pointer capture: Continues receiving events outside element
• Touch action: Prevents default touch behaviors (scrolling, zooming)

Sensitivity Tuning

Drag Sensitivity

Controls how much the value changes per pixel of movement:

• Default: 0.005 (approximately 200px throw for full range)
• Fine control: Lower values (e.g., 0.001)
• Coarse control: Higher values (e.g., 0.02)

Wheel Sensitivity

Separate from drag sensitivity to allow independent tuning:

• Default: 0.005
• Decoupled: Prevents "too fast" scrolling on low-resolution parameters that use boosted drag sensitivity

Adaptive Behavior

For stepped parameters, sensitivity is automatically adjusted to ensure responsiveness:

effectiveSensitivity = Math.max(baseSensitivity, stepSize / 30)

This ensures that a single step change requires no more than 30 pixels of movement.

Accessibility

ARIA Attributes

Components automatically receive appropriate ARIA attributes:

• Continuous controls: role="slider", aria-valuenow, aria-valuemin, aria-valuemax, aria-label
• Boolean controls: role="button", aria-pressed, aria-label
• Discrete controls: role="button", aria-label

Focus Management

• Tab navigation: All controls are focusable
• Visual feedback: High-contrast highlight effect on focus (replaces default outline)
• Keyboard shortcuts: Full keyboard support for all interaction types

Text Selection Prevention

Text selection is automatically disabled (user-select: none) during drag operations to prevent accidental selection.

Cursor Behavior

The interaction system provides intelligent cursor feedback based on the control's interaction capabilities and state.

Continuous Controls

Cursor depends on interaction configuration:

• Disabled: not-allowed
• Wheel only: ns-resize (vertical)
• Horizontal drag: ew-resize (horizontal)
• Vertical drag: ns-resize (vertical)
• Bidirectional drag: move (bidirectional)
• Circular drag: Custom circular cursor
• Clickable only: pointer

Boolean & Discrete Controls

• Interactive: pointer
• Non-interactive: Default browser cursor

Cursor Customization

All cursor values are customizable via CSS variables:

• --audioui-cursor-clickable: Clickable controls (default: pointer)
• --audioui-cursor-bidirectional: Bidirectional drag (default: SVG data URI)
• --audioui-cursor-horizontal: Horizontal drag (default: SVG data URI)
• --audioui-cursor-vertical: Vertical drag (default: SVG data URI)
• --audioui-cursor-circular: Circular drag (custom circular cursor)
• --audioui-cursor-noneditable: Non-editable controls (default: default)
• --audioui-cursor-disabled: Disabled controls (default: not-allowed)

Note: The library uses SVG data URIs for bidirectional, horizontal, and vertical cursors to ensure consistent behavior across browsers.

Performance Considerations

Lazy Listeners

Global mousemove/touchmove listeners are only attached during an active drag session and removed immediately after.

Refs for State

Mutable state (drag start position, current value) is tracked in useRef to avoid stale closures and unnecessary re-renders during high-frequency events.

Native Events

Wheel handling uses native non-passive listeners (via AdaptiveBox) to reliably prevent page scrolling, which React's synthetic events cannot always guarantee.

Accumulators

For stepped parameters, drag and wheel accumulators prevent unnecessary value updates and ensure smooth interaction even with slow movements.

Integration with Control Primitives

The interaction system is fully integrated with Control Primitives:

• ContinuousControl: Uses useContinuousInteraction
• DiscreteControl: Uses useDiscreteInteraction
• BooleanControl: Uses useBooleanInteraction

When building custom controls, the interaction system is automatically available through the primitives.

Next Steps

• Learn about Audio Parameters for parameter-based value management
• Explore Control Primitives for building custom controls with full interaction support
• Check out component-specific documentation for interaction examples