Audio

A project has two sources of sound. Audio plays a file. onFrame schedules Web Audio synthesis at a frame. Both are mixed into one stereo track at 48 kHz.

Play a fileCopy Anchor Link

import { Audio, Sequence, staticFile } from 'remotion';

<Sequence from={30} durationInFrames={60}>
  <Audio src={staticFile('vo/line-1.wav')} volume={0.85} />
</Sequence>

src is a file path. Use staticFile for a file in public/. The props are Remotion's:

A clip starts at the frame where its Sequence starts. Inside a <Freeze>, an Audio clip is silent.

A volume that rises over the first second:

<Audio
  src={staticFile('tone.wav')}
  volume={(frame) => interpolate(frame, [0, 30], [0, 1], { extrapolateRight: 'clamp' })}
/>

Video plays the sound of its file too. Its volume, muted and toneFrequency props work the same way.

Synthesise soundCopy Anchor Link

onFrame schedules a function to run at a frame. The function gets a Web Audio context and the time to play at:

import { onFrame } from '@kinetographdev/audio';

onFrame(4, (ctx, t) => {
  const osc = ctx.createOscillator();
  const gain = ctx.createGain();
  osc.frequency.setValueAtTime(1800, t);
  osc.frequency.exponentialRampToValueAtTime(900, t + 0.05);
  gain.gain.setValueAtTime(0.25, t);
  gain.gain.exponentialRampToValueAtTime(0.001, t + 0.06);
  osc.connect(gain).connect(ctx.destination);
  osc.start(t);
  osc.stop(t + 0.07);
});

A few rules apply:

• Call onFrame at the top level of the module, not inside a component. Each call adds one callback for one frame.
• t is in seconds from the start of the rendered range. Schedule against t, not ctx.currentTime.
• The context is an OfflineAudioContext from node-web-audio-api. It runs on the CPU.
• For noise, use random from remotion with a named seed. The same seed gives the same sound every render:

import { random } from 'remotion';

for (let i = 0; i < data.length; i++) data[i] = random(`whoosh-${i}`) * 2 - 1;

Mix and renderCopy Anchor Link

render mixes every Audio clip and every onFrame callback in the frames it draws. --range mixes only those frames.

The output codec follows the container:

• .mp4 and .mov use AAC. .mov also takes PCM.
• .webm uses Opus.
• .gif has no sound.

Set the codec and bit rate with --audio-codec and --audio-bitrate. The default is 96 kbit/s per channel. See the CLI reference.

Check the loudnessCopy Anchor Link

npx kinetograph audio analyze examples/rajz-cards/index.tsx --scenes 0..60,60..125

The command mixes the audio and prints:

• the integrated loudness in LUFS, measured to ITU-R BS.1770-4,
• the true peak in dBTP,
• the number of clipped samples,
• the silences of 0.5 seconds or more below -60 dBFS.

--range limits the frames that are mixed. --scenes measures frame ranges on their own, as well as the whole range. Add --json for the same numbers as an object, with the keys integratedLufs, truePeakDb, clippedSamples, silences and scenes.

Code APICopy Anchor Link

@kinetographdev/audio exports onFrame, clearOnFrame, renderAudio and analyzeAudio, which the CLI uses. The signatures are in AUTHORING.md, section 8.

For the gaps in the audio elements, see Known issues.