by Xenova
Open source · 2k downloads · 51 likes
MusicGen Small is an artificial intelligence model designed to generate music from text descriptions or instructions. It can create original tracks in just a few seconds, with sound quality optimized for web applications thanks to its ONNX-optimized weights. The model stands out for its ease of use and accessibility, particularly through an online demo that allows testing without complex installation. It caters to both content creators and developers looking to integrate music generation into their projects. Its lightweight and efficient approach makes it a versatile tool for musical experimentation or rapid soundtrack production.
https://huggingface.co/facebook/musicgen-small with ONNX weights to be compatible with Transformers.js.
If you haven't already, you can install the Transformers.js JavaScript library from NPM using:
npm install @huggingface/transformers
Example: Generate music with Xenova/musicgen-small.
import { AutoTokenizer, MusicgenForConditionalGeneration, RawAudio } from '@huggingface/transformers';
// Load tokenizer and model
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/musicgen-small');
const model = await MusicgenForConditionalGeneration.from_pretrained('Xenova/musicgen-small', {
dtype: {
text_encoder: 'q8',
decoder_model_merged: 'q8',
encodec_decode: 'fp32',
},
});
// Prepare text input
const prompt = 'a light and cheerly EDM track, with syncopated drums, aery pads, and strong emotions bpm: 130';
const inputs = tokenizer(prompt);
// Generate audio
const audio_values = await model.generate({
...inputs,
max_new_tokens: 500,
do_sample: true,
guidance_scale: 3,
});
// (Optional) Write the output to a WAV file
const audio = new RawAudio(audio_values.data, model.config.audio_encoder.sampling_rate);
audio.save('musicgen.wav');
We also released an online demo, which you can try yourself: https://huggingface.co/spaces/Xenova/musicgen-web
Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).