par brain-bzh
Open source · 30k downloads · 22 likes
REVE-base est un modèle de langage spécialisé dans le traitement des signaux EEG, basé sur une architecture de transformateur. Entraîné sur plus de 60 000 heures de données EEG issues de 92 ensembles de données et 25 000 sujets, il se distingue par sa capacité à s'adapter à différentes configurations d'électrodes et à une grande variété de tâches liées à l'EEG. Son encodage positionnel 4D innovant lui permet de traiter des signaux de longueur et d'arrangement variables, offrant ainsi une flexibilité sans précédent. Ce modèle est particulièrement utile pour des applications médicales, neuroscientifiques ou liées à l'interface cerveau-machine, où l'analyse de données EEG est cruciale. Développé par la BRAIN team et l'Université de Montréal, REVE-base se positionne comme un outil puissant pour l'extraction de représentations EEG polyvalentes et généralisables.
REVE (project page here) is a transformer-based foundation model for EEG signal processing. It was trained on 60k hours of EEG data from various sources and is designed to be adaptable to any electrode configuration and a wide range of EEG-based tasks.
REVE (Representation for EEG with Versatile Embeddings), a pretrained encoder explicitly designed to generalize across diverse EEG signals. REVE introduces a novel 4D positional encoding scheme that enables it to process signals of arbitrary length and electrode arrangement. Using a masked autoencoding objective, we pretrain REVE on over 60,000 hours of EEG data from 92 datasets spanning 25,000 subjects.
Developed by the BRAIN team and UdeM
Funded by: This research was supported by the French National Research Agency (ANR) through its AI@IMT program and grant ANR-24-CE23-7365, as well as by a grant from the Brittany region. Further support was provided by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (NSERC), by funding from the Canada Research Chairs program and the Fonds de recherche du Québec – Nature et technologies (FRQ-NT). This work was granted access to the HPC resources of IDRIS under the allocation 2024-AD011015237R1 made by GENCI, as well as HPC provided by Digital Alliance Canada.
Example script to extract embeddings with REVE, using our position bank:
from transformers import AutoModel
pos_bank = AutoModel.from_pretrained("brain-bzh/reve-positions", trust_remote_code=True)
model = AutoModel.from_pretrained("brain-bzh/reve-base", trust_remote_code=True)
eeg_data = ... # EEG data as a torch Tensor (batch_size, channels, time_points), must be sampled at 200 Hz
electrode_names = [...] # List of electrode names corresponding to the channels in eeg_data
positions = pos_bank(electrode_names) # Get positions (channels, 3)
# Expand the positions vector to match the batch size
positions = positions.expand(eeg_data.size(0), -1, -1) # (batch_size, channels, 3)
output = model(eeg_data, positions)