par facebook
Open source · 517k downloads · 204 likes
BART base est un modèle de langage de type encodeur-décodeur spécialisé dans la génération et la compréhension de texte. Il combine un encodeur bidirectionnel, similaire à BERT, et un décodeur autorégressif inspiré de GPT, ce qui lui permet de traiter des tâches variées comme la traduction, le résumé automatique ou la réponse aux questions. Pré-entraîné sur des corpus en anglais, il excelle particulièrement après un fine-tuning sur des jeux de données supervisés, bien qu'il puisse aussi être utilisé directement pour des tâches comme le remplissage de texte. Son approche unique, basée sur la reconstruction de texte corrompu, lui confère une grande polyvalence pour des applications nécessitant à la fois compréhension et génération. Ce modèle se distingue par sa capacité à s'adapter efficacement à différentes tâches grâce à sa structure flexible et son entraînement robuste.
BART model pre-trained on English language. It was introduced in the paper BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension by Lewis et al. and first released in this repository.
Disclaimer: The team releasing BART did not write a model card for this model so this model card has been written by the Hugging Face team.
BART is a transformer encoder-decoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.
BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering).
You can use the raw model for text infilling. However, the model is mostly meant to be fine-tuned on a supervised dataset. See the model hub to look for fine-tuned versions on a task that interests you.
Here is how to use this model in PyTorch:
from transformers import BartTokenizer, BartModel
tokenizer = BartTokenizer.from_pretrained('facebook/bart-base')
model = BartModel.from_pretrained('facebook/bart-base')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
@article{DBLP:journals/corr/abs-1910-13461,
author = {Mike Lewis and
Yinhan Liu and
Naman Goyal and
Marjan Ghazvininejad and
Abdelrahman Mohamed and
Omer Levy and
Veselin Stoyanov and
Luke Zettlemoyer},
title = {{BART:} Denoising Sequence-to-Sequence Pre-training for Natural Language
Generation, Translation, and Comprehension},
journal = {CoRR},
volume = {abs/1910.13461},
year = {2019},
url = {http://arxiv.org/abs/1910.13461},
eprinttype = {arXiv},
eprint = {1910.13461},
timestamp = {Thu, 31 Oct 2019 14:02:26 +0100},
biburl = {https://dblp.org/rec/journals/corr/abs-1910-13461.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}