par facebook
Open source · 183k downloads · 200 likes
BART Large est un modèle de langage avancé basé sur une architecture encodeur-décodeur, combinant les forces d'un encodeur bidirectionnel (comme BERT) et d'un décodeur autorégressif (comme GPT). Pré-entraîné sur de vastes corpus en anglais, il excelle particulièrement dans les tâches de génération de texte, telles que la summarisation, la traduction automatique ou la reformulation, grâce à sa capacité à reconstruire des textes à partir de versions corrompues. Il s'avère également performant pour des tâches de compréhension comme la classification de texte ou les questions-réponses après un fine-tuning adapté. Ce modèle se distingue par sa polyvalence, permettant aussi bien des usages génériques comme le remplissage de texte que des applications spécialisées via des versions affinées sur des jeux de données spécifiques. Son approche innovante de pré-entraînement par corruption et reconstruction en fait un outil puissant pour traiter une grande variété de défis en traitement automatique du langage.
BART model pre-trained on English language. It was introduced in the paper BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension by Lewis et al. and first released in this repository.
Disclaimer: The team releasing BART did not write a model card for this model so this model card has been written by the Hugging Face team.
BART is a transformer encoder-decoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. BART is pre-trained by (1) corrupting text with an arbitrary noising function, and (2) learning a model to reconstruct the original text.
BART is particularly effective when fine-tuned for text generation (e.g. summarization, translation) but also works well for comprehension tasks (e.g. text classification, question answering).
You can use the raw model for text infilling. However, the model is mostly meant to be fine-tuned on a supervised dataset. See the model hub to look for fine-tuned versions on a task that interests you.
Here is how to use this model in PyTorch:
from transformers import BartTokenizer, BartModel
tokenizer = BartTokenizer.from_pretrained('facebook/bart-large')
model = BartModel.from_pretrained('facebook/bart-large')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
@article{DBLP:journals/corr/abs-1910-13461,
author = {Mike Lewis and
Yinhan Liu and
Naman Goyal and
Marjan Ghazvininejad and
Abdelrahman Mohamed and
Omer Levy and
Veselin Stoyanov and
Luke Zettlemoyer},
title = {{BART:} Denoising Sequence-to-Sequence Pre-training for Natural Language
Generation, Translation, and Comprehension},
journal = {CoRR},
volume = {abs/1910.13461},
year = {2019},
url = {http://arxiv.org/abs/1910.13461},
eprinttype = {arXiv},
eprint = {1910.13461},
timestamp = {Thu, 31 Oct 2019 14:02:26 +0100},
biburl = {https://dblp.org/rec/journals/corr/abs-1910-13461.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}