par xlnet
Open source · 381k downloads · 82 likes
XLNet est un modèle de langage avancé, pré-entraîné sur l'anglais, qui excelle dans la compréhension et la génération de texte en exploitant une approche innovante de modélisation autorégressive généralisée. Grâce à son architecture basée sur Transformer-XL, il capture efficacement les dépendances à long terme, ce qui le rend particulièrement performant pour des tâches complexes comme la réponse à des questions, l'analyse de sentiments ou le classement de documents. Contrairement à d'autres modèles, XLNet combine les avantages des approches autorégressives et auto-encodantes, offrant une flexibilité accrue pour l'adaptation à des tâches variées. Il est principalement conçu pour être affiné sur des applications spécifiques, comme la classification de séquences ou l'extraction d'entités, plutôt que pour la génération de texte libre. Son efficacité et sa polyvalence en font un outil de choix pour les chercheurs et les développeurs travaillant sur des projets de traitement automatique du langage naturel.
XLNet model pre-trained on English language. It was introduced in the paper XLNet: Generalized Autoregressive Pretraining for Language Understanding by Yang et al. and first released in this repository.
Disclaimer: The team releasing XLNet did not write a model card for this model so this model card has been written by the Hugging Face team.
XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective. Additionally, XLNet employs Transformer-XL as the backbone model, exhibiting excellent performance for language tasks involving long context. Overall, XLNet achieves state-of-the-art (SOTA) results on various downstream language tasks including question answering, natural language inference, sentiment analysis, and document ranking.
The model is mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you.
Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at models like GPT2.
Here is how to use this model to get the features of a given text in PyTorch:
from transformers import XLNetTokenizer, XLNetModel
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
model = XLNetModel.from_pretrained('xlnet-base-cased')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
@article{DBLP:journals/corr/abs-1906-08237,
author = {Zhilin Yang and
Zihang Dai and
Yiming Yang and
Jaime G. Carbonell and
Ruslan Salakhutdinov and
Quoc V. Le},
title = {XLNet: Generalized Autoregressive Pretraining for Language Understanding},
journal = {CoRR},
volume = {abs/1906.08237},
year = {2019},
url = {http://arxiv.org/abs/1906.08237},
eprinttype = {arXiv},
eprint = {1906.08237},
timestamp = {Mon, 24 Jun 2019 17:28:45 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-1906-08237.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}