by google
Open source · 116k downloads · 11 likes
The *Reformer Crime and Punishment* model is a language model trained on Dostoevsky’s *Crime and Punishment*, specializing in generating literary text. It employs an optimized architecture designed to process long texts while maintaining strong narrative and stylistic coherence. The model excels at producing prose inspired by the author’s style, with particular attention to psychological subtleties and introspective dialogue. It is especially well-suited for crafting narratives, dialogues, or stylistic analyses that emulate Dostoevsky’s universe. What sets it apart is its ability to generate lengthy, structured texts while remaining faithful to the spirit of the original work.
Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated into English.
Crime and Punishment training data was taken from gs://trax-ml/reformer/crime-and-punishment-2554.txt and contains
roughly 0.5M tokens.
The ReformerLM model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb and the weights were converted to Hugging Face's PyTorch ReformerLM model ReformerModelWithLMHead.
The model is a language model that operates on small sub-word units. Text can be generated as follows:
model = ReformerModelWithLMHead.from_pretrained("google/reformer-crime-and-punishment")
tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
tok.decode(model.generate(tok.encode("A few months later", return_tensors="pt"), do_sample=True,temperature=0.7, max_length=100)[0])
# gives:'A few months later on was more than anything in the flat.
# “I have already.” “That’s not my notion that he had forgotten him.
# What does that matter? And why do you mean? It’s only another fellow,” he said as he went out, as though he want'