by GSAI-ML
Open source · 416k downloads · 354 likes
LLaDA 8B Instruct is an advanced language model with 8 billion parameters, fully trained from scratch, designed to rival models like LLaMA3 8B in terms of performance. Specialized in text comprehension and generation, it excels in instruction-based tasks, delivering precise and contextually relevant responses to a variety of requests. Its core capabilities include generating coherent text, solving complex problems, and adapting to specific instructions, making it particularly useful for applications requiring natural interaction with users. The model stands out for its innovative approach, combining diffusion techniques with an optimized architecture for high performance. It is ideal for use cases such as conversational assistance, automating text-based tasks, or linguistic data analysis.
We introduce LLaDA, a diffusion model with an unprecedented 8B scale, trained entirely from scratch, rivaling LLaMA3 8B in performance.
[2025-10-21] We have modified modeling_llada.py to support the input of attention_mask.