Xmodel15 The New Multilingual Large Language Model
3 min read
This paper introduces Xmodel-1.5, a new multilingual large language model (LLM) developed by Xiaoduo Technology’s AI Lab. Trained on a massive dataset, this 1-billion parameter model aims to improve cross-lingual understanding and generation, particularly in less-represented languages. The researchers also released a new Thai evaluation dataset to aid future research.
Multilingual Proficiency
Xmodel-1.5 demonstrates strong performance across multiple languages, including less common ones like Thai, Arabic, and French, in addition to English and Chinese. Benchmark comparisons against similar-sized models like OPT, Pythia, and TinyLLaMA show Xmodel-1.5 achieving superior results in various commonsense reasoning tasks. Multilingual evaluations using datasets like XCOPA, PIQA_AR, and Belebele_tha_thai further confirm its cross-lingual capabilities.
Instruction Tuning for Enhanced Performance
The model underwent instruction fine-tuning to improve its performance on instruction-based tasks, particularly within the e-commerce domain for Retrieval-Augmented Generation (RAG). This process involved a progressive curriculum learning strategy, incorporating datasets like Belle, infinity-instruct-subject, and RAG_mixed. Evaluation using ifeval and MT-Bench benchmarks, along with a custom Thai evaluation set, demonstrates the effectiveness of this instruction tuning.
Thai Evaluation Dataset Contribution
A key contribution of this research is the release of a new Thai evaluation dataset, annotated by students at Chulalongkorn University. This dataset provides a valuable resource for assessing the performance of language models in Thai and contributes to the development of more robust multilingual NLP systems.
Performance Evolution and Future Directions
Analysis of the model's performance evolution during pre-training reveals consistent improvement across various multilingual benchmarks. While the results are promising, the researchers acknowledge areas for future improvement, particularly in handling nuances like slang, gender differentiation, and formal/informal tone distinctions.
Conclusion
Xmodel-1.5 offers a significant advancement in multilingual LLMs, exhibiting strong performance across a diverse range of languages and tasks. The accompanying release of a Thai evaluation dataset further strengthens its contribution to the field. While acknowledging areas for future refinement, this work represents a valuable step towards more inclusive and effective cross-lingual communication and understanding.
