Why not transform chat large language models to non-English?

Graphical abstract

References

[1] Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C L, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J, Hilton J, Kelton F, Miller L, Simens M, Askell A, Welinder P, Christiano P, Leike J, Lowe R. Training language models to follow instructions with human feedback. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 2011

[2] Grattafiori A, Dubey A, Jauhri A, Pandey A, Kadian A, , . The Llama 3 herd of models. 2024, arXiv preprint arXiv: 2407.21783

[3] Zhang Z, Lee D H, Fang Y, Yu W, Jia M, Jiang M, Barbieri F. PLUG: leveraging pivot language in cross-lingual instruction tuning. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024, 7025−7046

[4] Hu E J, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W. LoRA: low-rank adaptation of large language models. In: Proceedings of the 10th International Conference on Learning Representations. 2022

[5] Zhu W, Lv Y, Dong Q, Yuan F, Xu J, Huang S, Kong L, Chen J, Li L. Extrapolating large language models to non-english by aligning languages. 2023, arXiv preprint arXiv: 2308.04948

[6] Pipatanakul K, Jirabovonvisut P, Manakul P, Sripaisarnmongkol S, Patomwong R, Chokchainant P, Tharnpipitchai K. Typhoon: Thai large language models. 2023, arXiv preprint arXiv: 2312.13951

[7] Üstün A, Aryabumi V, Yong Z, Ko W Y, D’souza D, Onilude G, Bhandari N, Singh S, Ooi H L, Kayid A, Vargus F, Blunsom P, Longpre S, Muennighoff N, Fadaee M, Kreutzer J, Hooker S. Aya model: an instruction finetuned open-access multilingual language model. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024, 15894–15939

[8] OpenAI. GPT-4 technical report. 2023, arXiv preprint arXiv: 2303.08774

[9] Zheng L, Chiang W L, Sheng Y, Zhuang S, Wu Z, Zhuang Y, Lin Z, Li Z, Li D, Xing E P, Zhang H, Gonzalez J E, Stoica I. Judging LLM-as-a-judge with MT-bench and Chatbot arena. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023, 2020

RIGHTS & PERMISSIONS

Higher Education Press