A survey of table reasoning with large language models

Abstract

Table reasoning aims to generate inference results based on the user requirement and the provided table. Enhancing the table reasoning capability of the model can aid in obtaining information efficiently. Recent advancements have positioned Large Language Models (LLMs) as the predominant method for table reasoning, primarily due to their substantial reduction in annotation costs and superior performance compared to previous methods. However, existing research still lacks a summary of LLM-based table reasoning works. Therefore, questions about which techniques can improve table reasoning performance in the era of LLMs and how to enhance table reasoning abilities in the future, remain largely unexplored. This gap significantly limits progress in research. To answer the above questions and advance table reasoning research with LLMs, we present this survey to analyze existing research, inspiring future work. In this paper, we analyze the mainstream techniques used to improve table reasoning performance in the LLM era. Also, we provide research directions from the improvement of existing methods to inspire future research.

Graphical abstract

Keywords

table reasoning / large language models / survey

References

[1] Jin N, Siebert J, Li D, Chen Q. A survey on table question answering: recent advances. In: Proceedings of the 7th China Conference on Knowledge Graph and Semantic Computing: Knowledge Graph Empowers the Digital Economy. 2022, 174–186

[2] Pasupat P, Liang P. Compositional semantic parsing on semi-structured tables. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015, 1470–1480

[3] Chen W, Wang H, Chen J, Zhang Y, Wang H, Li S, Zhou X, Wang W Y. TabFact: a large-scale dataset for table-based fact verification. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[4] Zhao W X, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z, Du Y, Yang C, Chen Y, Chen Z, Jiang J, Ren R, Li Y, Tang X, Liu Z, Liu P, Nie J Y, Wen J R. A survey of large language models. 2023, arXiv preprint arXiv: 2303.18223

[5] Zhang T, Yue X, Li Y, Sun H. TableLlama: towards open large generalist models for tables. In: Proceedings of 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024, 6024−6044

[6] Zhong R, Snell C, Klein D, Eisner J. Non-programmers can label programs indirectly via active examples: a case study with text-to-SQL. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 5126−5152

[7] Sun R, Arik S, Sinha R, Nakhost H, Dai H, Yin P, Pfister T. SQLPrompt: in-context text-to-SQL with minimal labeled data. In: Proceedings of the Association for Computational Linguistics: EMNLP 2023. 2023, 542−550

[8] Ni A, Iyer S, Radev D, Stoyanov V, Yih W T, Wang S, Lin X V. LEVER: learning to verify language-to-code generation with execution. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 26106−26128

[9] Chang S, Fosler-Lussier E. Selective demonstrations for cross-domain text-to-SQL. In: Proceedings of the Association for Computational Linguistics: EMNLP 2023. 2023, 14174−14189

[10] Gao D, Wang H, Li Y, Sun X, Qian Y, Ding B, Zhou J . Text-to-SQL empowered by large language models: a benchmark evaluation. Proceedings of the VLDB Endowment, 2024, 17( 5): 1132–1145

[11] Ye Y, Hui B, Yang M, Li B, Huang F, Li Y. Large language models are versatile decomposers: decomposing evidence and questions for table-based reasoning. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023, 174−184

[12] Cheng Z, Xie T, Shi P, Li C, Nadkarni R, Hu Y, Xiong C, Radev D, Ostendorf M, Zettlemoyer L, Smith N A, Yu T. Binding language models in symbolic languages. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[13] Saha S, Yu X, Bansal M, Pasunuru R, Celikyilmaz A. MURMUR: modular multi-step reasoning for semi-structured data-to-text generation. In: Proceedings of the Association for Computational Linguistics: ACL 2023. 2023, 11069−11090

[14] Wang Z, Zhang H, Li C L, Eisenschlos J M, Perot V, Wang Z, Miculicich L, Fujii Y, Shang J, Lee C Y, Pfister T. Chain-of-table: evolving tables in the reasoning chain for table understanding. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[15] Xia R, Zhang B, Ye H, Yan X, Liu Q, Zhou H, Chen Z, Dou M, Shi B, Yan J, Qiao Y. ChartX & ChartVLM: a versatile benchmark and foundation model for complicated chart reasoning. 2024, arXiv preprint arXiv: 2402.12185

[16] Sun Y, Guo Z, Yu H, Liu C, Li X, Wang B, Yu X, Zhao T. QDA-SQL: questions enhanced dialogue augmentation for multi-turn text-to-SQL. 2024, arXiv preprint arXiv: 2406.10593

[17] Hong Z, Yuan Z, Chen H, Zhang Q, Huang F, Huang X. Knowledge-to-SQL: enhancing SQL generation with data expert LLM. In: Proceedings of the Association for Computational Linguistics ACL 2024. 2024, 10997−11008

[18] Aly R, Guo Z, Schlichtkrull M S, Thorne J, Vlachos A, Christodoulopoulos C, Cocarascu O, Mittal A. The fact extraction and VERification over unstructured and structured information (FEVEROUS) shared task. In: Proceedings of the 4th Workshop on Fact Extraction and VERification (FEVER). 2021, 1−13

[19] Wuehrl A, Menchaca Resendiz Y, Grimminger L, Klinger R. What makes medical claims (UN)verifiable? Analyzing entity and relation properties for fact verification. In: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024, 2046−2058

[20] Guo Z, Schlichtkrull M, Vlachos A . A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 2022, 10: 178–206

[21] Nan L, Hsieh C, Mao Z, Lin X V, Verma N, Zhang R, Kryściński W, Schoelkopf H, Kong R, Tang X, Mutuma M, Rosand B, Trindade I, Bandaru R, Cunningham J, Xiong C, Radev D, Radev D . FeTaQA: free-form table question answering. Transactions of the Association for Computational Linguistics, 2022, 10: 35–49

[22] Yu T, Zhang R, Yang K, Yasunaga M, Wang D, Li Z, Ma J, Li I, Yao Q, Roman S, Zhang Z, Radev D. Spider: a large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In: Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. 2018, 3911−3921

[23] Lu P, Qiu L, Chang K W, Wu Y N, Zhu S C, Rajpurohit T, Clark P, Kalyan A. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[24] Cheng Z, Dong H, Wang Z, Jia R, Guo J, Gao Y, Han S, Lou J G, Zhang D. HiTab: a hierarchical table dataset for question answering and natural language generation. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022, 1094−1110

[25] Katsis Y, Chemmengath S, Kumar V, Bharadwaj S, Canim M, Glass M, Gliozzo A, Pan F, Sen J, Sankaranarayanan K, Chakrabarti S. AIT-QA: question answering dataset over complex tables in the airline industry. In: Proceedings of 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track. 2022, 305−314

[26] Lu X, Pan L, Liu Q, Nakov P, Kan M Y. SCITAB: a challenging benchmark for compositional reasoning and claim verification on scientific tables. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 7787−7813

[27] Gupta V, Mehta M, Nokhiz P, Srikumar V. INFOTABS: inference on tables as semi-structured data. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 2309−2324

[28] Parikh A, Wang X, Gehrmann S, Faruqui M, Dhingra B, Yang D, Das D. ToTTo: a controlled table-to-text generation dataset. In: Proceedings of 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020, 1173−1186

[29] Zhao Y, Qi Z, Nan L, Mi B, Liu Y, Zou W, Han S, Chen R, Tang X, Xu Y, Radev D, Cohan A. QTSumm: query-focused summarization over tabular data. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 1157−1172

[30] Moosavi N, Rücklé A, Roth D, Gurevych I. SciGen: a dataset for reasoning-aware text generation from scientific tables. In: Proceedings of the 1st Neural Information Processing Systems Track on Datasets and Benchmarks. 2021

[31] Chen W, Chen J, Su Y, Chen Z, Wang W Y. Logical natural language generation from open-domain tables. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 7929−7942

[32] Chen M, Wiseman S, Gimpel K. WikiTableT: a large-scale data-to-text dataset for generating Wikipedia article sections. In: Proceedings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021, 193−209

[33] Zhong V, Xiong C, Socher R. Seq2SQL: generating structured queries from natural language using reinforcement learning. 2017, arXiv preprint arXiv: 1709.00103

[34] Yu T, Zhang R, Er H, Li S, Xue E, Pang B, Lin X V, Tan Y C, Shi T, Li Z, Jiang Y, Yasunaga M, Shim S, Chen T, Fabbri A, Li Z, Chen L, Zhang Y, Dixit S, Zhang V, Xiong C, Socher R, Lasecki W, Radev D. CoSQL: a conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases. In: Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019, 1962−1979

[35] Shi T, Zhao C, Boyd-Graber J, Daumé III H, Lee L. On the potential of Lexico-logical alignments for semantic parsing to SQL queries. In: Proceedings of the Association for Computational Linguistics: EMNLP 2020. 2020, 1849−1864

[36] Lee C H, Polozov O, Richardson M. KaggleDBQA: realistic evaluation of text-to-SQL parsers. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021, 2261−2273

[37] Li J, Hui B, Qu G, Yang J, Li B, Li B, Wang B, Qin B, Geng R, Huo N, Zhou X, Ma C, Li G, Chang K C C, Huang F, Cheng R, Li Y. Can LLM already serve as a database interface? A big bench for large-scale database grounded text-to-SQLs. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2024, 1835

[38] Zhao B, Ji C, Zhang Y, He W, Wang Y, Wang Q, Feng R, Zhang X. Large language models are complex table parsers. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 14786−14802

[39] Sui Y, Zhou M, Zhou M, Han S, Zhang D. Table meets LLM: can large language models understand structured table data? A benchmark and empirical study. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. 2024, 645−654

[40] Singha A, Cambronero J, Gulwani S, Le V, Parnin C. Tabular representation, noisy operators, and impacts on table structure understanding tasks in LLMs. In: Proceedings of NeurIPS 2023 Second Table Representation Learning Workshop. 2023

[41] Li H, Zhang J, Li C, Chen H. RESDSQL: decoupling schema linking and skeleton parsing for text-to-SQL. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. 2023, 13067−13075

[42] Pourreza M, Rafiei D. DIN-SQL: decomposed in-context learning of text-to-SQL with self-correction. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023, 1577

[43] Wang D, Dou L, Zhang X, Zhu Q, Che W. DAC: decomposed automation correction for text-to-SQL. 2024, arXiv preprint arXiv: 2408.08779

[44] Zha L, Zhou J, Li L, Wang R, Huang Q, Yang S, Yuan J, Su C, Li X, Su A, Zhang T, Zhou C, Shou K, Wang M, Zhu W, Lu G, Ye C, Ye Y, Ye W, Zhang Y, Deng X, Xu J, Wang H, Chen G, Zhao J. TableGPT: towards unifying tables, nature language and commands into one GPT. 2023, arXiv preprint arXiv: 2307.08674

[45] Yang B, Tang C, Zhao K, Xiao C, Lin C. Effective distillation of table-based reasoning ability from LLMs. In: Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 2024, 5538−5550

[46] Bian J, Qin X, Zou W, Huang M, Zhang W. HELLaMA: LLaMA-based table to text generation by highlighting the important evidence. 2023, arXiv preprint arXiv: 2311.08896

[47] Wu Z, Feng Y. ProTrix: building models for planning and reasoning over tables with sentence context. 2024, arXiv preprint arXiv: 2403.02177

[48] Zhang X, Zhang J, Ma Z, Li Y, Zhang B, Li G, Yao Z, Xu K, Zhou J, Zhang-Li D, Yu J, Zhao S, Li J, Tang J. TableLLM: enabling tabular data manipulation by LLMs in real office usage scenarios. 2024, arXiv preprint arXiv: 2403.19318

[49] Xie T, Wu C H, Shi P, Zhong R, Scholak T, Yasunaga M, Wu C S, Zhong M, Yin P, Wang S I, Zhong V, Wang B, Li C, Boyle C, Ni A, Yao Z, Radev D, Xiong C, Kong L, Zhang R, Smith N A, Zettlemoyer L, Yu T. UnifiedSKG: unifying and multi-tasking structured knowledge grounding with text-to-text language models. In: Proceedings of 2022 Conference on Empirical Methods in Natural Language Processing. 2022, 602−631

[50] Liu T, Wang F, Chen M. Rethinking tabular data understanding with large language models. In: Proceedings of 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024, 450−482

[51] Li Z, Xie T. Using LLM to select the right SQL query from candidates. 2024, arXiv preprint arXiv: 2401.02115

[52] Gan Y, Chen X, Huang Q, Purver M, Woodward J R, Xie J, Huang P. Towards robustness of text-to-SQL models against synonym substitution. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021, 2505−2515

[53] Chen W. Large language models are few(1)-shot table reasoners. In: Proceedings of the Association for Computational Linguistics: EACL 2023. 2023, 1120−1130

[54] Nan L, Zhao Y, Zou W, Ri N, Tae J, Zhang E, Cohan A, Radev D. Enhancing text-to-SQL capabilities of large language models: a study on prompt design strategies. In: Proceedings of the Association for Computational Linguistics: EMNLP 2023. 2023, 14935−14956

[55] Sui Y, Zou J, Zhou M, He X, Du L, Han S, Zhang D. TAP4LLM: table provider on sampling, augmenting, and packing semi-structured data for large language model reasoning. 2023, arXiv preprint arXiv: 2312.09039

[56] Zhang H, Cao R, Chen L, Xu H, Yu K. ACT-SQL: in-context learning for text-to-SQL with automatically-generated chain-of-thought. In: Proceedings of the Association for Computational Linguistics: EMNLP 2023. 2023, 3501−3532

[57] Min D, Hu N, Jin R, Lin N, Chen J, Chen Y, Li Y, Qi G, Li Y, Li N, Wang Q. Exploring the impact of table-to-text methods on augmenting LLM-based question answering with domain hybrid data. In: Proceedings of 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track). 2024, 464−482

[58] Lei F, Luo T, Yang P, Liu W, Liu H, Lei J, Huang Y, Wei Y, He S, Zhao J, Liu K. TableQAKit: a comprehensive and practical toolkit for table-based question answering. 2023, arXiv preprint arXiv: 2310.15075

[59] Kothyari M, Dhingra D, Sarawagi S, Chakrabarti S. CRUSH4SQL: collective retrieval using schema hallucination for Text2SQL. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 14054−14066

[60] Kong K, Zhang J, Shen Z, Srinivasan B, Lei C, Faloutsos C, Rangwala H, Karypis G. OpenTab: advancing large language models as open-domain table reasoners. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[61] Xue S, Jiang C, Shi W, Cheng F, Chen K, Yang H, Zhang Z, He J, Zhang H, Wei G, Zhao W, Zhou F, Qi D, Yi H, Liu S, Chen F. DB-GPT: empowering database interactions with private large language models. 2024, arXiv preprint arXiv: 2312.17449

[62] Wang T, Lin H, Han X, Sun L, Chen X, Wang H, Zeng Z. DBCopilot: scaling natural language querying to massive databases. 2023, arXiv preprint arXiv: 2312.03463

[63] Wang B, Ren C, Yang J, Liang X, Bai J, Zhang Q W, Yan Z, Li Z. MAC-SQL: multi-agent collaboration for text-to-SQL. 2023, arXiv preprint arXiv: 2312.11242

[64] Shi W, Xu R, Zhuang Y, Yu Y, Zhang J, Wu H, Zhu Y, Ho J, Yang C, Wang M D. EHRAgent: code empowers large language models for few-shot complex tabular reasoning on electronic health records. 2024, arXiv preprint arXiv: 2401.07128

[65] Han S, Yoon J, Arik S O, Pfister T. Large language models can automatically engineer features for few-shot tabular learning. In: Proceedings of the 41st International Conference on Machine Learning. 2024

[66] Cao Y, Chen S, Liu R, Wang Z, Fried D. API-assisted code generation for question answering on varied table structures. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 14536−14548

[67] Jiang J, Zhou K, Dong Z, Ye K, Zhao X, Wen J R. StructGPT: a general framework for large language model to reason over structured data. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 9237−9251

[68] Nan L, Zhang E, Zou W, Zhao Y, Zhou W, Cohan A. On evaluating the integration of reasoning and action in LLM agents with database question answering. In: Proceedings of the Association for Computational Linguistics: NAACL 2024. 2024, 4556−4579

[69] Zhang Y, Henkel J, Floratou A, Cahoon J, Deep S, Patel J M . ReAcTable: enhancing ReAct for table question answering. Proceedings of the VLDB Endowment, 2024, 17( 8): 1981–1994

[70] Dou L, Gao Y, Pan M, Wang D, Che W, Lou J G, Zhan D . UNISAR: a unified structure-aware autoregressive language model for text-to-SQL semantic parsing. International Journal of Machine Learning and Cybernetics, 2023, 14( 12): 4361–4376

[71] Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, Chi E H, Le Q V, Zhou D. Chain-of-thought prompting elicits reasoning in large language models. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 1800

[72] Cui L, Xu Y, Lv T, Wei F. Document AI: Benchmarks, models and applications. 2021, arXiv preprint arXiv: 2111.08609

[73] Zhang L, Hu A, Xu H, Yan M, Xu Y, Jin Q, Zhang J, Huang F. TinyChart: efficient chart understanding with visual token merging and program-of-thoughts learning. 2024, arXiv preprint arXiv: 2404.16635

[74] Wei J, Xu N, Chang G, Luo Y, Yu B, Guo R. mChartQA: a universal benchmark for multimodal chart question answer based on vision-language alignment and reasoning. 2024, arXiv preprint arXiv: 2404.01548

[75] Zheng M, Feng X, Si Q, She Q, Lin Z, Jiang W, Wang W. Multimodal table understanding. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024, 9102−9124

[76] Ni J, Young T, Pandelea V, Xue F, Cambria E . Recent advances in deep learning based dialogue systems: a systematic survey. Artificial Intelligence Review, 2023, 56( 4): 3055–3155

[77] Yu T, Zhang R, Yasunaga M, Tan Y C, Lin X V, Li S, Er H, Li I, Pang B, Chen T, Ji E, Dixit S, Proctor D, Shim S, Kraft J, Zhang V, Xiong C, Socher R, Radev D. SParC: cross-domain semantic parsing in context. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 4511−4523

[78] Gao Y, Xiong Y, Gao X, Jia K, Pan J, Bi Y, Dai Y, Sun J, Guo Q, Wang M, Wang H. Retrieval-augmented generation for large language models: a survey. 2024, arXiv preprint arXiv: 2312.10997

[79] Dou L, Gao Y, Liu X, Pan M, Wang D, Che W, Zhan D, Kan M Y, Lou J G. Towards knowledge-intensive text-to-SQL semantic parsing with formulaic knowledge. In: Proceedings of 2022 Conference on Empirical Methods in Natural Language Processing. 2022, 5240−5253

[80] Jiang Z, Mao Y, He P, Neubig G, Chen W. OmniTab: pretraining with natural and synthetic data for few-shot table-based question answering. In: Proceedings of 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022, 932−942

[81] Zhao G, Yang P. Table-based fact verification with self-labeled keypoint alignment. In: Proceedings of the 29th International Conference on Computational Linguistics. 2022, 1401−1411

[82] Yin P, Neubig G, Yih W T, Riedel S. TaBERT: pretraining for joint understanding of textual and tabular data. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020, 8413−8426

[83] Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, Lee Y T, Li Y, Lundberg S, Nori H, Palangi H, Ribeiro M T, Zhang Y. Sparks of artificial general intelligence: early experiments with GPT-4. 2023, arXiv preprint arXiv: 2303.12712

[84] Liu Q, Yang D, Zhang J, Guo J, Zhou B, Lou J G. Awakening latent grounding from pretrained language models for semantic parsing. In: Proceedings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021, 1174−1189

[85] Yu W, Iter D, Wang S, Xu Y, Ju M, Sanyal S, Zhu C, Zeng M, Jiang M. Generate rather than retrieve: large language models are strong context generators. In: Proceedings of the Eleventh International Conference on Learning Representations. 2023

[86] Longpre S, Hou L, Vu T, Webson A, Chung H W, Tay Y, Zhou D, Le Q V, Zoph B, Wei J, Roberts A. The flan collection: designing data and methods for effective instruction tuning. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 22631−22648

[87] Liu W, Zeng W, He K, Jiang Y, He J. What makes good data for alignment? A comprehensive study of automatic data selection in instruction tuning. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[88] Bukharin A, Zhao T. Data diversity matters for robust instruction tuning. 2024, arXiv preprint arXiv: 2311.14736

[89] Liu Q, Zheng X, Muennighoff N, Zeng G, Dou L, Pang T, Jiang J, Lin M. RegMix: data mixture as regression for language model pre-training. 2024, arXiv preprint arXiv: 2407.01492

[90] Xu C, Sun Q, Zheng K, Geng X, Zhao P, Feng J, Tao C, Jiang D. WizardLM: empowering large language models to follow complex instructions. 2023, arXiv preprint arXiv: 2304.12244

[91] Luo Z, Xu C, Zhao P, Sun Q, Geng X, Hu W, Tao C, Ma J, Lin Q, Jiang D. WizardCoder: empowering code large language models with evol-instruct. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[92] Chang T Y, Jia R. Data curation alone can stabilize in-context learning. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023, 8123−8144

[93] Tonglet J, Reusens M, Borchert P, Baesens B. SEER: a knapsack approach to exemplar selection for in-context HybridQA. In: Proceedings of 2023 Conference on Empirical Methods in Natural Language Processing. 2023, 13569−13583

[94] Wang D, Dou L, Zhang X, Zhu Q, Che W. Improving demonstration diversity by human-free fusing for text-to-SQL. 2024, arXiv preprint arXiv: 2402.10663

[95] Xie Y, Kawaguchi K, Zhao Y, Zhao X, Kan M Y, He J, Xie M Q. Self-evaluation guided beam search for reasoning. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023, 1802

[96] Yang C, Wang X, Lu Y, Liu H, Le Q V, Zhou D, Chen X. Large language models as optimizers. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[97] Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K R, Cao Y. ReAct: synergizing reasoning and acting in language models. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[98] Cai T, Wang X, Ma T, Chen X, Zhou D. Large language models as tool makers. In: Proceedings of the 12th International Conference on Learning Representations. 2024

[99] Yao S, Yu D, Zhao J, Shafran I, Griffiths T L, Cao Y, Narasimhan K R. Tree of thoughts: deliberate problem solving with large language models. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023, 517

RIGHTS & PERMISSIONS

Higher Education Press