Large language models: Technology, intelligence, and thought

Abstract

Large language models (LLMs) represent a novel technological species in the realm of general intelligence. Their problem-solving approach is not based on “first principles” (logocentrism) but rather on empirical learning from observed data. LLMs possess the ability to extract intuitive knowledge from vast amounts of data, enabling them to offer flexible and effective solutions in the face of complex and dynamic scenarios. The general intelligence characteristics of LLMs are mainly reflected in three aspects: technologically, they exhibit common sense, deep reasoning, strong generalization, and natural human-computer interaction; in terms of intelligence, they demonstrate memory-driven core features, powerful data-driven learning capabilities, and exceptional generalization abilities; in terms of thought, they possess highly human-like cognitive traits such as contextual understanding, analogy, and intuitive reasoning. These capabilities collectively suggest that LLMs can adapt to a wide range of complex, open-ended scenarios, presenting a stark contrast to traditional models that emphasize formal logic, quantitative analysis, and narrowly defined problem structures. As such, the rise of LLMs is likely to drive significant shifts in AI theory and application, potentially redefining how intelligent systems approach decision-making, strategic reasoning, and contextual understanding in uncertain and dynamic environments.

Graphical abstract

Keywords

large language models / technological characteristics / intelligence features / thinking patterns / general intelligence

References

[1] BenderE MKoller A (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data In: Proceedings of the 58th annual meeting of the association for computational linguistics, 5185–5198

[2] Bengio Y, Courville A, Vincent P, (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35( 8): 1798–1828

[3] Brown T, Mann B, Ryder N, Subbiah M, Kaplan J D, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33: 1877–1901

[4] Browning J, LeCun Y, (2023). Language, common sense, and the Winograd schema challenge. Artificial Intelligence, 325: 104031

[5] BubeckSEldan RGehrkeJHorvitzEKamarE LeePLeeY T LiYLundbergS (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712

[6] Burton J W, Lopez-Lopez E, Hechtlinger S, Rahwan Z, Aeschbach S, Bakker M A, Becker J A, Berditchevskaia A, Berger J, Brinkmann L, Flek L, Herzog S M, Huang S, Kapoor S, Narayanan A, Nussberger A M, Yasseri T, Nickl P, Almaatouq A, Hahn U, Kurvers R H J M, Leavy S, Rahwan I, Siddarth D, Siu A, Woolley A W, Wulff D U, Hertwig R, (2024). How large language models can reshape collective intelligence. Nature Human Behaviour, 8( 9): 1643–1655

[7] ChomskyNRoberts IWatumullJ (2023). Noam chomsky: The false promise of ChatGPT. The New York Times, March 8. Available at the website of nytimes.com

[8] ChristianoP FLeikeJBrownT MarticMLegg SAmodeiD (2017). Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30

[9] DevlinJChang MLeeKToutanovaK (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1: 4171–4186

[10] DongQLi LDaiDZhengCMaJ LiRXiaH XuJWuZ ChangB (2024). A survey on in-context learning In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 1107–1128

[11] Duan Y, Edwards J S, Dwivedi Y K, (2019). Artificial intelligence for decision making in the era of Big Data—Evolution, challenges and research agenda. International Journal of Information Management, 48: 63–71

[12] Fodor J A, Pylyshyn Z W, (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28( 1–2): 3–71

[13] Gui J, Chen T, Zhang J, Cao Q, Sun Z, Luo H, Tao D, (2024). A survey on self-supervised learning: Algorithms, applications, and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46( 12): 9052–9071

[14] HendrycksDBurns CBasartSZouAMazeikaM SongDSteinhardt J (2020). Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

[15] Hinton G E, Osindero S, Teh Y W, (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18( 7): 1527–1554

[16] Hinton G E, Salakhutdinov R R, (2006). Reducing the dimensionality of data with neural networks. Science, 313( 5786): 504–507

[17] KalliniJPapadimitriou IFutrellRMahowaldKPottsC (2024). Mission: Impossible Language Models. In: Proceedings of the 62nd Annual Meeting of the Association for Compartational Linguistics. Bangkok, Thailand. 14691–14714

[18] KaplanJMcCandlish SHenighanTBrownT BChessB ChildRGray SRadfordAWuJAmodeiD (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

[19] LeCun Y, Bengio Y, Hinton G, (2015). Deep learning. Nature, 521( 7553): 436–444

[20] LeCunYMisra I (2021). Self-supervised learning: The dark matter of intelligence. Meta, March 4. Available at the website of ai.meta.com

[21] Lehr S A, Caliskan A, Liyanage S, Banaji M R, (2024). Chatgpt as research scientist: Probing GPT’s capabilities as a research librarian, research ethicist, data generator, and data predictor. Proceedings of the National Academy of Sciences of the United States of America, 121( 35): e2404328121

[22] Lu Y, Adrados J S, Chand S S, Wang L, (2021). Humans are not machines—Anthropocentric human–machine Symbiosis for ultra-flexible smart. Engineering, 7( 6): 734–737

[23] MarcusG F (2003). The Algebraic Mind: Integrating Connectionism and Cognitive Science. MIT Press

[24] Meng J, (2024). AI emerges as the frontier in behavioral science. Proceedings of the National Academy of Sciences of the United States of America, 121( 10): e2401336121

[25] PetroniFRocktäschel TLewisPBakhtinAWuY MilleA HRiedel S (2019). Language models as knowledge bases? arXiv preprint arXiv:1909.01066

[26] Ren M, Chen N, Qiu H, (2023). Human–machine collaborative decision-making: An evolutionary roadmap based on cognitive intelligence. International Journal of Social Robotics, 15( 7): 1101–1114

[27] Romera-Paredes B, Barekatain M, Novikov A, Balog M, Kumar M P, Dupont E, Ruiz F J, Ellenberg J S, Wang P, Fawzi O, Kohli P, Fawzi A, (2024). Mathematical discoveries from program search with large language models. Nature, 625( 7995): 468–475

[28] Singhal K, Azizi S, Tu T, Mahdavi S S, Wei J, Chung H W, Scales N, Tanwani A, Cole-Lewis H, Pfohl S, Payne P, Seneviratne M, Gamble P, Kelly C, Babiker A, Schärli N, Chowdhery A, Mansfield P, Demner-Fushman D, Agüera y Arcas B, Webster D, Corrado G S, Matias Y, Chou K, Gottweis J, Tomasev N, Liu Y, Rajkomar A, Barral J, Semturs C, Karthikesalingam A, Natarajan V, (2023). Large language models encode clinical knowledge. Nature, 620( 7972): 172–180

[29] TamkinABrundage MClarkJGanguliD (2021). Understanding the capabilities, limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503

[30] TuringA M (2009). Computing Machinery and Intelligence. Springer Netherlands, 23–65

[31] VaswaniA (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30

[32] Vemprala S H, Bonatti R, Bucker A, Kapoor A, (2024). ChatGPT for robotics: Design principles and model abilities. IEEE Access: Practical Innovations, Open Solutions, 12: 55682–55696

[33] Wake N, Kanehira A, Sasabuchi K, Takamatsu J, Ikeuchi K, (2024). GPT-4v (ision) for robotics: Multimodal task planning from human demonstration. IEEE Robotics and Automation Letters, 9( 11): 10567–10574

[34] Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le Q V, Zhou D, (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35: 24824–24837

[35] WiggersK (2018). Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality. VentureBeat. Retrieved December 31, 2024. Available at the website of venturebeat.com

[36] XieS MMin S (2022). How does in-context learning work? A framework for understanding the differences from traditional supervised learning. The Stanford AI Lab Blog. Retrieved December 31, 2024. Available at the website of stanford.edu

RIGHTS & PERMISSIONS

Higher Education Press