LLMKB:Large Language Models with Knowledge Base Augmentation for Conversational Recommendation

Abstract

Conversational recommender systems(CRSs) focus on refining preferences and providing personalized recommendations through natural language interactions and dialogue history. Large language models(LLMs) have shown outstanding performance across various domains, thereby prompting researchers to investigate their applicability in recommendation systems. However, due to the lack of task-specific knowledge and an inefficient feature extraction process, LLMs still have suboptimal performance in recommendation tasks. Therefore, external knowledge sources, such as knowledge graphs(KGs) and knowledge bases(KBs), are often introduced to address the issue of data sparsity. Compared to KGs, KBs possess higher retrieval efficiency, making them more suitable for scenarios where LLMs serve as recommenders. To this end, we introduce a novel framework integrating LLMs with KBs for enhanced retrieval generation, namely LLMKB. LLMKB initially leverages structured knowledge to create mapping dictionaries, extracting entity-relation information from heterogeneous knowledge to construct KBs. Then, LLMKB achieves the embedding calibration between user information representations and documents in KBs through retrieval model fine-tuning. Finally, LLMKB employs retrieval-augmented generation to produce recommendations based on fused text inputs, followed by post-processing. Experiment results on two public CRS datasets demonstrate the effectiveness of our framework. Our code is publicly available at the link:https:/ / anonymous. 4open. science/ r/ LLMKB-6FD0.

Keywords

recommender system / large language model(LLM) / knowledge base(KB)

References

[1] SUN Y M, ZHANG Y. Conversational recommender system[C]// The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. New York: ACM, 2018:235-244.

[2] CHEN Q B, LIN J Y, ZHANG Y C, et al. Towards knowledge-based recommender dialog system[EB/OL].(2019-09-03)[2024-08-06]. https://arxiv.org/pdf/1908.05391.

[3] LI R, EBRAHIMI KAHOU S, SCHULZ H, et al. Towards deep conversational recommendations[C]// Conference and Workshop on Neural Information Processing Systems. New York: ACM, 2018:9748-9758.

[4] WANG X L, ZHOU K, WEN J R, et al. Towards unified conversational recommender systems via knowledge-enhanced prompt learning[C]// Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2022:1929-1937.

[5] ZHOU K, ZHAO W X, BIAN S Q, et al. Improving conversational recommender systems via knowledge graph based semantic fusion[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2020:1006-1014.

[6] PAHUNE S, CHANDRASEKHARAN M. Several categories of large language models(LLMs):a short survey[EB/OL].(2023-07-05)[2024-08-06]. https://arxiv.org/pdf/2307.10188.

[7] OpenAI. ChatGPT:optimizing language models for dialogue[EB/OL].(2022-11-30)[2024-08-03]. https://openai.com/blog/chatgpt.

[8] TOUVRON H, MARTIN L, STONE K, et al. LLAMA 2:open foundation and fine-tuned chat models[EB/OL].(2023-07-19)[2024-08-03]. https://arxiv.org/pdf/2307.09288.

[9] DAI S H, SHAO N L, ZHAO H Y, et al. Uncovering ChatGPT’s capabilities in recommender systems[C]// Proceedings of the 17th ACM Conference on Recommender Systems. New York: ACM, 2023:1126-1132.

[10] HOU Y P, ZHANG J J, LIN Z H, et al. Large language models are zero-shot rankers for recommender systems[EB/OL].(2024-01-24)[2024-08-03]. https://arxiv.org/pdf/2305.08845.

[11] LI J J, ZHANG W T, WANG T, et al. GPT4Rec:a generative framework for personalized recommendation and user interests interpretation[EB/OL].(2023-04-08)[2024-08-03]. https://arxiv.org/pdf/2304.03879.

[12] HE Z K, XIE Z H, JHA R, et al. Large language models as zero-shot conversational recommenders[C]// Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. New York: ACM, 2023:720-730.

[13] ZHANG F Z, YUAN N J, LIAN D F, et al. Collaborative knowledge base embedding for recommender systems[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016:353-362.

[14] QIU Z C, TAO Y, PAN S R, et al. Knowledge graphs and pre-trained language models enhanced representation learning for conversational recommender systems[EB/OL].(2024-05-01)[2024-08-03]. https://arxiv.org/pdf/2312.10967.

[15] LUO H R, TANG Z C, et al. ChatKBQA:a generate-then-retrieve framework for knowledge-base question answering with fine-tuned large language models[EB/OL].(2023-10-30)[2024-08-03]. https://arxiv.org/pdf/2310.08975.

[16] LEWIS P, PEREZ E, PIKTUS A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C]// Conference and Workshop on Neural Information Processing Systems. New York: ACM, 2020:9459-9474.

[17] GAO Y F, XIONG Y, GAO X Y, et al. Retrieval-augmented generation for large language models:a survey[EB/OL].(2024-03-27)[2024-08-03]. https://arxiv.org/pdf/2312.10997.

[18] HUANG C K, YU T, XIE K G, et al. Foundation models for recommender systems:a survey and new perspectives[EB/OL].(2024-02-17)[2024-08-03]. https://arxiv.org/pdf/2402.11143.

[19] MA K X, CHENG H, ZHANG Y, et al. Chain-of-skills:a configurable model for open-domain question answering[C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2023:1599-1618.

[20] YU D H, ZHU C G, FANG Y W, et al. KG-FiD:infusing knowledge graph in fusion-in-decoder for open-domain question answering[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022:4961-4974.

[21] LI X X, ZHAO R C, CHIA Y K, et al. Chain of knowledge:a framework for grounding large language models with structured knowledge bases[EB/OL].(2024-02-21)[2024-08-03]. https://arxiv.org/pdf/2305.13269.

[22] YU W H, ZHANG H M, PAN X M, et al. Chain-of-note:enhancing robustness in retrieval-augmented language models[EB/OL].(2023-11-03)[2024-08-03]. https://arxiv.org/pdf/2311.09210.

[23] CHRISTAKOPOULOU K, RADLINSKI F, HOFMANN K. Towards conversational recommender systems[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016:815-824.

[24] LEI W Q, ZHANG G Y, HE X N, et al. Interactive path reasoning on graph for conversational recommendation[C]// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020:2073-2083.

[25] HAYATI S A, KANG D, ZHU Q, et al. INSPIRED:toward sociable recommendation dialog systems[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP). Stroudsburg: ACL, 2020:8142-8152.

[26] ZHOU Y H, ZHOU K, ZHAO W X, et al. C²-CRS:contrastive learning for conversational recommender system[C]// Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. New York: ACM, 2022:1488-1496.

[27] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[EB/OL].(2020-07-22)[2024-08-03]. https://arxiv.org/pdf/2005.14165.

[28] ZHANG Y Z, SUN S Q, GALLEY M, et al. DIALOGPT:large-scale generative pre-training for conversational response generation[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics:System Demonstrations. Stroudsburg: ACL, 2020:270-278.

[29] BAO K Q, ZHANG J Z, ZHANG Y, et al. TALLRec:an effective and efficient tuning framework to align large language model with recommendation[C]// Proceedings of the 17th ACM Conference on Recommender Systems. New York: ACM, 2023:1007-1014.

[30] FANG J B, GAO S, REN P J, et al. A multi-agent conversational recommender system[EB/OL].(2024-02-02)[2024-08-03]. https://arxiv.org/pdf/2402.01135.

[31] FENG Y, LIU S C, XUE Z H, et al. A large language model enhanced conversational recommender system[EB/OL].(2023-08-11)[2024-08-03]. https://arxiv.org/pdf/2308.06212.

[32] CHEEKULA S K, KAPANIPATHI P, DORAN D, et al. Entity recommendations using hierarchical knowledge bases[C]// CEUR Workshop Proceedings. Aachen: CEUR-WS, 2015:5145.

[33] ZHAI J Y, ZHENG X W, WANG C D, et al. Knowledge prompt-tuning for sequential recommendation[C]// Proceedings of the 31st ACM International Conference on Multimedia. New York: ACM, 2023:6451-6461.

[34] CHASE H. Langchain:building applications with LLMs through composability[EB/OL].[2024-08-03]. https://www.tkcnn.com/github/hwchase17/langchainjs.html.

[35] LIN Y K, LIU Z Y, SUN M S, et al. Learning entity and relation embeddings for knowledge graph completion[J]. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015, 29(1):2181-2187.

[36] SANH V, WEBSON A, RAFFEL C, et al. Multitask prompted training enables zero-shot task generalization[EB/OL].(2022-03-17)[2024-08-03]. https://arxiv.org/pdf/2110.08207.

[37] WANG Z G, NG P, MA X F, et al. Multi-passage BERT:a globally normalized BERT model for open-domain question answering[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP). Stroudsburg: ACL, 2019:5877-5881.

[38] REIMERS N, GUREVYCH I. Sentence-BERT:sentence embeddings using Siamese BERT-networks[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP). Stroudsburg: ACL, 2019:3980-3990.

[39] KIM J, KWON Y, JO Y, et al. KG-GPT:a general framework for reasoning on knowledge graphs using large language models[C]// Findings of the Association for Computational Linguistics:EMNLP 2023. Stroudsburg: ACL, 2023:9410-9421.

[40] JOHNSON J, DOUZE M, JEGOU H. Billion-scale similarity search with GPUs[J]. IEEE Transactions on Big Data, 2021, 7(3):535-547.

[41] SUBRAMANIAN S, TRISCHLER A, BENGIO Y, et al. Learning general purpose distributed sentence representations via large scale multi-task learning[EB/OL].(2018-03-30)[2024-08-03]. https://arxiv.org/pdf/1804.00079.

[42] AUER S, BIZER C, KOBILAROV G, et al. DBpedia:a nucleus for a web of open data[M]//ABERER K, CHOI K S, NOY N, et al. eds. The Semantic Web. Heidelberg: Springer-Verlag, 2007:722-735.

[43] ZENG A H, LIU X, DU Z X, et al. GLM-130B:an open bilingual pre-trained model[EB/OL].(2023-10-25)[2024-08-03].https://arxiv.org/pdf/2210.02414.

[44] HE Z K, XIE Z H, STECK H, et al. Reindex-then-adapt:improving large language models for conversational recommendation[EB/OL].(2024-05-20)[2024-08-03]. https://arxiv.org/pdf/2405.12119.

[45] ZHANG J J, XIE R B, HOU Y P, et al. Recommendation as instruction following:a large language model empowered recommendation approach[EB/OL].(2023-05-11)[2024-08-03]. https://arxiv.org/pdf/2305.07001.

[46] ZHOU K, WANG X L, ZHOU Y H, et al. CRSLab:an open-source toolkit for building conversational recommender system[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing:System Demonstrations. Stroudsburg: ACL, 2021:185-193.

[47] ZHANG Y, LI Y F, CUI L Y, et al. Siren’s song in the AI ocean:a survey on hallucination in large language models[EB/OL].(2023-09-03)[2024-08-03]. https://arxiv.org/pdf/2309.01219.

[48] CHEN J W, DONG H D, WANG X, et al. Bias and debias in recommender system:a survey and future directions[J]. ACM Transactions on Information Systems, 2023, 41(3):1-39.

[49] PAPINENI K, ROUKOS S, WARD T, et al. BLEU:a method for automatic evaluation of machine translation[C]// Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg: ACL, 2015:311-318.