Generation and human-expert evaluation of interesting research ideas using knowledge graphs and large language models
先进的AI系统能够访问数百万篇研究论文，可能会激发出供人类单独构思之外的新研究想法。然而，这些AI生成的想法有多有趣，以及我们如何提高它们的质量呢？在这里，我们介绍了SciMuse，一种通过一个基于超过5800万篇科学论文的不断演变的知识图谱生成个性化研究想法的系统，该接口连接了GPT-4。我们与德国马克斯·普朗克学会的100多个研究小组负责人进行了大规模的人评估，他们根据想法的可信度对4000多个个性化研究想法进行了排名。这个评估让我们能够了解科学兴趣与知识图谱核心属性之间的关系。我们发现，数据有效的机器学习能够高精度地预测研究兴趣，使我们能够优化生成的研究想法的兴趣水平。这项工作代表了一个向可能激发不可预见的合作并建议有趣的科学方向迈进的步骤。
Advanced artificial intelligence (AI) systems with access to millions of research papers could inspire new research ideas that may not be conceived by humans alone. However, how interesting are these AI-generated ideas, and how can we improve their quality? Here, we introduce SciMuse, a system that uses an evolving knowledge graph built from more than 58 million scientific papers to generate personalized research ideas via an interface to GPT-4. We conducted a large-scale human evaluation with over 100 research group leaders from the Max Planck Society, who ranked more than 4,000 personalized research ideas based on their level of interest. This evaluation allows us to understand the relationships between scientific interest and the core properties of the knowledge graph. We find that data-efficient machine learning can predict research interest with high precision, allowing us to optimize the interest-level of generated research ideas. This work represents a step towards an artificial scientific muse that could catalyze unforeseen collaborations and suggest interesting avenues for scientists.
https://arxiv.org/abs/2405.17044
https://arxiv.org/pdf/2405.17044.pdf