AI Insights: A Case Study on Utilizing ChatGPT Intelligence for Research Paper Analysis
Resource type
Preprint
Authors/contributors
- De Silva, Anjalee (Author)
- Wijekoon, Janaka L. (Author)
- Liyanarachchi, Rashini (Author)
- Panchendrarajan, Rrubaa (Author)
- Rajapaksha, Weranga (Author)
Title
AI Insights: A Case Study on Utilizing ChatGPT Intelligence for Research Paper Analysis
Abstract
This paper discusses the effectiveness of leveraging Chatbot: Generative Pre-trained Transformer (ChatGPT) versions 3.5 and 4 for analyzing research papers for effective writing of scientific literature surveys. The study selected the \textit{Application of Artificial Intelligence in Breast Cancer Treatment} as the research topic. Research papers related to this topic were collected from three major publication databases Google Scholar, Pubmed, and Scopus. ChatGPT models were used to identify the category, scope, and relevant information from the research papers for automatic identification of relevant papers related to Breast Cancer Treatment (BCT), organization of papers according to scope, and identification of key information for survey paper writing. Evaluations performed using ground truth data annotated using subject experts reveal, that GPT-4 achieves 77.3\% accuracy in identifying the research paper categories and 50\% of the papers were correctly identified by GPT-4 for their scopes. Further, the results demonstrate that GPT-4 can generate reasons for its decisions with an average of 27\% new words, and 67\% of the reasons given by the model were completely agreeable to the subject experts.
Repository
arXiv
Archive ID
arXiv:2403.03293
Date
2024-03-05
Accessed
12/03/2024, 20:52
Short Title
AI Insights
Library Catalogue
Extra
arXiv:2403.03293 [cs]
Citation
De Silva, A., Wijekoon, J. L., Liyanarachchi, R., Panchendrarajan, R., & Rajapaksha, W. (2024). AI Insights: A Case Study on Utilizing ChatGPT Intelligence for Research Paper Analysis (arXiv:2403.03293). arXiv. http://arxiv.org/abs/2403.03293
Link to this record