Unveiling the Thematic Landscape of Generative Pre-trained Transformer (GPT) Through Bibliometric Analysis

Introduction: the Generative Pre-trained Transformer (GPT) is a deep learning language model architecture developed by OpenAI. Aim: to describe the knowledge networks (both at the theoretical and country levels) of the Generative Pre-trained Transformer (GPT) as an emerging technology. Results: 222 Documents were identified, of which 69 were articles, 50 were conference papers, 36 were editorials, 29 were notes, 19 were letters, 14 were reviews, 3 were conference reviews, and 2 were short surveys. In terms of the number of documents per year, 2 were found in 2019, 10 in 2020, 22 in 2021, 44 in 2022, and 144 in 2023. The year-on-year growth rate was over 100 % in all years. The subject area with the highest number of documents was Computer Science with 90 documents. The most productive countries in relation to GPT were the United States with 60 documents, followed by China with 19, the United Kingdom with 18, India with 15, and Australia with 12. Co-occurrence illustrated the centrality of Artificial Intelligence, Natural Language Processing, Deep Learning, and the term Human around ChatGPT and GPT. Conclusions: this bibliometric study aimed to describe the knowledge networks of the Generative Pre-trained Transformer (GPT) as an emerging technology. Although only 222 documents were found, this study revealed a high level of international scientific collaboration in the field. The results suggest that GPT is a highly relevant technology with a wide range of potential applications in natural language processing, artificial intelligence, and deep learning. Moreover, the study was able to qualitatively characterize the main thematic areas surrounding GPT, including its applications in chatbots, text generation, machine translation, sentiment analysis, and more.


INTRODUCTION
The Generative Pre-trained Transformer (GPT) is a deep learning language model architecture developed by OpenAI. The latest version of GPT is GPT-4, which was released in 2023 and is one of the largest and most powerful language models that exist, with 100 trillion parameters, nearly 600 times more than its predecessor. (1) The current relevance of GPT-4 and the GPT architecture in general is very high due to its ability to generate natural and coherent text in a wide variety of natural language processing tasks. This tool has been used in a large number of applications, including chatbots, text generation, automatic translation, sentiment analysis, and more. (1) In addition, GPT has driven research in the field of deep learning and has led to new innovations and improvements in language model architecture. GPT's ability to generate high-quality text and its adaptability to a wide variety of natural language processing tasks has led to the creation of new applications and opportunities in the field of artificial intelligence. (2,3) The objective of this bibliometric study is primarily focused on describing the knowledge networks (both at the theoretical and country levels) of the Generative Pre-trained Transformer (GPT) as an emerging technology.

METHODS
The bibliometric study utilized Scopus as the database for collecting relevant publications related to the Generative Pre-trained Transformer (GPT) as an emerging technology. Scopus is one of the largest abstract and citation databases of peer-reviewed literature, providing a comprehensive overview of research output in various fields.

Data Collection
A search was conducted on Scopus using the following search string: TITLE-ABS-KEY("Generative Pre-trained Transformer" OR "ChatGPT" OR "GptChat" OR "Chat GPT" OR "GPT Chat").

Data Analysis
The retrieved data was analyzed using the bibliometric software Vosviewer. Vosviewer is a popular bibliometric analysis tool that allows for the visualization and analysis of bibliometric data, including the generation of maps of science and network diagrams.
The bibliometric analysis focused on the identification of key authors and countries that have contributed to the literature on GPT.
Network analysis was conducted to visualize the co-occurrence of keywords in the GPT literature. The analysis was used to identify clusters of related keywords and to explore the relationships between different areas of research related to GPT.
The year-on-year growth rate was over 100 % in all years (figure 1).  The most productive countries in relation to GPT were the United States with 60 documents, followed by China with 19, the United Kingdom with 18, India with 15, and Australia with 12. figure 2 figure 3 shows the co-authorship networks.

DISCUSSION
GPT is a powerful tool in the field of natural language processing, and its impact on scientific and technological development cannot be overstated. Its ability to generate natural and coherent text in response to user input has been a game-changer in the development of chatbots and virtual assistants.
These Moreover, the development of ChatGPT has also had a significant impact on the field of artificial intelligence as a whole. Its use of deep learning techniques and advanced algorithms has led to new breakthroughs in the development of language models and other AI applications. As researchers continue to refine and improve ChatGPT, it is likely that we will see even more impressive developments in the field of AI in the years to come. This technology has already proven to be a powerful tool for advancing scientific and technological progress, and its potential for future innovations is virtually limitless. (4,5,6) In addition to its practical applications, ChatGPT has also had an important impact on the way we think about language and communication. By simulating human-like conversations, ChatGPT has helped to shed new light on the complexities of human language and the ways in which we use it to convey meaning. As researchers continue to study and refine ChatGPT and other natural language processing tools, we are likely to gain a deeper understanding of the way language works and the ways in which it shapes our interactions with the world around us. This knowledge has the potential to inform a wide range of fields, from linguistics and psychology to philosophy and beyond. (7,8,9) Several countries have been actively involved in the development of Generative Pre-trained Transformer (GPT) technology. The United States is the most productive country, with 60 documents related to GPT found in the Scopus database. The country has been at the forefront of artificial intelligence research, with major companies like OpenAI, Google, and Microsoft headquartered there. China is the second most productive country, with 19 GPT-related documents found in Scopus. The country has been investing heavily in AI research and development, and its companies are known for their advancements in natural language processing and machine learning.
The United Kingdom and India are also prominent contributors to GPT-related research, with 18 and 15 documents found in Scopus, respectively. The UK is known for its strong academic tradition and has several topranked universities that have been involved in GPT research. India, on the other hand, has a large tech industry and is home to several companies that have been developing AI and machine learning technologies. Australia is another country that has been active in GPT research, with 12 documents found in Scopus. The country has a strong research focus on natural language processing and has several universities that have been involved in GPT-related research.
The potential uses of natural language processing based on artificial intelligence in scientific research are vast and promising. One of the most significant applications is the ability to analyze and extract valuable insights from large amounts of unstructured data. With AI-powered tools such as chatbots and virtual assistants, researchers can efficiently interact with data sources, extract relevant information, and gain deeper insights into complex scientific problems. Furthermore, AI can help automate routine tasks such as data cleaning and data analysis, freeing up researchers' time to focus on more critical tasks. (10,11,12,13) The studies found in the bibliometric analysis indicate that the main themes of research in the field of natural language processing based on AI include artificial intelligence, natural language processing, deep learning, and human. These themes suggest that researchers are interested in exploring how AI can be used to improve natural language processing technologies and how these technologies can be used to enhance human interaction with technology. For example, AI-powered chatbots and virtual assistants can provide personalized recommendations to users, assist in language learning, and provide customer service. Moreover, AI can help researchers better understand how humans interact with language, opening up new avenues of research in fields such as psychology and linguistics.
The potential uses of natural language processing based on AI in scientific research are vast and promising. (14) The studies found in the bibliometric analysis indicate that research in this field is focused on improving AI-powered technologies and understanding how these technologies can be used to enhance human interaction with technology. As AI continues to evolve and become more advanced, it is likely that we will see even more innovative uses of natural language processing in scientific research.

Limitations
One potential limitation of the study is the use of Scopus as the only database. Scopus covers a large number of journals, but it is possible that some relevant publications may have been missed. Additionally, the study only covers publications up to early 2023, and new research may have been published since then.

CONCLUSIONS
In conclusion, this bibliometric study aimed to describe the knowledge networks of the Generative Pretrained Transformer (GPT) as an emerging technology. Although only 222 documents were found, this study revealed a high level of international scientific collaboration in the field. The results suggest that GPT is a highly relevant technology with a wide range of potential applications in natural language processing, artificial intelligence, and deep learning. Moreover, the study was able to qualitatively characterize the main thematic areas surrounding GPT, including its applications in chatbots, text generation, machine translation, sentiment analysis, and more. The findings suggest that GPT has significant potential to drive innovation and contribute to the advancement of the field of artificial intelligence.
Overall, despite the limited number of documents found, this study provides valuable insights into the knowledge networks and potential applications of GPT. The high level of international collaboration and the identified thematic areas suggest that GPT is a promising area for future research and development.