Thoughts on Using AI in Scientific Research

Out of curiosity, I used AI to write the first version of this post. In the meta-spirit, I wanted to see how AI would assess the use of itself. While the AI-generated images seemed a bit generic and unrelated, I was quite happy with the fact that it addressed issues of implicit bias directly, as this is one of the biggest issues I see with using AI regularly. I also appreciated that the post addressed the need to build transparent AI systems and concerns about ethical data usage for AI training. There were, however, important points to address when using AI as a tool in scientific research and reporting, such as the importance of prompting, the prevalence of AI hallucinations, and the requirement for researchers to cite AI use appropriately.

Implicit bias

I want to dive in a little bit deeper on implicit bias. The first AI post said:

AI systems learn from historical data, which may carry existing societal biases.

This is true, AI systems learn from historical data, which certainly carries implicit biases. These implicit biases can impact the diversity in the research teams as well as the diversity of interpretation of results.

Diagram of a neural network illustrating how interconnected nodes use weighted functions to process inputs and deliver outputs, highlighting the potential for implicit biases based on training datasets and parameter optimizations.

In order to understand the origin of these biases let's look at Internet user data used in training these AI systems. Pew Research Center reported Internet usage data in February 2016. At this point Internet user data was largely biased by socio-economic standing, race, and gender. When we train AI on historical data, it learns from data that largely favors the opinions of men in advanced economies particularly in the United States and Europe.

For more updated statistics without interpretation, check out Statista's reports on internet usage. The Statistica data shows that internet access in China and India have drastically increased since the Pew Research Center's report in 2016. It is important to know what are the training years of the AI program you are using and therefore where in the biases may originate. When using AI, it is important to be informed of cut-off dates for training sets. Computer City released information concerning cut-off dates for commonly used AI systems but seeing as this is constantly evolving be sure that you are checking cut-off dates for the AI systems you are actively employing.

AI Hallucinations and Prompting

Because generative AI systems like chatGPT are principally language models, generate language that resembles human communication patterns. This does not mean that the ideas they are communicating necessarily hold value. For example, when a colleague of mine asked chatGPT to summarize a research article she had written, chatGPT used all of the correct vocabulary, yet arrived at the exact opposite conclusion. Keep an eye out for these hallucinations and be careful to use language models to help formulate language and not ideas. When necessary, fact check AI to verify the information.

In this vain, I will take a second to mention that it is not a smart idea to use AI to fill in gaps in articles to be overwritten at later stages in collaboration. This can easily lead to miscommunications between researchers and oversights in leaving AI-generated ideas in cutting-edge research. For academics, which are often reporting on their novel observations about the world, AI has no training set to meaningfully complete these ideas. Keep AI use limited to language formulation.

Prompting can make or break your AI outputs. Be careful to prompt specifically and approach AI with a targeted goal that meets the AIs generative capabilities. Further, keep a log of prompts used. Many scientific journals and communications accept the use of generative AI systems, but they do ask that it be cited specifically. They may request to see the prompts used in the composition of the paper, so keep track and be transparent.

Take-Aways

Language models mimic our speech patterns, not our thought patterns. AI can add the final touches to a project to make it look polished, but the content of the product should always be clearly defined in the prompts and it should be fact-checked and verified by the author at every turn. Lastly, whenever using AI, always cite it appropriately according to journal guidelines.