The study has prompted calls for updated quality checks to identify AI-generated synthetic data in scientific research. Biostatistician Jack Wilkinson is leading a project to design tools to assess potentially problematic studies, but warns that advances in generative AI could soon offer ways to circumvent these protocols. The capacity of AI to create fake but realistic data sets is a next level of worry, according to Elisabeth Bik, a microbiologist and independent research-integrity consultant.
Key takeaways:
- Researchers have used the technology behind the AI chatbot ChatGPT to create a fake clinical-trial data set to support an unverified scientific claim, raising concerns about research integrity.
- The AI-generated data compared the outcomes of two surgical procedures and wrongly indicated that one treatment is better than the other, which was contrary to real clinical trials.
- Upon closer scrutiny, the AI-generated data failed authenticity checks, showing mismatches in participant sex and unusual clustering of age values, among other issues.
- Experts warn that advances in generative AI could soon offer ways to circumvent authenticity checks, and suggest that AI-based solutions might be needed to automate these checks.