The authors also discuss the potential for AI to exacerbate existing problems in scientific research, such as reproducibility and replicability crises. They argue that the adoption of AI in various fields has been too rapid, leading to a lack of critical inquiry and quality control. However, they also see glimmers of hope, noting that the problem can be mitigated by a culture change where researchers exercise more care in their work and reproducibility studies are incentivized. They conclude by suggesting that a portion of AI-for-science funding should be diverted to better training, critical inquiry, meta-science, reproducibility, and other quality-control efforts.
Key takeaways:
- The article discusses the issue of 'AI hype' and how it is not only produced by companies and media, but also by AI researchers themselves, leading to flawed research and overconfidence in AI capabilities.
- It highlights the problem of 'leakage' in machine learning, which is a pervasive error affecting hundreds of papers across various disciplines, and how this contributes to reproducibility failures in ML-based science.
- The authors argue that the root causes of reproducibility and replicability crises in many scientific fields include the publish-or-perish culture, bias for positive results, lack of incentives for debunking faulty studies, and lack of consequences for publishing poor quality work.
- Despite the challenges, the authors see glimmers of hope in the form of initiatives like the REFORMS checklist for ML-based science and the potential for a culture change where researchers exercise more care in their work and reproducibility studies are incentivized.