Hi, thank you for your question. The key thing is that the authors are reporting real results from real research. I'm not sure if it matters how much of the text is generated by AI since AI might autocomplete every sentence or paragraph in the future. I also think that means the % of AI we see in real research will grow over time meaning that if we had a cutoff, it would have to keep changing. Right now, if we want to ensure that the research has been done, requiring data and code from authors is a really good way to go. While it's possible to machine-generate fake code and data, it's still very hard to make that work compared with just writing a fake paper. Of course, authors are reluctant to provide data and code, so we still recommend getting good referees on everything.

Hi Yatao, I've not had direct experience with the tool you mentioned, but I think it all depends on how the tool works. As I mentioned during my segment, most AI detection tools may have bias against non-native English speakers if the method of detection relies on the complexity of your sentence structure. If you have learned any language as a second language, you tend to follow the grammatical rules that you were taught, so your sentence structures follow a predictable flow. This is what the AI detection system is using to determine the statistical probability that your work might be AI generated because AI writes based on statistical prediction of the most likely next expected word.

There is still value in detecting AI usage, but I think every tool will have their own shortcomings in the same way that academic research also has limitations. The key thing is to understand what the limitations are before using any tool.

No

No. Depending on how it is used, it may even increase plagiarism.

I don't think AI will be the solution to this. Instead, increased education on and awareness of healthcare and scientific landscapes in developing countries for journal editors will make a bigger difference. A colleague shared a case were a publication from Australia received a desk rejection because the 'patient sample size was too small'. It reality, the sample size was an accurate representation of the patient population in Australia.

AI is just a tool. Whether it increases or reduces bias is really dependent on how aware the person using the AI system is.

It's a good question. We've always taken the view that the AI shouldn't make decisions and humans have to be in the loop for that. The reality is always going to be more nuanced. Some decisions are harmless (e.g. deciding which algorithm to use in order to process a case efficiently) and can be automated as long as it's responsible to do so. So that's where human Oversight comes in - we need there to be more high-level orchestration with people making responsible decisions about how that's done. This pushes more of the grind work to the machines freeing up people's time for the important decisions.