The company behind the NHS GP Claims that its chatbot beat can diagnose medical conditions as accurately as a GP. It sparked a row between the software’s creators and UK doctors. Babylon, It revealed the artificial intelligence bot at an event held at the Royal College of Physicians. But another medical professional body said it doubtedthe AI’s abilities. “No app or algorithm will be able to do what a GP does,” said the Royal College of General Practitioners. “An app might be able to pass an automated clinical knowledge test – but the answer to a clinical scenario isn’t always cut and dried. “There are many factors to take into account, a great deal of risk to manage, and the emotional impact a diagnosis might have on a patient to consider.” But NHS England chairman Sir Malcolm Grant – who attended the unveil – appeared to be more receptive. “It is difficult to imagine the historical model of a general practitioner, which is after all the foundation stone of the NHS and medicine, not evolving,” he said. “We are at a tipping point of how we provide care. “This is why we are paying very close attention to what you’ve been doing and what other companies are doing.” The chatbot AI has been tested on what Babylon said was a representative set of questions from the Membership of the Royal College of General Practitioners exam. The MRCGP is the final test set for trainee GPs to be accredited by the organisation. Babylon said that the first time its AI sat the exam, it achieved a score of 81%. It added that the average mark for human doctors was 72%, based on results logged between 2012 and 2017. Babylon said it had used example questions published directly by the college and that some had indeed been made publicly available. “We would be delighted if they could formally share with us their examination papers so I could replicate the exam exactly. That would be great,” Babylon chief executive Ali Parsa told the BBC.
To further test the AI, Babylon partnered with doctors at two US organisations – Stanford Primary Care and Yale New Haven Health – as well as doctors from the Royal College of Physicians. It said they had developed 100 real-life scenarios to test the AI