AI & medical consultations – opportunities and risks of tools like ChatGPT and Claude
What if medical consultations no longer start with a waiting room – but with an AI chat window.
Artificial intelligence has found its way into multiple areas within the field of medicine [1]: from patients’ perspectives in the form of fitness trackers or chatbots, for example, and from medical professionals’ perspectives with regard to radiological image analysis or even diagnostics.
When preparing for doctor-patient consultations, patients primarily use large language models (LLMs), such as the well-known and universally accessible models ChatGPT and Claude.
What are these LLMs and what does their use mean for patients? Discover how even patients can make good use of them but also where their limits lie – and how important it is for you, as a human, to remain attentive and active when it comes to medical consultations.
What are large language models? Large language models (LLMs) are mathematical language models with the ability to independently generate texts on the basis of complex probability calculations. They are trained using massive datasets, i.e. accessible texts, e.g. from the Internet, that are made available to them.
Opportunities and risks of LLMs
LLMs can help you collect information, including assessing symptoms, formulating questions or explaining medical terms. This makes them a valuable tool for both before and after doctor-patient consultations.
At the same time, LLMs also bring with them risks that require them to be used carefully. The biggest of these is that LLMs do not know everything. For example, a large-scale study determined that in around 32% of cases, LLMs could even be deceived by misinformation.
AI can generate ‘hallucinations’ [4] – incorrect information that sounds so convincing in the polished language of LLMs that people think it is true.
Bias also plays a role: some disease progressions or patient groups are under or over-represented, which can potentially lead to inaccurate assumptions. [5]
There is also the matter of data protection. Personal health data should not be input into AI tools without careful consideration, as how it is processed is often unclear. [4]
Key challenges: flood of information and unsuitable structure
The vast amount of information output by LLMs can quickly seem overwhelming. In particular, if there is a lack of further structure and support, AI content can help to provoke fears and concerns. [5]
The lack of structure can ultimately result in us having gathered lots of information but still failing to retain a clear picture during the doctor-patient consultation itself. [7] AI answers are often lengthy, unstructured and not tailored to the specific situation in the consultation. We are on our own when deciding what to prioritise and how to usefully integrate information into consultations.
Conclusion: AI can support you both before and after medical consultations – but only if you remain attentive and active
AI is not a substitute for doctor-patient consultations. However, it can offer you useful support as a patient. For example, you can obtain clear explanations of specialist terms or ask about medical correlations. In particular, this can make complex diagnoses or treatment options easier to grasp.
However, AI is only an option if you remain attentive and always check the veracity of the information.
You should also protect yourself against the flood of information. In addition to AI, we recommend using a tool like EverAsk where you can prepare your questions and inputs in a structured manner tailored to your upcoming medical consultation.
Thanks to EverAsk, patients can use clear categories – objectives, questions and notes. The structure of EverAsk was developed in collaboration with the University Hospital Basel and on the basis of proven communication models.
Structure questions instead of drowning in information. This enables your active participation in consultations in line with your needs.
In stressful situations in particular, the structure of EverAsk prevents you from falling into a passive role. Instead, you remain active even when under pressure, continue to ask targeted questions and can therefore make better-founded decisions. [6]
The aim is to structure your questions instead of drowning in information. This enables your active participation in consultations in line with your needs.
Practical info: safe and sensible use of AI
LLMs can provide information – EverAsk provides structure
Use LLMs, for example, to obtain clear explanations of medical terms
Critically question LLM answers (think hallucinations and bias):
- Use more than just one LLM and compare the answers.
- Check source information by opening the links provided – and if there are no links, ask for sources.
- Cross-check important answers with trustworthy information sources.
- Use the deep research function for important questions (costs may apply).
Only enter anonymised information into LLMs and AI tools
Use the post-processing function in EverAsk to monitor and record notes
Always clarify any outstanding points with a doctor directly
Quote from Peter Hug, Head of Services and Products, Member of the Management Board
Until recently, people used to consult ‘Dr Google’. Now, they use ChatGPT and co. It doesn’t take long to get a plausible-sounding answer – but how reliable is the AI diagnosis? On the one hand, AI is a practical tool but on the other, caution is also required as not all sources are medically reliable. We therefore advise our policyholders to use the symptom checker in the Well app or contact the Medi24 telemedicine advice centre by phone or chat (DoctorChat) in the event of medical questions. This gives them quick and free access to professional medical advice – 24/7
Sources
- Sprecher, F., Brauchbar, M., & Frey, S. „Künstliche Intelligenz (KI) im Schweizer Gesundheitswesen”: Studie KI im Schweizerischen Gesundheitswesen.pdf
- Hautz et al. (2025). Diagnoses supported by a computerised diagnostic decision support system versus conventional diagnoses in emergency patients (DDX-BRO): a multicentre, multiple-period, double-blind, cluster-randomised, crossover superiority trial. The Lancet Digital Health, 7(2):136-144.
- Hswen Y, Rubin R. (2025). An AI Chatbot Outperformed Physicians and Physicians Plus AI in a Trial—What Does That Mean? JAMA. 333(4):273–276.
- Bundesärztekammer: Thesenpapier „KI in der Gesundheitsversorgung”: https://www.bundesaerztekammer.de/fileadmin/user_upload/BAEK/Themen/Digitalisierung/Thesenpapier_KI_in_der_Gesundheitsversorgung_03.2025.pdf
- Hassan, N., Slight, R., Bimpong, K. et al. (2024). Systematic review to understand users perspectives on AI-enabled decision aids to inform shared decision making. npj Digit. Med. 7, 332.
- PMC (PubMed Central): „Implementation of AI in healthcare” https://pmc.ncbi.nlm.nih.gov/articles/PMC12287119/
Patientenprogramme und Compliance: Deutsches Gesundheitsportal (Studie): https://www.deutschesgesundheitsportal.de/wp-content/uploads/pdfs/patientenprogramme-unterstuetzen-therapietreue.pdf