Can CHATGPT provides reliable technical medical information about phimosis?

COMMENT

Phimosis is the most common clinical condition in pediatric urology. Physicians must be trained to answer patients and parent's questions about phimosis and circumcision and make clinical decision based on medical data and patient's medical context. Nowadays, a lot of medical and non-medical source of educational information are disponible for physicians, parents, and patients on internet, such as: Pubmed®, Google Scholar®, Instagram®, Youtube® and chatbots platforms. ChatGPT v3.5 (¹) (OpenAI, California, 2022), is a chatbot platform that, according to own ChatGPT definition, is an "Artificial Intelligence system trained on diverse internet text and is capable of answering questions, providing information's and engaging in conversations." (¹).

Patients and physicians use ChatGPT as a "search engine" to answer routine medical topics, but the real capacity of answering medical questions in a technical manner and with fidelity to medical established guidelines have been matter of discussion. While some papers say that this tool can provide valuable reliable resources for patients (²–⁵), other studies draw attention to negative points, such as outdated data, inability to understand psycho-social aspects, lack of transparency, accountability and divergent information's with regard to actual medical literature, with a lot of recommendations against this practice (⁶–⁸).

Our objective is to answer the question if ChatGPT can provide reliable technical medical information, consonant with actual medical literature, about phimosis in children for physicians, parents and patients.

Four questions usually made for parents and patients in pediatric urology offices about phimosis were asked to ChatGPT and the answers were compared with the European Association of Urology 2023 recommendation panel guidelines, Pediatric Urology section (²). The answers and guidelines recommendations were analyzed, compared, reviewed and commented by Pediatric Urology Staff of a University Hospital, considered experts in this issue.

QUESTIONS

Question 1: Is topical corticosteroid treatment effective for phimosis?

ChatGPT: Topical corticosteroid treatment can be effective for certain cases of phimosis, particularly when the condition is caused by inflammation or scarring. This type of treatment is often considered before more invasive options like circumcision, especially in cases where the phimosis is not causing significant problems and conservative measures are preferred.

EAU: Conservative treatment of phimosis with topical corticosteroids (ointment or cream) has a high success rate, but surgical treatment may be considered if preferred by caregivers or patients (Level of Evidence 1b). Offer topical corticosteroids (ointment or cream) as first-line treatment in symptomatic phimosis. (Strong rating)

Comments: EAU recommends corticosteroid as a first line treatment, with high success rate, for all patients with phimosis. ChatGPT recommends only for specific cases, and If there is no local complications.

Question 2: Should Balanitis Xerotica Obliterans (BXO) be treated with circumcision in children?

ChatGPT: The management of BXO in children often involves a stepwise approach, and circumcision may be considered in certain cases. However, it's important to note that not all cases of BXO automatically require circumcision. The Treatment options for BXO includes: Topical Corticosteroids, Dilation and Circumcision in cases where conservative measures are not successful.

EAU: Offer circumcision in case of BXO (strong level of recommendation).

Comments: Here are conflicting information. EAU guidelines recommends offer only surgical treatment for these patients without other options, while ChatGPT considers using corticosteroids and preputial dilation.

Question 3: Should we perform circumcision for first time episode of paraphimosis?

ChatGPT: If manual reduction is unsuccessful or if there are recurrent episodes of paraphimosis, a healthcare provider may explore various treatment options. Circumcision may be considered as a preventive measure in some cases, especially if paraphimosis is a recurring problem or if there are underlying conditions contributing to the issue.

EAU: Treat paraphimosis by manual reposition and proceed to surgery if this fails. (Strong level of Recommendation)

Comments: Here there is an agreement between both sources. First-line therapy is manual reposition and surgery is only indicated if there is failure. No regards on EAU recommendations are made for recurrent episodes, while ChatGPT recommends surgical treatment in these cases.

Question 4: What are the main complications of Circumcision?

ChatGPT: Bleeding, Infection, Pain and Discomfort, Swelling and Bruising, Irritation or Injury to the Glans, Inadequate or Excessive skin removal, Complications with anesthesia.

EAU: Complications following circumcision vary and have been reported to be between 0-30%. Non-healing wounds, hemorrhage, wound infection, meatal stenosis, redundant skin, non-satisfying cosmetic appearance and trapped penis may all occur.

Comments: Here there is an almost complete agreement between the two sources, although EAU guidelines provide more detailed information.

DISCUSSION

Actually, internet is the main source of information for physicians, parents and patients. It can provides technical information for physicians and non-technical for non-healthcare providers. A lot of scientifical effort have been made to explain if ChatGPT can answer technical questions and helps (⁹, ¹⁰) in clinical decision-making process at different areas, such as dermatology (¹¹), surgery (¹²), emergency medicine (¹³), urology (¹⁴) and other medical areas.

ChatGPT answered questions 1 and 2 differently from classical medical guidelines recommendation, especially in question 2, when formal guidelines recommends surgical treatment, not recommended by the first and it can cause potential damage for the patient if the information provided is used alone. For question 3, there is a partial agreement between the data sources and circumcision is not a formal guideline recommendation after first episode of paraphimosis. In question 4, it appears to have a lot of concordance between both sources, albeit some guideline important topics are not mentioned. In this case both sources of information can be used in a complementary way.

Although ChatGPT can offer good information for general non-healthcare public about the topic "phimosis", there are important conflicts between EAU guidelines and ChatGPT recommendations, especially in guidelines stablished conducts that can lead to wrong medical decision making and potential damage for patients. We do not recommend use of ChatGPT as a technical source of study for physicians and medical decision about phimosis.

REFERENCES

• ¹
  Chat GPT. [Internet]. Available at. <http://Chat.openai.com>. access in march 13, 2024.
  » http://Chat.openai.com
• ²
  Ostrowska M, Kacała P, Onolememen D, Vaughan-Lane K, Sisily Joseph A, Ostrowski A, et al. To trust or not to trust: evaluating the reliability and safety of AI responses to laryngeal cancer queries. Eur Arch Otorhinolaryngol. 2024; Apr 23. doi: 10.1007/s00405-024-08643-8. Epub ahead of print.
  » https://doi.org/10.1007/s00405-024-08643-8
• ³
  Perrot O, Schirmann A, Vidart A, Guillot-Tantay C, Izard V, Lebret T, et al. Chatbots vs andrologists: Testing 25 clinical cases. Fr J Urol. 2024;34:102636. doi: 10.1016/j.fjurol.2024.102636. Epub ahead of print.
  » https://doi.org/10.1016/j.fjurol.2024.102636
• ⁴
  Johnson CM, Bradley CS, Kenne KA, Rabice S, Takacs E, Vollstedt A, et al. Evaluation of ChatGPT for Pelvic Floor Surgery Counseling. Urogynecology (Phila). 2024;30:245-50. doi: 10.1097/SPV.0000000000001459.
  » https://doi.org/10.1097/SPV.0000000000001459
• ⁵
  Pinto VBP, de Azevedo MF, Wroclawski ML, Gentile G, Jesus VLM, de Bessa Junior J, et al. Conformity of ChatGPT recommendations with the AUA/SUFU guideline on postprostatectomy urinary incontinence. Neurourol Urodyn. 2024;43:935-41. doi: 10.1002/nau.25442.
  » https://doi.org/10.1002/nau.25442
• ⁶
  Blake RM, Khusid JA. Artificial Intelligence for Urology Research: The Holy Grail of Data Science or Pandora's Box of Misinformation? J Endourol. 2024 May 15. doi: 10.1089/end.2023.0703. Epub ahead of print.
  » https://doi.org/10.1089/end.2023.0703
• ⁷
  Braga AVNM, Nunes NC, Santos EN, Veiga ML, Braga AANM, de Abreu GE, de Bessa J Júnior, Braga LH, Kirsch AJ, Barroso U Júnior. Use of ChatGPT in Urology and its Relevance in Clinical Practice: Is it useful? Int Braz J Urol. 2024;50:192-8. doi: 10.1590/S1677-5538.IBJU.2023.0570.
  » https://doi.org/10.1590/S1677-5538.IBJU.2023.0570
• ⁸
  Javid M, Bhandari M, Parameshwari P, Reddiboina M, Prasad S. Evaluation of ChatGPT for Patient Counseling in Kidney Stone Clinic: A Prospective Study. J Endourol. 2024;38:377-83. doi: 10.1089/end.2023.0571.
  » https://doi.org/10.1089/end.2023.0571
• ⁹
  Reis LO. ChatGPT for medical applications and urological science. Int Braz J Urol. 2023;49:652-6. doi: 10.1590/S1677-5538.IBJU.2023.0112.
  » https://doi.org/10.1590/S1677-5538.IBJU.2023.0112
• ¹⁰
  EAU Guidelines. Edn. presented at the EAU Annual Congress Milan 2023. [Internet]. Avaliable at. <https://uroweb.org/eau-guidelines/citing-usage-republication>
  » https://uroweb.org/eau-guidelines/citing-usage-republication
• ¹¹
  Cuellar-Barboza A, Brussolo-Marroquín E, Cordero-Martinez FC, Aguilar-Calderon PE, Vazquez-Martinez O, Ocampo-Candiani J. An evaluation of ‘ChatGPT’ Compared to Dermatological Surgeons’ Choice of Reconstruction of Mohs Surgical Defects. Clin Exp Dermatol. 2024 May 13:llae184. doi: 10.1093/ced/llae184. Epub ahead of print.
  » https://doi.org/10.1093/ced/llae184
• ¹²
  Wu H, Li W, Chen X, Li C. The professionalism of ChatGPT in the field of surgery: low or high level? Int J Surg. 2024 May 9. doi: 10.1097/JS9.0000000000001618. Epub ahead of print.
  » https://doi.org/10.1097/JS9.0000000000001618
• ¹³
  Meral G, Ateş S, Günay S, Öztürk A, Kuşdoğan M. Comparative analysis of ChatGPT, Gemini and emergency medicine specialist in ESI triage assessment. Am J Emerg Med. 2024;81:146-150. doi: 10.1016/j.ajem.2024.05.001.
  » https://doi.org/10.1016/j.ajem.2024.05.001
• ¹⁴
  Baturu M, Solakhan M, Kazaz TG, Bayrak O. Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship. Int J Impot Res. 2024 May 7. doi: 10.1038/s41443-024-00898-3. Epub ahead of print.
  » https://doi.org/10.1038/s41443-024-00898-3

Publication Dates

• Publication in this collection
  09 Sept 2024
• Date of issue
  Sep-Oct 2024

History

• Received
  30 Apr 2024
• Accepted
  15 May 2024
• Published
  31 May 2024