Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine (NEJM - Correspondence)

By Dr Frank Yap, MD -

TO THE EDITOR (Part 1)

With regard to the special report by Lee et al. (March 30 issue), we have concerns about the reproducibility of the results and the need for proper reporting of the parameters used in the GPT-4 (Generative Pretrained Transformer 4) artificial intelligence (AI) model. We attempted to reproduce the results using ChatGPT (March 23 version) with GPT-4, and we could not reproduce the conversations. Do the authors have experience with the reproducibility of GPT-4 outputs when it is given the same query at different times? Is the corpus of learned knowledge changing over time? What specific parameters should users specify if they want reproducible outputs from ChatGPT?

Reproducibility is a cornerstone of good medical practice; will GPT-4 be able to deliver reproducible results? This will be necessary if we are to have reliable AI-driven solutions in medicine.

Yuki Kataoka, M.D., Dr.P.H.
Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan

Ryuhei So, M.D., M.P.H.
Okayama Psychiatric Medical Center, Okayama, Japan



Reference:

1. Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med 2023;388:1233-1239.


TO THE EDITOR (Part 2)

In their special report, Lee et al. discuss the potential use of GPT-4 in the medical context, including applications in medical note-taking, innate medical knowledge, and medical consultation. We think that an important use of GPT-4 and related programs will be in medical education. Adapting medical education to this new tool will require changes in both educators and students — for example, the development of personalized learning methods and collaborative environments, as well as discussions about ethical implications related to AI and potential biases.1,2 AI will be a powerful ally in the management of medical knowledge, allowing users to apply accumulated knowledge to the processes of diagnostic and therapeutic reasoning in order to support accurate decision making.3 For this to happen, students need training and familiarity with these systems, starting at the beginning of medical training. Do the authors agree that educators must shift from being transmitters of information to being guides in the development of competencies, skills, and critical thinking and that students must be increasingly active in learning to use these new sources of information responsibly?

Alexandre C. Fernandes, M.D.
Edmond and Lily Safra International Institute of Neuroscience, Macaíba, Brazil

Maria E.V.C. Souto
State University of Rio Grande do Norte, Mossoró, Brazil



References:

1. Char DS, Shah NH, Magnus D. Implementing machine learning in health care — addressing ethical challenges. N Engl J Med 2018;378:981-983.

2. Haupt CE, Marks M. AI-generated medical advice-GPT and beyond. JAMA 2023;329:1349-1350.

3. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44-56.

Response

The authors reply: In response to Kataoka and So: parts of the GPT-4 AI model are constantly evolving, and therefore its outputs change over time. Although it may be technically feasible to “freeze” the system so that it does not change, GPT-4 also uses some elements of randomness when analyzing its inputs and generating its outputs. This means that even if the system were frozen, it would still tend to give different outputs even when prompted with precisely the same inputs. (In this sense, GPT-4 behaves more like a human being than like a software application.) The changes are mainly in the choice of words, the verbosity of responses, and the format of answers (e.g., in narrative form or as a bulleted list). However, GPT-4 can also produce conceptually different or even contradictory answers, especially in situations in which the input question has no specific or known correct answer.

Fernandes and Souto raise the issue of the importance of chatbots in medical education. As we noted in our article, the potential for benefit exists; we hope it is realized in the future.

Peter Lee, Ph.D.
Sebastien Bubeck, Ph.D.
Microsoft Research, Redmond, WA

Joseph Petro, M.S., M.Eng.
Nuance Communications, Burlington, MA

Reposted from: https://www.nejm.org/doi/full/10.1056/NEJMc2305286


Comments

Post a Comment

Labels

Show more

Popular posts from this blog

10 Best Natural Ozempic Alternatives 2024

By Dr Frank Yap, MD -
Image
If there's anything the Ozempic craze has taught us, it's that people are hungry for weight loss solutions. But as the world leans into wellness, many are now looking beyond pharmaceuticals, turning their gaze to nature's arsenal for blood sugar balance and metabolic health. The 2023 report from the World Obesity Federation estimates that the economic impact of overweight and obesity will surpass $4.3 trillion annually by 2035. In the U.S., the CDC  reports the prevalence of obesity in adults was 41.9% measured from 2017 to March 2020. This represented an increase from 30.5% from 1999 to 2000. The estimated annual medical costs were roughly $173 billion in 2019 dollars. Prescriptions for semaglutide have doubled since mid-2021, rising to more than 1.2 million. In April 2023, a spokesperson for Novo Nordisk, maker of semaglutide, said the drugs are now available in pharmacies nationwide. Semaglutide is sold under the name Ozempic as a diabetes dr
Read more

10 Best Vitamin C Serums Recommended by Dermatologists 2024

By Dr Frank Yap, MD -
Image
Vitamin C serum is a serum containing vitamin C that you can apply on your skin. This vitamin is vital for skin growth and repair. It helps with collagen production too. Make up can cover blemishes but the best vitamin C serums can resolve what you’re trying to cover up in the first place! All it takes is a few drops once a day to lighten blemishes, hyper pigmentation, acne scars, the start of wrinkles, dull skin, etc. Vitamin C serum has a wide array of benefits, and most of them are what we have been hoping for our skin. Even skin tone, blur fine lines, remove acne spots, and achieve a natural glow. Among all other skincare ingredients and products marketed these days, Vitamin C is simply the queen of them all.  With this, we are excited to share with you our top picks of Vitamin C serums. 1. Eva Naturals Skin Clearing Serum Vitamin C+, 30ml - 2% Retinol, 3.5% Niacinamide, 5% Hyaluronic Acid, 2% Salicylic Acid, 10% MSM, 20% Vitamin C  - Skin Clearing, Anti-Aging Skin Repair, Face Ser
Read more

Can Diet and Lifestyle influence your Risk of getting Cancer? Let the Science Speak (2024)

By Dr Frank Yap, MD -
Image
This article compiles and analyses over 300 research references and supporting studies related to cancer and the realm of supplements, diet and lifestyle. Most of us know Steve Jobs, Chadwick Boseman (Black Panther), Robin Gibb (Bee Gees), Donna Summer, Farrah Fawcett, Eartha Kitt, Peter Jennings, Paul Newman, Patrick Swayze, Sydney Pollack, Michael Crichton, Bob Denver, Ted Kennedy, Jerry Orbach, Anne Bancroft, William Rehnquist, and Tony Snow, just to name a few. What do they have in common? They all died from 'cancer'. Prevention holds greater value than both cure and early detection. Nonetheless, the incentive for profit often leans away from investing in preventive measures. This tendency is reflected in the limited coverage of preventative strategies within mainstream media channels. The quest for cancer prevention and treatment has been shrouded in ambiguity when it comes to the world of dietary supplements. The bewildering array of options leaves us questioning their ef
Read more

9 Best Vitamin C Serums for Brighter Skin 2024

By Dr Frank Yap, MD -
Image
Vitamin C is an antioxidant which protects skin cells from damaging free radicals caused by UV exposure. It also helps lighten hyperpigmentation and brown spots, even out skin tone, and enhance skin radiance. Can I Use Vitamin Serum Everyday? Vitamin C should be used everyday for best results. These serums are most effective when your skin is properly exfoliated. What Are The Side Effects of Vitamin C Serum? Vitamin C serums are generally safe for all skin types, however hyper-sensitive skin types might experience side-effects such as.. - Slight Irritation - Itching - Tingling Sensation - Redness Best Vitamin C Serums for Brighter Skin  1. Vichy LiftActiv Vitamin C Serum and Brightening Skin Corrector Buy on Amazon - Anti Aging Serum for Face  - With 15% Pure Vitamin C, Hyaluronic Acid, Vitamin E and antioxidants - For Brighter, Firmer Skin - In 10 days skin appears brighter and feels firmer. - Helps hydrate, brighten, firm, hydrate & visibly reduce fine lines.  - Formulated with 1
Read more

10 Best Cosmeceutical Ingredients of 2024

By Dr Frank Yap, MD -
Image
A cosmeceutical is essentially a skincare product that contains a biologically active compound that is thought to have pharmaceutical effects on the skin. Both pharmaceutical and cosmeceutical grade skin care products actively effect skin at a cellular level, whereas cosmetic products have a shorter-term effect. They are usually used by doctors to restore pH, remove pigmentation, or restore the skin barrier. Cosmeceuticals must scientifically verify the claims stated on their packaging; as it is accepted that these formulations are backed by science, clinical studies, and rigorous testing. The main areas of concerns that these formulations address are ageing skin, sensitive/sensitized skin, sun damaged skin, acne/oily skin, loss of tone and elasticity, as well as superficial lines & wrinkles to encourage rapid cellular turnover that will renew and re-texturize the skin. Peptides composed of chains of amino acids give proteins their structure, and transport nutrients and signal mess
Read more

7 Best Vitamin C Serums for Hyperpigmentation 2024

By Dr Frank Yap, MD -
Image
Vitamin C serums are serums that contain vitamin C which you can apply on your skin. These serums have a wide array of benefits. They help even skin tone, blur fine lines, remove acne spots, and achieve a natural glow.  They are also vital for skin growth and repair. They help with collagen production too. How Does Vitamin C Treat Hyperpigmentation? Vitamin C reduces the effects of hyperpigmentation by decreasing melatonin formation and fading dark spots. Check out the recommendations below for the best Vitamin C Serums online! Best Vitamin C Serums for Hyperpigmentation 1.  Neoprosone Vitamin C Serum, 30ml - For Dark Circle, Wrinkles & Dark Spots, Hyperpigmentation Treatment  - With Alpha Arbutin and Castor Oil - Promotes collagen and elastin synthesis. Antioxidants also prevent damage to our skin by neutralizing free radicals, which are generated by our bodies when exposed to certain conditions, including pollution and UV rays.  - Best Vitamin C Serum for Dark Spots - NEOPROSONE
Read more

Linoleic Acid vs Linolenic Acid: What's the Difference?

By -
Image
Linoleic acid (LA) is an omega-6 fatty acid, while α-linolenic acid (ALA) is an omega-3 fatty acid. Both are the two essential fatty acids required in the diet. They help to support healthy metabolism, good cognitive function, and healthy skin and hair growth. This multipurpose compound is essential for healthy tissue and cell growth. What is Linoleic Acid? Linoleic acid, whose name is derived from the Greek words "linon," which means "flax," and "oleic," which means "produced from oil," is an unsaturated fat. Unsaturated fats are essential for encouraging cell regeneration and keeping skin smooth, elastic, and young-looking. Linoleic acid can be used on all skin types, but those with dry or acne-prone skin may find it most beneficial. Acne patients have been demonstrated to have reduced linoleic acid levels in their skin surface lipids. Is Gamma Linolenic Acid the Same as Linoleic Acid? Gamma linolenic acid, is produced in the body from linoleic
Read more

Linoleic Acid vs Oleic Acid: What's the Difference?

By -
Image
What is Linoleic Acid? Linoleic acid, whose name is derived from the Greek words "linon," which means "flax," and "oleic," which means "produced from oil," is an unsaturated fat. Unsaturated fats are essential for encouraging cell regeneration and keeping skin smooth, elastic, and young-looking. Linoleic acid can be used on all skin types, but those with dry or acne-prone skin may find it most beneficial. Acne patients have been demonstrated to have reduced linoleic acid levels in their skin surface lipids. What is Oleic Acid? Oleic acid is a mono-unsaturated omega-9 fatty acid found in various animal and vegetable sources. Oleic acid is considered to be highly stable to oxidation and also can enhance the activity of antioxidants and antipolymerization agents. ( Source ) Difference Between Oleic Acid and Linoleic Acid Linoleic Acid Vs Oleic Acid for Skin Your ability to choose between the fatty acids will depend on your skin type. Linoleic acid-r
Read more

7 Best Cetylpyridinium Chloride Mouthwash Brands 2023

By Dr Frank Yap, MD -
Image
A study, published in Nature ( Takeda 2022 ) concluded that CPC (CetylPyridinium Chloride) may inhibit SARS-CoV-2 infection even at lower concentrations. Another team of international scientists has also demonstrated the usefulness of cetylpyridinium chloride-containing mouthwashes in inhibiting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and reducing the risk of viral transmission. The study is published in the Journal of Medical Microbiology (2022). A scientific review, published in the  Journal of Formosan Medical Association  (2022) also concluded that mouthwash containing CPC shows potential for reducing the viral load of COVID-19 virus in the throat. Instruction: twice a day or after having been out for prevention, and three times a day for early treatment of COVID-19. What is Cetylpyridinium Chloride (CPC) Made Of? Cetylpyridinium chloride (CPC) is obtained by quaternization of pyridine with cetyl chloride What Does Cetylpyridinium Chloride Do in Mouthwash? CPC
Read more

Chlormequat: Breakfast Cereals Scrutinized for Pesticide That May Harm Reproduction

By Dr Frank Yap, MD -
Imagine starting your day with a bowl of cereal that could be silently affecting your family’s health. Recent studies show that chlormequat, a pesticide linked to reproductive issues, has been found in popular breakfast cereals like Quaker Oats and Cheerios. As this substance infiltrates the American food supply, the potential risks to our health and future generations loom larger, raising urgent questions about the safety of our everyday food choices. In a study published in the Journal of Exposure Science & Environmental Epidemiology on Feb. 15, 2024, researchers revealed alarming findings regarding the prevalence of chlormequat. Chlormequat was detected in the urine of 4 out of 5 people or 80 percent of Americans tested. Additionally, 92 percent of oat-based foods tested contained chlormequat, including Quaker Oats and Cheerios. This study—the first to report urinary chlormequat measurements in adults living in the United States—highlights the possible widespread presence of c
Read more

Archive

Show more