Will Generative AI Have Long-Term Benefits?
By Paris Marx
Remember the excitement in early 2023 when Sam Altman and others claimed generative AI would transform the world? They promised AI-powered chatbots would revolutionize education, healthcare, and professional tasks. Now, the evidence is trickling in, and it seems those bold claims were exaggerated.
Education: AI as a Crutch?
Researchers from the University of Pennsylvania studied how ChatGPT impacted Turkish high school students’ math performance. Their findings? Far from improving learning, the chatbot became a crutch, potentially harming students’ ability to solve problems independently. In the study, students who used ChatGPT performed 48% better on practice problems but did 17% worse on actual tests compared to those who didn’t rely on AI. Even when ChatGPT only provided hints, students still failed to perform better in real test conditions than their peers who studied without AI. This suggests that generative AI may hinder, rather than help, long-term learning outcomes when used as a shortcut.
Healthcare: Trusting AI to a Fault
AI has been promoted as a game-changer in medicine, particularly in fields like cancer screening. However, recent studies highlight a troubling trend known as automation bias. When AI was introduced to assist radiologists in reviewing mammograms, the results were alarming. The radiologists became overly reliant on the AI’s suggestions—even when they were intentionally incorrect. Inexperienced and moderately experienced radiologists saw their accuracy plummet from 80% to just 22% when the AI made errors. Even highly experienced professionals were not immune, with their accuracy dropping to 45%. These findings raise serious concerns about the reliability of AI in high-stakes medical contexts.
Business: AI as a Document Assistant? Not Quite
While AI is touted as a powerful tool for business, its performance as a document assistant has come under scrutiny. Amazon recently conducted an analysis for the Australian Securities and Investments Commission to assess whether generative AI could accurately summarize documents. The results were underwhelming. The AI models consistently underperformed compared to humans, especially when tasked with summarizing complex documents like parliamentary submissions. Instead of streamlining work, the AI created more tasks for human reviewers. This aligns with a Federal Reserve Bank of New York survey showing that most companies adopting AI haven’t seen a significant reduction in jobs, despite early promises of workforce disruption.
The AI Bubble: What Comes Next?
As the hype around AI continues, studies like these challenge the narrative that generative AI will bring widespread, long-term benefits. While some investors and tech executives may profit immensely, the promised revolutionary impacts on education, healthcare, and business are far from guaranteed. For the rest of us, the question remains: will AI’s impact be as transformative as promised, or are we seeing the early signs of an overblown tech bubble?
