In the end of 2022, OpenAI introduced a Transformer-based Large Language Model (LLM) called “ChatGPT.” In opposition to what OpenAI’s staff expected, ChatGPT became the fastest-growing web-based app in history, reaching 100 million active users in two months (second only to Meta’s). The initial public opinions of ChatGPT were both excellent and indicating potential problems. In February 2023, Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher stated that generative artificial intelligence (AI) is similar to the intellectual revolution brought about by the printing press, this time consolidating and ‘distilling’ the storehouse of human knowledge. In March 2023, Eliezer Yudkowsky, foreseeing extinction-level risks, called for the world’s governments and militaries to shut down the AI project and “be willing to destroy a rogue datacenter by airstrike.” These initial opinions represent two extremes, but the reasoning that falls in between them is common in technology policy analysis: personal opinions of generative AI become part of the basic assumptions used in policy analyses. When assumptions of fundamental importance go unquestioned, it is very easy to fall into the trap of predicting future technological advancements based on current technological conditions. Technology policy analysts of all types do great work, but it is time to identify the gaps in our reasoning and aim for higher standards both individually and collectively. An example illustrates the general tendency. The Center for a New American Security’s Paul Scharre, in his book “Four Battlegrounds”—which in general provides a wealth of insights—speculates about the future of AI, leaning towards the idea that “Building larger, more diverse datasets may result in more robust models. Multimodal datasets may help to build models that can associate concepts represented in multiple formats, such as text, images, video, and audio.” This expectation is based on the idea that scaling up AI systems (making their internal capacity and training datasets larger) will lead to new capabilities, with a positive reference to Richard Sutton’s well-known argument in “The Bitter Lesson” about the benefits of such techniques. Not long after, Microsoft’s researchers contributed to a wave of overly optimistic claims about the future of LLMs with their provocatively titled, “Sparks of Artificial General Intelligence” paper on GPT-4. It is not hard to see how one’s personal impression of GPT-4 could lead to a similar sense of “We’re on the brink of something big here.” However, this is not a valid reason to allow the assumptions tied up in this sentiment to go unchecked in one’s analyses. A lot of research emphasizes the limitations of LLMs and other Transformer-based systems. Hallucinations (authoritative but factually incorrect statements) continue to trouble LLMs, with some researchers suggesting that these are simply inherent features of this technology. According to a recent study, voters using chatbots for basic information about the 2024 elections can easily be misinformed about hallucinated polling places and other false or outdated information. Other research shows that LLMs’ ability to form summarizations and lag behind humans in generalizing; the reasoning abilities of multimodal systems have a similar situation. OpenAI’s most recent creation, called “Sora,” can generate videos from text that look real but invents things and people that don't follow real-world physics. This goes against the idea that new modalities like image and video would lead to the reliable, strong, and explainable AI systems we want. Threadsthinking skills None of this says that there is excessive excitement in the technology world. Carnegie’s Matt O’Shaughnessy is right in pointing out that talk of “superintelligence” is likely to have a negative impact on policymaking due to the fundamental limitations of machine learning. And the Biden administration’s extensive executive order on AI in October 2023, while invoking the Defense Production Act to monitor some powerful AI systems, was more about tone than expected. real-life physics implored However, the problem we see here is not just an issue of excessive excitement but it is about getting stuck in analytic frameworks that are easily ignored in favor of quick publications and self-promotion. The disappointment with AI-enabled and autonomous drones on the battlefield in Ukraine should make us question the alleged rapid progress of breakthroughs happening in 2023. When it comes to quantum information science, it seems easier to find nuance, but there is little reflection when it comes to the future of quantum computing.
At the same time, today's generative AI is beginning to resemble a parody of Mao’s Continuous Revolution. Transforming this technology into human-like “general” intelligence or some other technological marvel always seems to be just one model upgrade away. It must not succumb to challenges from regulatory bodies or popular movements.
Nevertheless, some technology policy analysts are making choices that favor quick publications and self-promotion over deeper analysis and reflection. The disappointment with AI-enabled drones on the battlefield in Ukraine should give us pause about the alleged rapidity of breakthroughs occurring in 2023.superintelligenceThis suggests that there is an overhype in the technology world. Talk of “superintelligence” is likely to negatively affect policymaking due to the fundamental limitations of machine learning. executive order The Biden administration’s extensive executive order on AI in October 2023, which invoked the Defense Production Act to monitor some powerful AI systems, was more about tone than one might expect. computationally powerful computational strengthAI systemsThe Biden administration’s extensive executive order on AI in October 2023, while invoking the Defense Production Act to monitor certain computational powerful AI systems, was more in tone than one might expect.
Yet, the issue we identify here is not just about excessive excitement. It’s also about being stuck in analytic frameworks that are easily ignored in favor of quick publications and individual or organizational self-promotion. The disappointment with AI-enabled and autonomous drones on the battlefield in Ukraine should raise concerns about the alleged rapidity of fundamental breakthroughs occurring in 2023.analysis limitationThe frustration from the performance of AI-enabled and autonomous drones on the battlefield in Ukraine should make us reconsider the supposed rapid progress of breakthroughs happening in 2023. Additionally, when it comes to quantum information science, it appears easier to find nuance, but there is little individual or collective reflection when it comes to the future of quantum computing.
letdown quantum computing However, generative AI today is starting to look like a parody of Mao’s Continuous Revolution—the transformation of this technology into a human-like “general” intelligence or some other marvel of technological imagination is always one model upgrade away, and it cannot be allowed to succumb to challenges from regulatory bodies or popular movements. Overall, today's generative AI is starting to resemble a satire of Mao’s Continuous Revolution. The idea of transforming this technology into human-like “general” intelligence or some other technological wonder always seems to be just one model upgrade away, and it must not be allowed to be defeated by challenges from regulatory bodies or popular movements.The key lesson here is that policy analysts need to make choices when evaluating technology. Choosing certain assumptions over others leads the analyst to a specific set of potential policy options, possibly neglecting other options. to the detriment of others The first impression of new technologies is inevitable and can provide diverse opinions. However, the problem for policy analysis arises when practitioners fail to refine their ideas through rigorous intellectual criticism, and instead neglect other possibilities. Policy analysis combines elements from industry, domestic politics, and international affairs. Identifying a policy challenge is not enough—it must also link societal needs and values with anticipated or actual impacts from developments within its borders or abroad. Our intuition should be the focus of sincere and shared scrutiny. However, policy analysis is a blend of elements from industry, domestic politics, and international affairs. Simply recognizing a policy challenge is not sufficient—it must also establish a connection between societal needs and values and the expected or actual impacts of developments at home or abroad. Our intuition should be the subject of genuine and shared examination. Vincent J. Carchidi.
is a Non-Resident Scholar at the Strategic Technologies and Cyber Security Program at the Middle East Institute. He is also part of the 2024 Cohort of Foreign Policy for America’s NextGen Initiative.
Too often, technology policy analysis relies on individual opinions instead of collective critical thinking. The policy field should strive for better standards. only hype in the technology world. Carnegie’s Matt O’Shaughnessy correctly notes that talk of “superintelligence” is likely to negatively influence policymaking because of machine learning’s fundamental limitations. Additionally, the Biden administration’s extensive October 2023 executive order on AI, while dramatically invoking the Defense Production Act to authorize the monitoring of certain computationally powerful AI systems, was more diverse in tone than one might expect.
Yet, the problem we identify here is not a hype problem per se. Hype is a result of getting stuck in analytic frames that are too easily ignored in favor of quick publications and individual or organizational self-promotion. Lest we mistakenly believe this is just a peculiar LLM-specific tendency, the disappointment of AI-enabled and autonomous drones on the battlefield in Ukraine should raise eyebrows about the alleged rapidity of fundamental breakthroughs occurring in 2023. Moreover, it is easier to find nuance in the domain of quantum information science, but at the same time, little individual or collective reflection appears to arise when its crown jewel of quantum computing begins to see its future downgraded.
Nevertheless, generative AI today is starting to look like a parody of Mao’s Continuous Revolution—the transformation of this technology into a human-like “general” intelligence or some other marvel of technological imagination is always one model upgrade away, and it cannot be allowed to succumb to challenges from regulatory bodies or popular movements.
The takeaway is that policy analysts make choices when assessing technology. The choice of certain assumptions over others presents the analyst with a certain set of possible policy options at the expense of others. That individuals have first impressions about new technologies is unavoidable and can serve as a source of diversity of opinion. The problem for policy analysis arises when practitioners fail to pour their first (or second, or third, etc.) impressions into a shared crucible that exposes unstable ideas to high-temperature intellectual criticism, thereby guiding them toward an articulation of specific policy challenges and solutions without unduly neglecting other possibilities wholesale.
Policy analysis generally is a concoction of ingredients from industry, domestic politics, and international affairs. Merely identifying that a policy challenge exists is not done de novo but from an intuitive link between the needs and values of a society and anticipated or actual impacts by developments within its borders or abroad. That intuition—we all have it—should be the focus of our honest and shared scrutiny.
Vincent J. Carchidi is a Non-Resident Scholar at the Middle East Institute’s Strategic Technologies and Cyber Security Program. He is also a member of Foreign Policy for America’s NextGen Initiative 2024 Cohort.
