The National Geospatial-Intelligence Agency is using machine learning and computer vision in all aspects of its operations, from the battlefield to geopolitical analysis. The agency is using rapidly advancing artificial intelligence technologies to help military leaders observe battlefield activity more closely and give policymakers a better understanding of global threats and dynamics.
"We are very enthusiastic about the direction of AI applications," said NGA’s director, Vice Adm. Frank Whitworth, in an interview with SpaceNews.
Whitworth stated that a combination of rapidly advancing AI capabilities and updated business processes is changing how the agency provides intelligence to inform national security decision-making at the highest levels.
The agency, based in Springfield, Virginia, is expanding on the work of Project Maven, a Pentagon initiative started in 2017 to utilize AI for analyzing drone footage and satellite imagery.
Challenges of growth
At the beginning, the program struggled with the limitations of early computer vision technology, which required painstakingly annotating drone videos frame by frame to teach algorithms how to track vehicles across deserts.
Project Maven also faced political and media controversy in 2018 when Google employees objected to the company providing the Pentagon with Google’s open-source machine learning library for computer vision applications, which were being used to enhance drone strike targeting.
However, these initial challenges served as a catalyst, as Maven paved the way for the intelligence community’s adoption of AI over the following years. NGA is now working to incorporate machine learning and computer vision across its entire data analysis process.
Whitworth explained that with AI, analysts can uncover hidden threats and areas of interest within the enormous amounts of imagery and data agencies collect daily. He added that a new set of tools is enabling NGA to provide more timely and accurate intelligence.
Overflow of data
The government’s adoption of machine learning to extract insights from massive data streams has significant implications for the commercial remote sensing industry. Companies have invested billions of dollars in next-generation Earth observation satellite constellations that produce large amounts of imagery and geospatial data.
Whitworth stated that NGA is eager to take full advantage of the capabilities being developed by the private sector. The challenge is to be able to turn these massive data streams into usable intelligence.
"The sheer volume of geospatial data is staggering," Whitworth said, comparing it to attempting to send high-resolution photos from a mobile phone without a signal. "That challenge, multiplied a billionfold, is what we face on a national scale."
In addition to the practical uses of AI, he mentioned that the agency wants to make broader use of the technology, to monitor global events and provide policymakers with early warnings of potential flashpoints.
A new initiative at NGA is to seamlessly integrate AI into the daily workflow of analysts. These AI-powered systems would, for example, alert analysts to a potential threat emerging halfway across the globe, Whitworth said. "This is a much bigger opportunity that we’re really excited about."
The competition between the United States and China to use AI is really important, especially in the context of a war, according to a DoD intelligence official. The Chinese are racing to be the first to use AI in weapons, and the quality and accuracy of data is crucial for winning a war.
The Pentagon and China’s People’s Liberation Army believe that AI and machine learning are essential for quick data processing and fast decision-making, allowing them to go after a lot of targets quickly and effectively. This is how they see the future of warfare.
AI in war situations
Frontline combat units like the U.S. Army’s 18th Airborne Corps are now using AI tools from Project Maven, according to Trey Treadwell, NGA’s associate director for capabilities.
Using computer vision and detailed datasets, analysts can quickly examine satellite photos and drone videos to find specific targets like vehicles, aircraft, and military equipment. This gives commanders and intelligence analysts a clear operational picture, according to Treadwell at the 39th Space Symposium.
The agency sees AI as a way to enhance their capabilities, rather than letting AI or machines take over completely. For example, it can help in avoiding using expensive missiles to take down inexpensive drones.
The Maven project is about pushing boundaries and using cutting-edge technologies, according to the quote by an NGA official. Field operators are the driving force, challenging analysts to find new layers of information from the data.
Going beyond computer vision
NGA plans to make use of more AI technology beyond computer vision, according to Mark Munsell, NGA’s director of data and digital innovation. Computer vision is just a part of AI technology that allows computers to understand visual information from images and videos.
The agency plans to utilize newer foundation models developed by AI leaders OpenAI, Anthropic, Google, and Microsoft, and customize and train those models with their own data, according to Munsell.
In the future, analytics will rely on big computer vision models and large language models, and there will be a lot more collaboration between humans and machines, according to Munsell.
NGA launched the ASPEN project to integrate AI tools into analysts’ workflows, aiming to empower analysts to identify activities of adversaries that may require notifying the White House or the Pentagon.
AI models are quickly becoming able to process and understand data from various sources, including text descriptions and audio, offering a more comprehensive understanding of a situation, as explained by Munsell.
Munsell, who was NGA’s chief technology officer at the start of Project Maven, emphasized how the program’s realignment under NGA has sharpened their focus on combat support activities once more. It has clarified their vision of how they should be supporting combatant commands in operations.
Sean Batir, Project Maven’s chief technology officer, stated that intelligence analysts are struggling to meet commanders’ desire for information while streamlining workflows. “One way we address this is by attempting to condense the information,” he said.
“I’ve been working on combining some of the newest open-source large language models, and we’re actually finding a very positive result,” he said. “I think that the future involves multimodal models that can process text and imagery, and we’ve already started experimenting with that in our workflows.
Training data
The work done under Project Maven is moving to the broader ASPEN initiative, Munsell said. “We have analysts that look at imagery, annotate imagery all day long, and we’re incorporating that training data creation into the analytic workflow.
ASPEN program manager Rob Shields explained that while Project Maven is mainly focused on enhancing speed and accuracy in targeting on the battlefield, ASPEN aims to offer insights to “stay ahead of our adversaries.
“Our goal is really to integrate at scale the analytic tools and processes to include computer vision, big data analytics, geographic information, geospatial intelligence reporting, and imagery exploitation services,” he said.
Project Maven has long relied on contractors for “image labeling,” or the process of adding descriptive tags or annotations to images to help machines interpret the visual content and understand what’s happening in the image.
One of the goals of the ASPEN program is for government analysts to perform image labeling as part of their regular workflow, said Shields. “And as images are labeled, shared and used across the program, it’s all about having those labels enhance the accuracy of our models.
“We have to increase speed without compromising accuracy,” he said.
Predictive Intelligence
NGA’s vision is to “anticipate adversary action and potential outcomes through pattern of life analysis, patterns of behavior, and trends through data analytics, identifying anomalous and escalating indicators,” he said. “ASPEN covers all the workflow from detection through reporting.
While NGA has been using AI in various capacities for over two decades, Munsell notes that the rapid technological progress in recent years is enabling the agency to take these applications to a completely new level. “Transformers took off in the 2017 and 2018 era when Project Maven started,” he explained. “The availability of high-performance compute, cloud storage — all these innovations converging is what has really unlocked the AI’s potential.
Transformers are a type of neural network inspired by the structure and function of the human brain, essentially a way for computers to learn and process information the way humans do, albeit in a much simpler way.
The rise of foundation models such as Generative Pre-trained Transformers (GPT) signifies a significant change, he explained.
These models are linked to popular chatbots like OpenAI’s ChatGPT. He said, “But they are much more powerful than that. They are able to perform tasks, becoming increasingly capable of doing human work, accessing applications and data sources, writing their own code, and achieving results that typically require human effort.”
Considering the rapid advancement of technology and convergence of innovation trends, Munsell expressed, “I believe we are witnessing something highly significant for the intelligence community and the Department of Defense in the coming years. There will be major advancements in the way we generate intelligence.”
