AI development needs high-performance computing to train new models, but AI’s unique characteristics change the equation on security.
By: Dave Dimlich
President of SD3IT
As artificial intelligence models grow in complexity, large-scale training increasingly depends on high-performance computing (HPC) environments that combine dense GPU compute, high-speed networking, massive datasets, specialized storage, automated data pipelines and multiple users and workloads operating simultaneously.
The systems supporting those AI workloads have become what Nvidia and others call AI factories: highly specialized environments that turn data into trained models, delivering inference and, ultimately, intelligence. HPC provides the horsepower that makes much of this possible by distributing enormous computational workloads across hundreds or thousands of GPUs, which allows organizations to train models and process datasets at a scale that traditional infrastructure cannot match.
But the other side of the coin—and with AI, there’s always another side of the coin—is security. Specifically, HPC needs a cybersecurity architecture that addresses the challenges introduced by AI. And those challenges are significantly different from those traditionally associated with HPC.
HPC Is Essential to Training Frontier AI Models
More than just processing data at high speeds, AI models are trained to analyze data, recognize patterns, predict what’s likely to happen next and steadily “learn” along the way, improving accuracy and eliminating mistakes on a regular basis.
That’s what makes HPC essential to training AI models. It provides parallel computing on a huge scale, using super-fast, specialized GPUs that serve enormous datasets across multiple nodes simultaneously. It uses schedulers such as Slurm or Kubernetes to allocate resources to keep GPUs productive rather than sitting idle. Distributed storage systems can manage petabytes of data and also help increase efficiency. All this capacity enables AI systems to capture feedback and enhance performance.
Security, meanwhile, has to operate inside that environment without creating a bottleneck, particularly when the HPC environment becomes an AI factory. A security framework must protect AI models and sensitive data, while reaping the full benefits of HPC.
The national effort toward promoting AI recognizes that performance and cybersecurity need to find a balance, and that the key is in the architecture. The National Institute of Standards and Technology’s SP-800-234, its High-Performance Computing Security Overlay, addresses the infrastructure needs for large, complex simulations, including the training of AI and machine learning (ML). Its release, building on the moderate baseline defined in NIST SP 800-53B, underscores the difference AI brings to HPC computing models, with its high requirements for complex operations and speed.
The Assets Are Different. So Are the Threats.
In addition to its enormous processing requirements, AI can also present other security challenges. An AI factory contains extraordinarily valuable assets. Model weights typically contain key intellectual property. Training datasets can contain sensitive, proprietary or mission-critical information. GPU firmware and the infrastructure supporting the training pipeline also can become targets. And the AI pipeline itself is continuously ingesting data, training models and refining parameters, all at machine speed.
And there is also the element of the unknown. Throughout HPC’s history, even the most powerful supercomputers could always be counted on to stay in their lanes, behaving predictably according to a set of rules to achieve their assigned tasks. But new AI models, which are specifically designed to think through problems and take complex actions on their own, shatter that mold. There’s also a concern over their newly emerging ability to break free of sandbox containments, going out into the wild without human supervision. Overall, AI’s less-than-predictable nature means that security teams can’t solely rely on known signatures or assumptions about what an application is supposed to do, but must increasingly implement runtime security to ensure compliance and safety.
The environment also creates several other security challenges. For one thing, conventional security agents can compete for the same compute resources needed by the workload, creating additional overhead that can become significant when multiplied across a large cluster. Security cannot be allowed to starve the mission of the resources it was deployed to protect, but it also can’t be denied the ability to perform its function. For another, the entire environment needs to be protected because targets are not limited to conventional endpoints. Attackers may be interested in the training data, model weights, firmware, orchestration systems or the pipelines connecting all of them.
Infrastructure as Part of an AI Strategy
For organizations incorporating AI, the security architecture must align with the workload as well as the infrastructure. That means building security into the architecture from the beginning, including zero trust principles.
If organizations build dense GPU environments first and attempt to bolt security on afterward, competing priorities can quickly emerge. If you want security teams to have visibility and control, infrastructure teams to have performance, and your data scientists to have full access, the answer is to architect them together.
Treating security compute and monitoring functions as part of the HPC infrastructure can help prevent security functions from competing directly with production workloads while protecting data and models as they are being processed. Confidential computing and hardware-based trusted execution environments can provide another layer of protection for sensitive information during training and inference.
Segmentation, a critical element of zero trust, is also important. An AI factory may support multiple users, teams, models and workloads in the same physical cluster, but that doesn’t mean they can share an open environment. Infrastructure-level controls and microsegmentation can help isolate workloads and prevent unauthorized systems or shadow AI deployments from gaining access to sensitive resources.
Identity and access controls likewise must follow the same principle. The question is not simply who can access the cluster, but who can access a particular dataset, model, GPU resource or stage of the AI pipeline, and under what conditions.
Integration Is at the Core of HPC/AI Architecture
The AI factory is becoming the production floor for a new generation of computing, with HPC helping organizations address the computational demands of modern AI. And as AI models become larger and more capable, and more organizations move from experimentation to operational deployment, the infrastructure to both support and protect AI development becomes increasingly important. So does securing it.
SD3IT’s philosophy is that AI infrastructure is a matter of integrated technology, with dense GPU deployments ultimately working together with compute, networking, storage, power, cooling, virtualization and security. For federal and other mission-critical organizations, the stakes are even higher. An AI system that performs exceptionally well but cannot protect its data, maintain availability or withstand an attack is not a successful deployment.
SD3IT designs and delivers high-performance computing and dense GPU infrastructure tailored for modern AI workloads and resilient federal operations. Working with our stable of highly capable partners, our approach brings together compute, storage, networking, power and cooling, plus cybersecurity and lifecycle support to create infrastructure that is built for the mission.
The future of AI will depend on more than bigger models and faster GPUs. It will require infrastructure that can deliver performance, security and resilience at the same time. For AI factories and other mission-critical environments, those capabilities have to be designed to work together from the beginning.
Because the goal isn’t simply to build the fastest AI factory. It’s to build one the mission can trust.
About SD3IT
At SD3IT, we help federal agencies, defense organizations and commercial enterprises build secure, resilient technology environments that accelerate mission success. From AI-ready infrastructure and zero trust architectures to edge computing, data center modernization and systems integration, we design, drive and deliver solutions that help customers operate with confidence across today’s increasingly complex mission environments.