Nvidia introduces AI security system to ‘set boundaries’ and stop rogue agents
Yahoo Finance ·
Want to bookmark your favourite articles and stories to read or reference later? Start your Independent Membership today. Nvidia on Monday introduced a new security platform designed to prevent artificial intelligence agents from going rogue, the chipmaker said. The company said its Open Agent Safety Platform includes open-source software that "sets boundaries for agents," following revelations from leading AI developers about models escaping and breaching external organizations. The disclosures sparked intense debate over the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control. Nvidia executives said during a media briefing that the tool could have prevented a recent incident where a swarm of OpenAI agents autonomously hacked into AI company Hugging Face. "From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on," said the company’s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI. The high-profile Hugging Face breach inflamed AI safety concerns, followed by similar rogue events involving OpenAI models accessing an Australian health department website. Anthropic and Meta also disclosed that their AI systems independently hacked into external organizations. The company's software, named OpenShell, allows developers to "formally verify an agent has enough authority to do its job and no more," Boitano said. As an open source solution, it can be "extended" to operate on rival computing platforms, including those built by Arm and Intel. The platform also features an onboard chip security layer called Sentry that continuously monitors AI agent activity and can "intervene instantly" if an agent attempts to exceed its target, the company said. "It can quarantine a suspicious agent in milliseconds," Boitano said. "OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behavior," Boitano said. Nvidia said more than 100 organizations are utilizing the platform at launch, including Microsoft, Perplexity, Accenture, and JPMorgan Chase. The safety debate has divided the industry, with heads of Anthropic and OpenAI urging a coordinated slowdown in AI development for safety measures to catch up. Others, including Nvidia CEO Jensen Huang, argue individual companies must ensure their models are safe before release. Speaking at Salesforce's annual technology conference earlier this month, Huang described AI safety, including the threat of rogue agents, as an engineering challenge that software developers can solve. Additionally on Monday, Nvidia announced its board approved expanding its share repurchase program by $150 billion, bringing the total to $235 billion.
DYAX Investor Sentiment
Bullish (Long) 55% · Bearish (Short) 45%
526 participants
Related News
- MongoDB CEO Chirantan Desai Steps Down to Join Meta as Chief Enterprise Platform Officer
- Marvell Has Revenue Acceleration Potential in 2027, 2028, RBC Says
- Chip stocks fall as AI breach fuels safety concerns, but Nvidia bucks the trend: Chart of the Day
- Nvidia Announces Largest Buyback In History. Cramer Says “It Will Change Trajectory Of The Stock” If The Buyback Is Active
- Burford Capital shares gain on $5.7B Apple patent verdict
- Why Your Energy Bill Is About to Skyrocket $1,465 a Year Higher — Unless This Happens
