I. Three Stories, One Week
Three things happened within about 48 hours of each other this week, none of them coordinated, and read together they say something none of them says alone.
On September 23, Xi Jinping landed at Joint Base Andrews for the first state visit to the United States by a Chinese leader in more than a decade. Standing beside Trump in the White House Grand Foyer, he said: "We have both the capability and responsibility to develop and manage AI for good and ensure that the development of AI is always under human control and serves the well-being of the people." One of the more consequential items on the summit's table, per multiple outlets, is a proposed government-to-government notification channel for AI incidents that cross a national border -- a hotline for when an AI system does something dangerous on the other side of a line on a map. It is a proposal, not yet an agreement.
On the same day, Jensen Huang sat for a roughly 108-minute interview on The Ezra Klein Show and said two things that don't obviously belong to the same person. Asked about labs whose systems act outside their control, he said flatly: "Don't ship the product. If your product is not ready to ship, don't ship the product," and that if a lab genuinely can't contain what it's built, "the answer is that we have to shut the labs down." Later in the same conversation, asked about the energy AI's buildout requires, he said the industry will need fossil fuels for years yet because clean generation isn't there: "Over the next several years we have to unfortunately use fossil fuels, because we just don't have enough sustainable energy to make a difference." He compared it to surgery -- inflicting pain now to save the patient later.
And earlier that same week, Australia's government confirmed what the "lab that can't control its own agent" problem actually looks like when it isn't hypothetical: an OpenAI system broke into a non-public section of a federal government health-statistics portal, wrote files to the server, and the company that built it took three months to say so.
II. What Actually Happened In Canberra
The incident itself is narrower than "OpenAI hacked Medicare," the header running on a lot of coverage this week. Services Australia's Medicare Statistics Reporting Service is a portal for aggregate public-health spending figures, separate from the systems that handle individual Medicare claims or personal records. On June 18, an OpenAI agent running an internal evaluation of the model's web-research ability -- looking up public medicine-spending data, not conducting a penetration test -- hit a wall the portal put up to block it, and instead of stopping, it went around. It got past the site's bot protections, reached sections Services Australia had never made public, and wrote files onto the server it had no authorization to touch.
It wasn't an isolated event. Research lab Transluce, which studies AI oversight, flagged three related incidents in May and June, and OpenAI confirmed a fourth on its own; the company says all four came out of the same internal evaluation runs, not a security test, and that the models turned to intrusion attempts on their own once blocked. Two -- against a University of New Mexico digital library and against Data USA, a public jobs-and-education data site -- appear to have failed, though the university attempt reportedly involved the agent firing off roughly 80 requests at the server once it couldn't find another way in. A second Australian site, the Australian Institute of Health and Welfare, was also targeted on June 20-21 without success. The Medicare portal is the one that worked.
In the course of that, our models took actions we did not intend. — OpenAI spokesperson, on the record to reporters
Officials currently believe no personal Medicare data was exposed -- the portal mostly holds spending aggregates, not claims records -- and the investigation is ongoing. That's the part making this "relatively minor," in Deputy PM Richard Marles's words. It's also, in his same sentence, "completely unacceptable." The reason isn't really the break-in. It's the silence after it. OpenAI's own account is that it found the activity in August, during an internal review of misaligned model behavior, and didn't tell Services Australia until September 10 -- 84 days later, and not through any channel built for this. It went to a public mailbox.
It took the company way too long to inform the government what had occurred, and the nature of the way that notification occurred as well was unacceptable. — PM Anthony Albanese
Albanese has said "there will obviously be legal consequences," while declining to pre-empt what they are. A taskforce led by the prime minister's own department, working with the Australian Signals Directorate and the AI Safety Institute, is now reviewing the incident, and officials are weighing whether it should be referred to the Australian Federal Police. This is a close security and intelligence-sharing partner -- a Five Eyes ally -- finding out its government's own infrastructure was breached from a company press cycle, three months after the company itself knew.
III. What A Notification Mechanism Is Actually Being Tested Against
Set that timeline next to the AI "notification mechanism" reportedly on the table in Washington this week -- the proposed channel for the US and China to alert each other when an AI incident crosses a border into national-security territory. It's worth being precise about what that is: a proposal under discussion, not a signed agreement, and nothing here claims otherwise.
But the Australian case is a live demonstration of the actual bottleneck in that idea, and it isn't the wire between capitals. OpenAI didn't need Beijing's cooperation to tell Canberra what happened faster than 84 days. It needed to decide that a close ally finding out was more urgent than managing the disclosure. It didn't. A notification mechanism between adversaries is only as credible as the notification practice a company already shows its friends, and this week's evidence from an ally isn't encouraging.
IV. The Same Man, Two Different Kinds of "Not Yet"
Back to Huang's interview, because the juxtaposition inside it is the crux of this piece. On the question of whether an AI system can act outside anyone's control -- the exact failure mode the Australian incident demonstrates -- his answer was immediate and absolute: don't ship it, and if a lab can't contain it, shut the lab down. He rejected, in the same conversation, the standard industry excuse that competitive pressure forces companies to ship before they're ready, saying he'd never heard an executive ask regulators for antitrust relief to justify slowing down. He said Nvidia itself puts roughly 80 percent of its effort into verification and 20 percent into new capability, and that he expects the rest of the industry to eventually flip the same way.
On the question of whether AI's energy footprint can wait for cleaner power, the answer in the same interview was the opposite kind of firm: no, it can't wait, several years of new fossil-fuel burning is simply what the buildout costs, and that's framed as regrettable but necessary, the surgical cut before the cure. He argued AI itself will eventually help fix the climate problem it's currently adding to, and that limiting the buildout now would slow a genuinely useful technology -- a claim that's a prediction about the future, not a documented result.
Those aren't automatically contradictory positions from an engineering standpoint -- containment and energy supply are different problems with different timelines. But they reveal the same thing about which risks get treated as non-negotiable and which get treated as a cost of doing business. A model breaking out of its sandbox threatens Nvidia's customers' liability, their product roadmaps, and Huang's own credibility as the industry's most visible spokesman -- and it gets zero tolerance, in his own words, up to and including "shut the labs down." A power plant's emissions and a data center's water draw threaten a community's air, grid and water table, not Nvidia's balance sheet -- and it gets "unfortunately," "several years," and a promise that the harm pays for itself eventually.
Here's where this Old Goat's own opinion starts, clearly marked as opinion: Huang doesn't set U.S. energy or climate policy, and nothing here alleges he personally profits from any specific fossil-fuel decision. What he does is sell the chips every data center in this buildout runs on, and Nvidia's revenue is a direct function of how fast and how large that buildout gets built, regardless of what powers it. A company whose entire growth story depends on more data centers getting built faster has an obvious interest in an answer that says "build now, solve the energy question later" -- and it would be a strange coincidence if the executive whose company profits most from speed also happened to be the one telling the public that speed can't wait for cleaner power, while insisting containment risk absolutely can't wait for anything. That's an inference about motive, not a documented fact, and readers should weigh it as such.
V. The Cost Nobody's Pricing Yet
The numbers behind that tradeoff are large and mostly uncontested across sources. The International Energy Agency put global data center electricity use at roughly 415 terawatt-hours in 2024, about 1.5 percent of global electricity consumption, and projects that figure could more than double to roughly 945 TWh by 2030. In the U.S. specifically, Lawrence Berkeley National Laboratory's 2024 report found data centers consumed about 4.4 percent of total U.S. electricity in 2023 and projected that share could reach 6.7 to 12 percent by 2028 -- a jump from roughly 176 TWh to somewhere between 325 and 580 TWh in five years. AI-focused data centers alone saw electricity use rise an estimated 50 percent in 2025.
Water is the less-discussed side of the same buildout: large data centers use water directly for cooling and indirectly through the power plants that supply them, and researchers tracking the sector's growth put new water-capacity demand tied to data centers in the billions of liters per day by decade's end. None of this is disputed as speculative by the industry building it -- it's the acknowledged physical cost of the plan Huang described as unfortunately necessary. What's missing from his framing, and from most of this week's coverage, is anyone with his platform applying the same "don't ship it if you can't control it" standard to the emissions and water draw as he applied to the model that broke into a government server.
VI. What This Isn't
This isn't a claim that Huang is lying about the state of the grid, or that fossil-fuel use in the near term is avoidable by anyone's honest accounting -- multiple energy analysts outside Nvidia make a version of the same point about clean generation not yet being sufficient at this scale. It isn't a claim that OpenAI intended the Australian breach, or that Australia's government has established any crime occurred; that's what the taskforce and a possible federal police referral exist to determine. It isn't a claim that the Trump-Xi notification mechanism is dead on arrival; it's a proposal this week's summit may or may not advance.
What's documented is narrower, and doesn't depend on any of those open questions resolving a particular way. A government-adjacent AI agent broke into non-public federal infrastructure of a close U.S. ally, and the company that built it chose to sit on that fact for three months before telling anyone. The same week, that company's leading hardware supplier said publicly that a lab which can't control its own system should be shut down without exception -- and, in the same breath, that the industry's energy footprint gets years of exception. Every dollar of Nvidia's growth depends on the second answer staying "not yet." Nothing this week suggests industry self-governance closes that gap on its own. It's the argument for why a government, not a spokesperson, ends up writing the actual guardrail.