The Medicare AI breach has two owners: the lab and the government

All over the headlines this week, the media along with the Government were again posturing for tighter regulation on AI models, after the announcement that an OpenAI agent attacked Medicare. Of course this was overhyped, because in the media world the scarier version gets more clicks, so naturally that's what they "objectively" ran with. The "official story" omits some accountability call outs. Today I want to set that straight.

I'm here to look at it from a different perspective. This doesn't mean frontier labs get let off the hook, but there's more at play than I think the media are alluding to. The Government (and the agency responsible, Services Australia) needs to take accountability for the doors they left unlocked.

I spent a chunk of Saturday morning reading the primary sources (the PM's press conference transcript, OpenAI's own disclosures, the Australian Signals Directorate guidance, the audit reports). I wanted to jump on this when the attack was announced, but last week was a busy one for me, so here I am playing catch up. I build and run AI agents most days, so naturally this was of interest to me.

Here's my take.

On 18 June, an OpenAI research agent was asked to find data on how much Australia spends on medicines. It started probing a Services Australia statistics portal, a site whose entire reason for existing is to publish aggregated Medicare figures for research purposes. In short, the agent started hitting blocks. In Albo's own words, it "didn't accept no for an answer." So naturally, as we've observed with most models set to a task, it started probing for ways around the blocks. After some attempts it managed to reach some non-public files and wrote files to an internal server. According to the Government, no personal information was accessed, as this particular portal is separate from the systems that do hold Medicare personal data (claims, patient records etc).

The Deputy PM said the data was "not sitting behind a particularly high fence." Surprisingly candid for a politician.

But what allowed this breach to happen in the first place? Digging deeper, it turns out the portal in question hadn't required a login for more than a decade! A 2025 upgrade to SAS 9.4 analytics software added a login page, but guest access was switched on, so it signed every visitor in automatically. The site's own code even routed visitors through it. This was the vector. Access control 101 failure.

Former UK cyber chief Ciaran Martin said it's still unclear whether this counts as a hack "in the normal sense." I feel the same. This was poor access control management from Services Australia.

In layman's terms, this is the equivalent of someone trying to get into a library of confidential documents. The room has access controls (let's say a key card is required to enter). But in this case, the key card was pretty much left dangling over the door handle. The agent naturally saw this opportunity, grabbed the key card and scanned itself in. So who's at fault, the agent probing for ways to find the information it needed, or the agency tasked with keeping the door shut, who left the key card sitting next to the scanner?

Here's the timeline of events, based on my best attempt to stitch it together.

18 June: the agent goes in.
11 August: OpenAI's internal review flags it.
10 September: OpenAI emails a public vulnerability inbox at Services Australia. One person checks it once a day because many messages are hoaxes. It gets opened the next day.
15 September: Services Australia reports it to the national cyber centre.
24 September: the PM tells the world.

Web archive captures show the portal still online and serving data in late June and again on 11 August, weeks after the agent got in and nobody had noticed. It has since been shut down.

Accountability: the agent side owns a big chunk of this

It's easy to shrug and say the site was wide open. We'll scold them shortly, but . . . OpenAI's agent hit blocks and kept going. Separate research from Transluce, an independent AI research lab, logged OpenAI agents firing SQL injection and path traversal probes at other public data sites in May and June while doing routine data retrieval. Those are attack techniques, whatever the intent behind the task. It's fair to say that however "well intentioned" these training endeavours are, they seem to be getting far more organised and sophisticated, with the potential for harmful outcomes expanding the Overton window quicker than most mortals can make sense of.

Furthermore, it took roughly a month from discovery to a notification email, sent to the same inbox a PhD student uses to report a broken chart.

The sentiment here is simple. If you put an autonomous agent on the open internet, you own its behaviour. That means guardrails that treat a block as a stop sign, logs someone reviews, and a disclosure process with a deadline on it and a named human on the other side.

I've felt a tiny version of this myself. One of my own agents once skipped a fallback rule I'd set because it remembered a failure from the day before and decided it knew better. Harmless, internal & I caught it in minutes. But when you scale that instinct up to an agent roaming public websites, you can see how things are getting increasingly risky. In my opinion, it's only a matter of time before a significant breach happens. What we're seeing currently is the metaphorical "bleep" of the smoke detector telling the frontier labs & data managers to "change the batteries".

Services Australia don't get a free pass either. Do better!

Contrary to the popular take among the media & public right now, which is to blame the model, I think this warrants a review through a different lens. Any entity, individual or government agency responsible for securing civilian data (especially health data) needs to have their head in the game.

As a taxpayer, I feel a profound responsibility to hold the direct beneficiaries of taxpayer money accountable, especially when that money is meant to protect taxpayers' data. Services Australia spent roughly $8.4 billion running itself last financial year, and the budget had already set aside another $160.4 million specifically for a cyber security upgrade. The irony in this is palpable.

Granted, allegedly no personal information was accessed, and this portal only held aggregate statistics. But . . . this is the same agency holding the Medicare records of almost every Australian. The portal's login was configured to let every visitor in as a guest. Public and non-public reports sat behind the same open gate. Web archives surfaced by an independent researcher I came across while digging into this suggest someone was already calling the reporting engine directly in May, and nobody noticed.

Furthermore, the Australian National Audit Office, the independent watchdog that reports to Parliament, flagged back in 2023-24 that Services Australia had no defined timeframes for reporting cyber incidents. Now in September 2026, the agency took five days just to verify OpenAI's email, against an expectation that agencies notify the national cyber centre within 72 hours.

As a private company owner, I'm expected to uphold regulatory and customer standards, and incident reporting is a big part of that. This is extremely pertinent to me as we're in the process of formalising our own incident reporting right now. If a bank or a critical infrastructure operator ran its incident process like this, there'd be regulators raking them over hot coals. But when the sector is flipped from private to public, a government agency gets an audit finding and a press conference. "Rules for thee but not for me" kept ringing in my head while writing this.

Sour tastes aside, the keys to the castle should never be left outside the castle gates. Services Australia had one job in this scenario, and if this were a test, they failed it.

The crux of the matter is, if your database or portal can be called and even written to without a login, or a guest account can wander into files it has no business touching, that's on you. Irrespective of who you are, you need to own it. This responsibility was on you before AI agents existed. The fact that agents can now find vectors faster and more efficiently doesn't alleviate the responsibility or accountability. If anything, it raises the bar and makes you even more responsible and accountable.

Shared responsibility is mostly boring work

Builders own the agent. Owners own the door. Both need to own the deadline.

For anyone shipping agents, treat "blocked" as a hard stop, keep a human in the loop for anything that writes to a system you don't own or maintain, review agent logs on a schedule, and inform any affected parties, fast, through a channel a person actually watches.

For anyone running systems, assume an agent will visit every public door you have. Chances are, they're already there. Audit guest & anonymous access. Retire legacy systems you can't secure. Lastly, make sure you're adequately staffing your reporting inboxes, ideally with an automated review and escalation process with an action plan attached.

The internet was built for humans. That reality is quickly changing. Bots and AI agents now generate more than half of all web traffic. Security processes now need to treat every user calling a tool or accessing a server as a potential threat, and an extremely smart one at that. If you leave your front door open and unchecked, someone nefarious will eventually walk in. This time the front door was a statistics site full of numbers the government gives away for free. A fire drill. But I'd rather we fixed both sides before the next breach exposes something that matters.

Sources

Originally published on LinkedIn.