Menu

The AI tools nobody approved are already inside the breach numbers

The AI tools nobody approved are already inside the breach numbers

In May 2023, Samsung told staff at one of its largest divisions to stop using generative AI tools after engineers pasted sensitive material into ChatGPT. Bloomberg’s report on the ban described a company worried about data transmitted to such services sitting on external servers. The episode was treated at the time as a cautionary tale about one firm. It was closer to a preview.

Two years on, unsanctioned AI use has a name, a cost and a line in the breach statistics. The governance question it raises has drawn sustained commentary, including John Margerison, CEO of XFactorAi, who examined shadow AI as a cyberattack risk. The argument I want to make here is narrower and more awkward for most boards. Shadow AI is not primarily a discipline problem among staff. It is the predictable output of a control model that assumed software arrives through procurement.

The ban was the policy, and the ban did not hold

Blocking was the first instinct of almost every large employer, and it produced compliance on paper. Usage moved to personal devices, personal accounts and browser tabs nobody logs.

Microsoft and LinkedIn’s 2024 Work Trend Index, based on a survey of 31,000 people across 31 countries, found that 78 per cent of people using AI at work were bringing their own tools. That figure describes a workforce that has already decided, independently of any policy cycle, that these tools are worth the risk of using them quietly.

The reason is structural. A spreadsheet macro saves an hour a month; a model that drafts a client memo saves an afternoon. When the personal payoff is that large and the enforcement mechanism is an all-staff email, the email loses. Any security function still treating prohibition as its primary control is managing the record of AI use rather than the use itself.

The cost has stopped being hypothetical

Quantification arrived in 2025. IBM’s Cost of a Data Breach Report, produced with the Ponemon Institute and based on 3,470 interviews across 600 breached organisations, found breaches involving employees’ unauthorised use of AI tools cost an average of $4.63 million, around 16 per cent above the global average of $4.44 million.

The more damning number concerns oversight rather than price. Network World’s account of the same report notes that 13 per cent of organisations reported breaches of AI models or applications, that 97 per cent of those compromised lacked proper access controls, and that among organisations with AI governance policies in place, only 34 per cent carried out regular audits to detect unauthorised AI use.

Read those together and the picture is a governance layer that exists on paper and almost never looks. A policy without an audit routine is a statement of intent.

The real loss is often reasoning, not records

Data exfiltration dominates the conversation because it fits an established category and an established remedy. It is also the easier half of the problem.

When an analyst runs a pricing scenario through a consumer chatbot and pastes the conclusion into a board pack, what leaves the organisation is one thing and what enters it is another. The output carries no provenance, no version of the model, no record of the prompt that shaped it. Six months later, when someone asks why the assumption was set where it was, there is no answer to retrieve.

That is an auditability failure, and it compounds silently. Regulated firms discover it during a supervisory review. Everyone else discovers it in litigation or in a post-incident reconstruction that cannot be completed.

You cannot scope an incident against an inventory you do not have

Incident response rests on a question answered quickly: what was touched. Shadow AI removes the ability to answer it.

If staff have been using a dozen free tools across personal accounts, the containment team has no list of destinations, no log of what was uploaded and no contractual route to ask the provider. The forensic timeline stops at the browser. That is one plausible reason unauthorised AI use carries a cost premium rather than merely a higher incidence: the investigation runs longer and ends less conclusively.

Breach notification obligations assume you can determine the categories of personal data involved and the number of people affected. An organisation that cannot enumerate where its data went is left choosing between over-notifying and guessing.

Sanctioned software is now the largest shadow AI surface

Here is where the usual remedy breaks down. Most AI governance programmes are built around a register of approved tools, which treats shadow AI as a set of rogue applications to be found and blocked.

The assumption is already out of date. AI features now arrive inside software that passed procurement years ago: summarisation in the meeting platform, drafting in the CRM, model-backed search across the document store. Nobody downloaded anything. A supplier shipped a release, and the data-processing behaviour of an approved system changed without a new purchase order or a fresh risk assessment.

Vendor inventories cannot catch that, because the vendor is on the list. Catching it requires watching contract change notices and feature releases with the same attention given to new purchases, which is work almost no organisation has staffed.

The firms that get ahead of this will stop asking which tools people are allowed to use and start asking which decisions are permitted to depend on a model whose behaviour nobody logged. That question survives the next product launch. The approved-tools list does not.

No comments

Leave a Reply

Most Shared Posts

Write For Us