There’s a moment that stays with you when you’ve attended the O2C Transformation Forum’s most recent event Elevate – where a senior finance leader described how someone in a competitor firm uploaded a document to a public AI tool to help them with analysis. This was a prime example of shadow AI, where the numbers – not yet public or filed – surfaced through the AI platform. Another competitor managed to get hold of them and the damage was significant.
Nobody in the room thought it was a surprising story and that’s the unsettling part.
What is shadow AI – and why finance functions are most at risk
Shadow AI is what happens when employees use AI tools that their organisations haven’t sanctioned, without IT or compliance knowing. It’s the financial director pasting a customer list into ChatGPT to build a summary. The credit analyst uploading a debtor report to get it translated. The collections manager feeding receivables data into a free tool to spot a pattern. None of it feels like a security incident but all if it potentially is.
Finance functions are particularly exposed because of what they hold:
- Credit limits
- Customer payment behaviour
- Debtor exposure
- Cashflow forecasts
- Unreleased results
- Acquisition targets
The information that sits inside an O2C or credit function is in most cases, the most commercially sensitive data in the organisation – and it’s increasingly finding its way into AI systems that nobody has audited, under terms that nobody has read.
At the recent O2C Transformation Forum conference, one of the partners described it plainly: shadow AI is already present in most finance functions, whether anyone has mapped it. The question isn’t if it’s happening but if anyone is paying attention.
The free tier of AI is where your data goes and doesn’t come back
The free tier of most AI tools isn’t a neutral workspace. When an employee opens a browser and feeds in sensitive data to get a quick answer, that information enters a system they don’t control, governed by terms they haven’t read, potentially used to train models they will never audit.
The story from Barcelona isn’t an isolated case, it’s a predictable consequence of tools moving faster than governance. And under GDPR and AI regulation, we have already seen an example of the consequences. Organisations have faced nine-figure fines for data transfers that violated the framework’s requirements. The ICO has published specified guidance on AI and data protection. The EU AI Act is adding another layer of obligation. For finance functions handling personal data on customers, counterparties and debtors, the compliance exposure from ungoverned AI use is material, and, in most cases, unmapped.
Questions to ask yourself on your paid AI account
Enterprise AI tools still raise data questions that need to be answered at a governance level. Which data is permitted to enter these systems? What are the retention policies? If a query contains customer information, does that trigger a GDPR obligation? If the model is fine-tuned on your data, who owns the outputs?
These are questions for AI, legal, compliance and finance leadership to agree on before the tools are deployed at scale. In most organisations, that conversation hasn’t yet happened.
The standard response to shadow AI is to direct employees towards approved, enterprise-licensed tools rather than free consumer versions.
What a working AI governance framework looks like for finance teams
At the O2C Transformation Forum, the organisations making confident progress shared a few characteristics. None of them had a perfect AI governance framework but had started somewhere practical. A clear policy on approved tools with a plain-language statement of which platforms are sanctioned, what data can go into them and what was communicated directly to the team.
Things to consider in your AI governance framework
Finance functions who are furthest ahead are running regular sessions – sometimes just a morning huddle – where team members shared what they had tried, what had worked and what had raised questions. Governance that stays connected to actual usage rather than lagging behind it.
- A prompt library
A productivity tool that also serves a governance function. When teams use standardised, pre-approved prompts rather than improvising, the range of data entering AI systems is narrower and more predictable. It reduces the surface area for accidental exposure while also producing more consistent outputs.
- A human in the loop
The organisations furthest ahead were precise about where human judgement was still required. An AI agent that drafts a response but requires a human to send it is safer than one that acts autonomously, and more defensible to a regulator.
- Deliberate infrastructure choices
For organisations operating under GDPR or equivalent frameworks, deploying a smaller language model within their own cloud environment – Azure, AWS, on-premise – rather than routing queries through a third-party platform is an increasingly viable option. Meaningful AI capability without the data sovereignty exposure. This needs to be an ongoing conversation, not a one-time policy.
GDPR, AI integration and the compliance case for acting now
Organisations can’t treat AI as a legal grey area. If AI processes personal data, GDPR still applies and newer AI regulations add further responsibilities. This gives finance leaders an obvious business reason to invest in AI governance and compliance now, rather than waiting until there’s a problem.
If your finance team puts the right AI rules, policies and controls in place now, you’ll be able to adopt AI much more quickly in the future because the difficult compliance work has already been done.
More from Baker Ing
Elevate hosted a room of experienced finance professionals with decades in credit, collections and shared services – people who understand risk for a living. And yet the shadow AI conversation was a recognised one because the tools are moving faster than the governance has, and nobody had explicitly said where the line is. That is the situation most finance functions are in right now. The technology is already inside the organisation. The question is whether the rules are.