More than 100 technology, cybersecurity and financial organisations are warning that the AI threat – AI enabled cyberattacks – could become significantly more capable. The question is whether businesses are ready to change with it.
In September 2000, three people from a relatively small internet company travelled to Dallas to meet the CEO of one of the biggest companies in America.
They had a proposition: Blockbuster could acquire Netflix for $50 million.
Netflix would take responsibility for the emerging online operation while Blockbuster continued doing what it did best: running its thousands of video rental stores.
At the time, the proposal probably sounded faintly ridiculous. Blockbuster was a hugely successful business. Netflix was a small, loss-making DVD-by-mail company. The internet was certainly changing things, but nobody knew quite how quickly or how completely that change would happen.
According to Netflix co-founder Marc Randolph’s account of the meeting, Blockbuster CEO John Antioco struggled not to laugh when Reed Hastings named the $50 million valuation.
Blockbuster declined and ten years later, filed for bankruptcy.
Netflix had transformed itself from a DVD-by-mail service into one of the world’s most powerful entertainment businesses. The lesson isn’t that Blockbuster should somehow have predicted Netflix.
It is that the environment was changing, the assumptions behind its hugely successful business model were changing with it, and Blockbuster wasn’t ready to change with it.
And that raises an interesting question for AI and our current thinking around cybersecurity. Are we ready for the change?
The AI threat warning comes from an unusual group
This week, more than 100 organisations from across technology, cybersecurity, financial services and critical infrastructure signed an open letter calling for what they describe as a global surge in cyber defence.
The list of signatories is striking. OpenAI and Anthropic are there, alongside Google, Microsoft, AWS, IBM, Oracle and Adobe. Cybersecurity companies including CrowdStrike, Cloudflare, Palo Alto Networks, Fortinet, Check Point, Darktrace and Zscaler have signed. So have financial organisations including Mastercard, Visa, Capital One, Citi and Nationwide Building Society.
These aren’t companies standing on the sidelines watching AI develop. They are building it, securing it, deploying it and relying on it. And their message is blunt: “The status quo won’t be enough.”
The letter, “A call for collective action on cyber defense”, warns that the AI threat – AI enabled cyberattacks – will become more widespread and sophisticated in the coming months as AI models become increasingly capable.
It calls for a global surge in cyber defence, particularly for critical infrastructure such as hospitals, water utilities and the systems that underpin the internet.
The signatories want organisations to fix weaknesses that have accumulated over years while giving defenders access to capable AI tools of their own.
And there is a striking sense of urgency running through the letter: “We have a limited window to strengthen cyber defences.”
That isn’t a warning coming from AI sceptics. It is coming from many of the companies building the technology.
What are they actually warning us about?
It would be easy to interpret the story as another warning about AI becoming dangerous. That isn’t really what the letter is saying. The concern is much more practical.
AI has the potential to accelerate many of the activities that attackers already undertake. Reconnaissance. Vulnerability discovery. Social engineering. Credential attacks. Code generation. Data analysis. And potentially the coordination of multiple stages of an attack.
What previously required significant expertise, time and manpower could increasingly be automated. That changes the economics of cybercrime.
An attacker doesn’t necessarily need a completely new technique. They may simply be able to find and exploit existing weaknesses much faster. And those weaknesses are already there.
Unpatched software. Weak authentication. Excessive permissions. Misconfigured systems. Legacy technology. Insecure applications. Third-party dependencies. Technical debt.
The open letter specifically highlights these long-standing weaknesses and the historic under-resourcing of security teams, particularly within critical infrastructure.
AI doesn’t necessarily have to invent a new way into an organisation. It may simply make it considerably faster and cheaper to find the doors we’ve already left unlocked.
The challenge may therefore be less about defending against a futuristic AI threat – the AI super-hacker – and more about making sure yesterday’s vulnerabilities cannot be exploited at tomorrow’s speed.
The people creating the technology are part of the warning. There is something else about this story that is difficult to ignore. Some of the organisations warning us are the very organisations developing the AI capabilities that could make these attacks possible.
OpenAI and Anthropic aren’t calling for AI development to stop. Instead, the letter calls for defensive AI to be put into the hands of more organisations, particularly those protecting critical infrastructure and operating with limited resources. It calls on governments to provide access to defensive capabilities and authorised testing. It asks technology companies to provide funding, training and practical support. It also calls for organisations to continuously test their systems against increasingly capable AI, share threat intelligence and distribute proven fixes.
In essence, the message is: AI is coming. Make sure the defenders have it too.
That is a rather different proposition from the usual debate about whether AI is good or bad. It is about recognising that the technology is changing the balance between attackers and defenders, and making sure defence doesn’t fall behind.
A lesson from Oppenheimer
There is an interesting historical parallel here, although the comparison needs to be handled carefully.
In 1945, J. Robert Oppenheimer and the scientists of the Manhattan Project succeeded in creating a technology that fundamentally changed the strategic landscape. The atomic bomb is obviously not an equivalent to artificial intelligence. AI is a general-purpose technology with enormous potential to benefit society. It is not, in itself, a weapon.
But Oppenheimer’s subsequent concerns provide a useful lesson.
Having helped create something extraordinarily powerful, he became increasingly concerned about what its existence meant for the world. He argued for international control of nuclear technology and became increasingly opposed to the development of even more destructive weapons.
The point isn’t that AI is another atomic bomb. The point is that technological breakthroughs can create consequences that extend far beyond the original purpose of the technology.
Sometimes the people closest to those breakthroughs are among the first to recognise that society needs to think about what happens next.
Geoffrey Hinton’s warning
Geoffrey Hinton provides a more direct modern example.
Often described as one of the “godfathers” of modern AI, Hinton’s work on neural networks helped establish some of the foundations of modern AI. Yet in 2023, he left Google so that he could speak more freely about his concerns over increasingly capable AI. Hinton’s concern has never simply been that AI is “bad”.
It has been about the speed of development and the possibility that AI capabilities could eventually exceed our ability to understand or control their consequences. He has repeatedly argued that uncertainty should not be a reason for inaction. And that makes this warning particularly interesting. Hinton was asking us to think about the consequences of increasingly capable AI.
Now many of the organisations actually developing those capabilities are telling us that one of those consequences is already beginning to emerge in a very specific area: cybersecurity.
The conversation has moved from “what might AI eventually be capable of?” to “how do we defend ourselves against what it is already beginning to enable?”
So, are we having our Blockbuster moment?
This is where the comparison becomes useful. Blockbuster’s failure wasn’t because its executives couldn’t understand the concept of online video. The world simply changed faster than the business was prepared to change with it.
The same question now applies to cybersecurity. For years, organisations have built security strategies around a relatively familiar model of attack. Human attackers conduct reconnaissance. Human attackers identify vulnerabilities. Human attackers create phishing campaigns, steal credentials, move through networks and work out how systems fit together.
AI doesn’t eliminate those activities but it could increasingly automate and accelerate them. And that potentially changes the threat landscape faster than traditional security models can adapt.
The question isn’t: “Can we predict exactly what an AI-powered cyberattack will look like?” We can’t.
The better question is: “Are we prepared for attackers who can operate faster, at greater scale and with capabilities that are becoming increasingly accessible?”
That is a much more answerable question.
What can businesses do now?
The first step isn’t necessarily to buy more technology, it is to understand what you already have.
Organisations need a clear view of their infrastructure, their critical systems, their data, their dependencies and the ways those systems can be accessed. You cannot protect what you cannot see. That means identifying and prioritising the vulnerabilities that represent the greatest risk. Patch what can be patched. Where critical systems cannot be taken offline, put appropriate compensating controls in place and verify that they actually work.
Identity and access deserve particular attention.
Strong authentication, least-privilege access and properly controlled privileged accounts become increasingly important when attackers can potentially automate the process of finding and exploiting compromised credentials.
Organisations also need to consider speed.
If an attacker can use AI to accelerate reconnaissance and intrusion, security teams cannot afford to rely entirely on slow, manual processes. Monitoring, detection and incident response need to be capable of identifying unusual behaviour and containing threats quickly.
Testing matters too.
It isn’t enough to assume that security controls work because they have been installed. Organisations should regularly test their defences, conduct vulnerability assessments and penetration testing, and increasingly consider how their systems stand up against AI-assisted attack techniques.
Businesses also need to understand their own use of AI.
AI is already appearing in development environments, productivity tools, customer services and business applications. Organisations need to know what these systems can access, what data they are handling and what permissions they have.
The same applies to AI-generated code. If AI can produce software faster, organisations need to ensure that security testing keeps pace with development.
And finally, businesses need to plan for the possibility that prevention will fail. Good resilience means having protected backups, tested recovery procedures, appropriate segregation of critical systems and the ability to restore essential services without relying on an environment that may itself have been compromised.
Security is about preventing the attack. Resilience is about surviving it.
We don’t need to predict the future
There is a temptation with AI to focus on the biggest questions.
- Will it become autonomous?
- Will it outperform humans?
- Will it discover vulnerabilities nobody has found?
- Will it eventually become impossible to control?
Those are important questions for governments, researchers and the technology industry, but businesses have a more immediate question to answer.
Are our cyber defences good enough for a world in which the attacker has AI on their side?
More than 100 organisations, including many of the companies developing that AI, are telling us that the answer may currently be no.
Oppenheimer’s experience reminds us that technological breakthroughs can have consequences far beyond their original purpose. Geoffrey Hinton reminds us that even those who understand the technology best may not be certain where it ultimately leads.
And the Blockbuster story offers perhaps the simplest business lesson of all. You don’t necessarily need to predict the future. You need to recognise when the world around you is changing and be ready to change with it.
AI may be changing the cybersecurity landscape faster than many organisations realise. We don’t know exactly what comes next, but we have been given the warning.
The question is whether we’re ready for the change.
Please feel free to get in touch if you would like to discuss the topic raised in this insight further.
Image: Exterior of last remaining Blockbuster Video location in Bend, August 2018 by Coasterlover1994,
Comments are closed.