External Threat Notifications: A Strategy for Response
A practical framework for organisations to evaluate and respond to external cyber threat notifications based on source reliability and information actionability.
For decades, technology companies have built walls around their products - carefully constructed boundaries that define how users can interact with their services. These walls weren't physical barriers, but rather a collection of security controls, access mechanisms, and fundamental assumptions about how humans and machines would interact with applications. Like the walls of a fortress, they were designed to keep unwanted automated access out while allowing legitimate human users to pass freely through well-defined gates.
But what happens when these walls - walls we've relied upon for our security architecture, product design, and risk management - start to disappear? What happens when the distinction between human and automated interaction becomes so blurred that our traditional defences no longer serve their purpose? What does it mean for the organisations that use these products?
In the realm of cyber security, we've long operated under certain assumptions about the boundaries between human and automated interactions. These assumptions have shaped how we build, secure, and monitor our applications. However, recent technological advances are rapidly eroding these traditional security boundaries - and many organisations haven't yet recognised the implications.
Historically, technology companies have designed their products with a clear vision of intended use: human users interacting through prescribed interfaces. While defending against automated threats like bots and scrapers has been a longstanding and growing challenge, organisations have fought for control through well-established mechanisms. API access management has allowed for legitimate third-party integrations, while tools like rate limiting, CAPTCHA systems, session monitoring, and network traffic analysis have helped identify and block unwanted automated access.
This model created clear walls around product use, allowing companies to maintain reasonable control over how their services were accessed and utilised. The boundaries between human and automated interaction were distinct, and security controls could be built around these assumptions.
Several emerging technologies and features have begun to challenge these traditional boundaries. While some of these capabilities aren't entirely new, their increasing accessibility and sophistication present novel challenges to our existing security models.
Advanced computer vision systems, coupled with Large Language Models (LLMs), can now interpret and interact with application interfaces just as a human would. These systems can "see" application output and take contextually appropriate actions, all while appearing to be legitimate user activity.
An example of this is Microsoft’s Recall, which when enabled takes screenshots of a user’s activity on their PC, interprets the information it sees, and stores the information in a local database. Recall was delayed a few months after a series of security concerns were raised, but was released into preview in the last week. Screenpipe is a cross-platform open-source tool that accomplishes similar capability in an extensible and open way, and also extends to video and audio content. These new technologies bypass some of the walls we thought existed by taking data presented in an authenticated user application context into uncontrolled repositories.
Extending on that are recent advances in AI capabilities. Claude's "computer use" functionality, for instance, allows an AI to directly control a computer - navigating websites, filling forms, clicking buttons, and even handling multi-step workflows. While Robotic Process Automation technology has had some of these capabilities for years, this isn't just simple automation; it's context-aware interaction that can adapt to changing situations, handle errors gracefully, and make decisions based on what it observes on screen. Claude can maintain state across interactions, remember previous steps, and adjust its approach based on the outcomes it observes. So there goes another wall - what can look like a user can be an AI tool, sitting in the context of a user, on their device, with their cookies, network origin, and probably a more reliable hit rate of determining which squares in a CAPTCHA contain traffic lights.
Lastly, the recent introduction of iPhone mirroring has increased the exposure of mobile applications to this broader ecosystem of automation tools. While they can also be exposed through emulators or development simulators, iPhone mirroring lowers the barrier to the average user, and creates a reason for developers to build things around its use. Computer use of mobile applications creates a new vector for unexpected application usage that bypasses traditional security controls, effectively removing another wall we thought we had in place.
These technological advances have already led to creative - and potentially concerning - applications. Consider how a group of university students demonstrated that Meta's Ray-Ban smart glasses could be repurposed for real-time facial recognition. While the glasses themselves limit access to raw video to only authorised applications, the students used computer vision to analyse Instagram livestreams (which were a permitted application), matching captured faces against commercial databases, and feeding back details to the wearer. This effectively turned a consumer product into an unintended surveillance system by jumping over the walls the developer thought they’d put in place.
In another instance, a developer combined Claude's computer use capabilities with iPhone mirroring to automate interactions on a mobile dating application, automatically matching with selected people and generating personalised initial messages - all while appearing indistinguishable from genuine user behaviour. These examples highlight how the walls we thought existed between intended and unintended use cases are rapidly disappearing.
The dissolution of traditional security boundaries should prompt us to think about where those boundaries are present in our own organisations.
First, organisations need to broaden their threat models to account for these new capabilities. How might data move in an expected way between systems, and what actions might be automated beyond our visibility? How could that impact the security of your organisation?
Second, we should begin treating user interfaces as de facto APIs, recognising that any visual interface could be programmatically controlled through computer vision and automation tools, but without some of the same markers that automation typically left. This change requires us to think differently about how we design and secure our applications. How should we change our approach to application logging and monitoring to better understand what applications are showing users and give us a chance to trace potential leakage through tools like Recall and Screenpipe?
Third, product teams should evaluate features through the lens of potential automated manipulation. We should be evaluating points where we really want a human user to be present and consider better methods of verification for these actions. This might involve implementing extra verification for critical actions, adding friction at strategic points where automated abuse could cause harm, developing new heuristics for detecting automated behaviour that appears human, and creating safeguards around features that could be combined in unexpected ways.
Finally, we need to consider what other "walls" we've taken for granted might be overcome by these and other emerging technologies. For instance, the boundaries between different applications within a device ecosystem, the separation between physical and digital interactions, the efficacy of MDM controls, or the distinctions between different user privilege levels might all be challenged by advancing technology capabilities.
In future articles, we'll explore these themes in more detail. An upcoming piece will delve into the security implications of computer control capabilities, examining both the opportunities and risks they present for organisations. We'll also investigate how businesses can adapt their security architecture to match this new landscape where traditional boundaries are increasingly fluid.
