1 reactions
Beyond being a superstar at identifying and exploiting vulnerabilities in digital systems, Anthropic’s Mythos frontier model is now being directed at encryption algorithms that are designed to resist quantum computing attacks.
The early results, as published by Anthropic , are an important glimpse into the future of digital security. Mythos has successfully degraded both the HAWK and a weaker version of AES encryption. The research does not impact current computer systems but it does showcase how researchers can leverage AI to advance attacks against algorithms that we will need in the future to protect our digital ecosystem!
Encryption underpins transactional security and privacy across the digital world. It protects online banking transactions, communications, webpage interactions, and the confidentiality of sensitive data. Unless we want to revert back to face-to-face transactions and paper documents, strong encryption is an absolute necessity!
Over the course of a week, small teams at Anthropic guided Mythos’s mostly autonomous work to achieve these early results. Yes, just one week. To put that into context, it was about a decade ago that NIST asked the brightest minds in the industry to develop robust algorithms that would be resistant to quantum computer-based attacks. Many candidates have undergone years of intense evaluation for becoming the replacements to current algorithms that protect us from traditional computers. Researchers have been testing and working to find weaknesses or break these proposed digital locks for years.
Mythos made progress in days.
HAWK was submitted as an additional potential signature in June of 2023. It benefitted from the deep learnings of previous submissions and research, to join other algorithms that had already been under evaluation but not yet put into practical use. AES, the other target, is perhaps the most commonly used encryption algorithm, which was adopted in 2001.
It is important to emphasize that Mythos did not completely break these algorithms, but rather found paths that reduce their robustness and potentially make future efforts to undermine them easier. With that said, researchers will continue to use frontier AI tools to test current and future security algorithms and they may eventually find ways to break them.
This is both good news, if we find and address weaknesses before any attackers, and potentially cataclysmic if malicious hackers are first to find and exploit algorithms after they have been widely adopted.
Strategically, we are seeing frontier AI models, like Mythos and others, showcase their aptitude at finding weaknesses in software, hardware, algorithms, processes, and people. The AI tools are working at a speed and in ways that humans cannot. That can bring truly amazing benefits, but is also accompanied by equally severe risks.
The key to success will be finding ways to seize the great benefits of AI adoption while managing the risks to acceptable levels.
For cybersecurity specifically, we must support and facilitate AI initiatives in support of business goals, while partnering to manage the cyber risks in acceptable ways.
#cybersecurity #AI #encryption
The early results, as published by Anthropic , are an important glimpse into the future of digital security. Mythos has successfully degraded both the HAWK and a weaker version of AES encryption. The research does not impact current computer systems but it does showcase how researchers can leverage AI to advance attacks against algorithms that we will need in the future to protect our digital ecosystem!
Encryption underpins transactional security and privacy across the digital world. It protects online banking transactions, communications, webpage interactions, and the confidentiality of sensitive data. Unless we want to revert back to face-to-face transactions and paper documents, strong encryption is an absolute necessity!
Over the course of a week, small teams at Anthropic guided Mythos’s mostly autonomous work to achieve these early results. Yes, just one week. To put that into context, it was about a decade ago that NIST asked the brightest minds in the industry to develop robust algorithms that would be resistant to quantum computer-based attacks. Many candidates have undergone years of intense evaluation for becoming the replacements to current algorithms that protect us from traditional computers. Researchers have been testing and working to find weaknesses or break these proposed digital locks for years.
Mythos made progress in days.
HAWK was submitted as an additional potential signature in June of 2023. It benefitted from the deep learnings of previous submissions and research, to join other algorithms that had already been under evaluation but not yet put into practical use. AES, the other target, is perhaps the most commonly used encryption algorithm, which was adopted in 2001.
It is important to emphasize that Mythos did not completely break these algorithms, but rather found paths that reduce their robustness and potentially make future efforts to undermine them easier. With that said, researchers will continue to use frontier AI tools to test current and future security algorithms and they may eventually find ways to break them.
This is both good news, if we find and address weaknesses before any attackers, and potentially cataclysmic if malicious hackers are first to find and exploit algorithms after they have been widely adopted.
Strategically, we are seeing frontier AI models, like Mythos and others, showcase their aptitude at finding weaknesses in software, hardware, algorithms, processes, and people. The AI tools are working at a speed and in ways that humans cannot. That can bring truly amazing benefits, but is also accompanied by equally severe risks.
The key to success will be finding ways to seize the great benefits of AI adoption while managing the risks to acceptable levels.
For cybersecurity specifically, we must support and facilitate AI initiatives in support of business goals, while partnering to manage the cyber risks in acceptable ways.
#cybersecurity #AI #encryption
3 reactions
The latest AI model went rogue, jumped the fence, and attacked another company. It is making headlines for a reason and is telling indicator of what is to come.
The Incident
In a nutshell, this is what happened:
1. OpenAI tests its latest model against the CyberGym AI cybersecurity benchmark, in an isolated sandbox, to evaluate how effective it is at vulnerability identification and exploitation.
2. The AI determines that the tests are so difficult that it would be easier to cheat.
3. So, it finds and exploits an unknown 0-day vulnerability to break out of the sandbox and get Internet access.
4. It then determines and targets Hugging Face as it believes that is where the answers are.
5. The AI conducts thousands of attacks simultaneously against Hugging Face, across multiple attack paths, and quickly adapts when blocked to find alternative paths as it moves laterally in search of answers.
6. Hugging Face detects the issues and attempts to use the top frontier AI models with the complex attack analysis, but is thwarted by cybersecurity guardrails. So, they use a Chinese-based open-weight model, running locally, to assist without the burdens of commercial security limitations.
There is so much to unwrap with this situation. Let’s break down the systemic issues.
Top Cybersecurity Strategic Red Flags:
1. Software-based sandboxes are not secure against top-tier AI models, which are designed to find and exploit vulnerabilities. This should be obvious. What is needed is a physically isolated, air-gapped dirty lab. Yes, think in terms of a Faraday cage supported by strict human security process controls as well.
2. When the AI determines that cheating a security test is the right thing to do, it is clear that there is insufficient or an absence of AI Ethics instituted into the model and supervising controls. This is a dangerous oversight of the developers. There are times when this should be allowed, but also should include human oversight to specifically allow such actions.
3. Why didn’t the infrastructure security controls, that oversee the sandbox enclave, trigger when isolation was undermined and Internet access was achieved? This is a failure of security compartmentalization oversight and security network access (i.e. Zero Trust implementation) for control, detection, and response to forbidden access.
4. Targeting Hugging Face was smart for the AI. No issues with the Hugging Face team as they were able to detect the issue and respond. They have learned many lessons, including the fact that it is terribly difficult to recognize malicious activity as it looks very much like legitimate AI development work.
5. The AI attack orchestration was overwhelming in quantity and speed, just as we have predicted it would be. The good news is the complexity was not very high in this case. It was a ton of fast dumb attacks that overwhelmed defenses. Brutish, but effective.
6. To analyze such AI-orchestrated attacks, it requires very powerful AI tools. But in this case, the latest frontier AI models were problematic because they had cybersecurity guardrails that inhibited much of the necessary work. Guardrails are important to prevent attackers from using them in malicious ways, but the other side of that blade is those limitations can degrade victims from understanding and responding to attacks.
This forced Hugging Face to use the Chinese open-weight GLM model, running locally, to do the work.
This poses a whole new set of concerns!
I appreciate the creative thinking and necessity of finding workable tools to respond to the crisis, but this scenario represents a wake-up call for the entire industry as we should not trust or rely upon AI models from adversarial nations that have a history of untrustworthy activity. Our industry needs a better solution, perhaps a trusted escalation path for temporary access to unrestricted frontier models during a crisis event. This even sounds like a good upsell opportunity for AI model developers.
What is Coming and How to Improve AI Security
First, let me say that we in the cybersecurity community have been proactively warning about these types of incidents. This entire situation is scary, but what worries me much more is that very soon it won't be thousands of dumb simultaneous attacks, but rather thousands of innovative and brilliant simultaneous attacks!
We must contend with the inevitable:
1. AI systems going rogue and acting in harmful ways will continue to happen. Until we have AI Ethics embedded and effective independent security oversight in place, AI will act in unscrupulous ways not intended by developers or users.
2. AI-orchestrated attacks will skyrocket and deliver an entirely new level of attack capabilities, to the detriment of victims. We will see attacks, both intentional and accidental, that combine the skyrocketing capability of AI to detect and exploit vulnerabilities, move laterally, and cause harms at machine speed.
3. AI tools and supporting processes are necessary for defenders to use as part of their protection and resilience activities. We must move faster to develop better AI enhanced cybersecurity capabilities and integrate them into our overall strategies and protective postures.
This is a race that the attackers are winning, and we cannot afford to lose. More work is required to develop and embrace AI ethical and security controls, while also moving faster to enable AI enhancements for defenders.
#cybersecurity
#AI
The Incident
In a nutshell, this is what happened:
1. OpenAI tests its latest model against the CyberGym AI cybersecurity benchmark, in an isolated sandbox, to evaluate how effective it is at vulnerability identification and exploitation.
2. The AI determines that the tests are so difficult that it would be easier to cheat.
3. So, it finds and exploits an unknown 0-day vulnerability to break out of the sandbox and get Internet access.
4. It then determines and targets Hugging Face as it believes that is where the answers are.
5. The AI conducts thousands of attacks simultaneously against Hugging Face, across multiple attack paths, and quickly adapts when blocked to find alternative paths as it moves laterally in search of answers.
6. Hugging Face detects the issues and attempts to use the top frontier AI models with the complex attack analysis, but is thwarted by cybersecurity guardrails. So, they use a Chinese-based open-weight model, running locally, to assist without the burdens of commercial security limitations.
There is so much to unwrap with this situation. Let’s break down the systemic issues.
Top Cybersecurity Strategic Red Flags:
1. Software-based sandboxes are not secure against top-tier AI models, which are designed to find and exploit vulnerabilities. This should be obvious. What is needed is a physically isolated, air-gapped dirty lab. Yes, think in terms of a Faraday cage supported by strict human security process controls as well.
2. When the AI determines that cheating a security test is the right thing to do, it is clear that there is insufficient or an absence of AI Ethics instituted into the model and supervising controls. This is a dangerous oversight of the developers. There are times when this should be allowed, but also should include human oversight to specifically allow such actions.
3. Why didn’t the infrastructure security controls, that oversee the sandbox enclave, trigger when isolation was undermined and Internet access was achieved? This is a failure of security compartmentalization oversight and security network access (i.e. Zero Trust implementation) for control, detection, and response to forbidden access.
4. Targeting Hugging Face was smart for the AI. No issues with the Hugging Face team as they were able to detect the issue and respond. They have learned many lessons, including the fact that it is terribly difficult to recognize malicious activity as it looks very much like legitimate AI development work.
5. The AI attack orchestration was overwhelming in quantity and speed, just as we have predicted it would be. The good news is the complexity was not very high in this case. It was a ton of fast dumb attacks that overwhelmed defenses. Brutish, but effective.
6. To analyze such AI-orchestrated attacks, it requires very powerful AI tools. But in this case, the latest frontier AI models were problematic because they had cybersecurity guardrails that inhibited much of the necessary work. Guardrails are important to prevent attackers from using them in malicious ways, but the other side of that blade is those limitations can degrade victims from understanding and responding to attacks.
This forced Hugging Face to use the Chinese open-weight GLM model, running locally, to do the work.
This poses a whole new set of concerns!
I appreciate the creative thinking and necessity of finding workable tools to respond to the crisis, but this scenario represents a wake-up call for the entire industry as we should not trust or rely upon AI models from adversarial nations that have a history of untrustworthy activity. Our industry needs a better solution, perhaps a trusted escalation path for temporary access to unrestricted frontier models during a crisis event. This even sounds like a good upsell opportunity for AI model developers.
What is Coming and How to Improve AI Security
First, let me say that we in the cybersecurity community have been proactively warning about these types of incidents. This entire situation is scary, but what worries me much more is that very soon it won't be thousands of dumb simultaneous attacks, but rather thousands of innovative and brilliant simultaneous attacks!
We must contend with the inevitable:
1. AI systems going rogue and acting in harmful ways will continue to happen. Until we have AI Ethics embedded and effective independent security oversight in place, AI will act in unscrupulous ways not intended by developers or users.
2. AI-orchestrated attacks will skyrocket and deliver an entirely new level of attack capabilities, to the detriment of victims. We will see attacks, both intentional and accidental, that combine the skyrocketing capability of AI to detect and exploit vulnerabilities, move laterally, and cause harms at machine speed.
3. AI tools and supporting processes are necessary for defenders to use as part of their protection and resilience activities. We must move faster to develop better AI enhanced cybersecurity capabilities and integrate them into our overall strategies and protective postures.
This is a race that the attackers are winning, and we cannot afford to lose. More work is required to develop and embrace AI ethical and security controls, while also moving faster to enable AI enhancements for defenders.
#cybersecurity
#AI
1 reactions
Today I have reached 200,000 followers on LinkedIn. My deepest and most sincere thanks to all my colleagues and followers for all the great conversations, sharing of personal insights, and the deep collaboration over the years!
When I started posting on LinkedIn, my goal was simple: to share knowledge, foster communication among the cybersecurity community, and build collaboration among the thought leaders in our industry. I never imagined this community would grow to this size or that there are even 200,000 people interested in cybersecurity strategy and insights.
But here we are. Cybersecurity has evolved over the past three decades to become one of the most important domains that stretches across every industry, governments, and critical infrastructure sectors. Together, we are the force that keeps the global digital ecosystem secure, resilient, and trustworthy. We face serious adversaries in a technological world that is constantly changing. We struggle with the inherent challenges of showcasing our value and justifying the friction that we impose to protect our environments against enemies that cannot be seen until it is too late.
Now more than ever, it is critical that we share our insights, co-develop innovative ideas, and partner together to stand stronger against those who seek to victimize, undermine, disrupt, and destroy our digital world.
Thank you to everyone who has followed, commented, challenged my perspectives, shared my content, and contributed to the conversations. The quality and engagement of this community is what makes it so valuable. Your expertise, collaboration, and willingness to exchange ideas have continually expanded my own thinking.
I look forward with hope (and an enormous cup of coffee) that our #cybersecurity community will continue to rise in pursuit of optimal risk management, keep pace with our adversaries, and relish in the continual challenges that come with protecting our digital world.
Thanks for being on this journey with me.
When I started posting on LinkedIn, my goal was simple: to share knowledge, foster communication among the cybersecurity community, and build collaboration among the thought leaders in our industry. I never imagined this community would grow to this size or that there are even 200,000 people interested in cybersecurity strategy and insights.
But here we are. Cybersecurity has evolved over the past three decades to become one of the most important domains that stretches across every industry, governments, and critical infrastructure sectors. Together, we are the force that keeps the global digital ecosystem secure, resilient, and trustworthy. We face serious adversaries in a technological world that is constantly changing. We struggle with the inherent challenges of showcasing our value and justifying the friction that we impose to protect our environments against enemies that cannot be seen until it is too late.
Now more than ever, it is critical that we share our insights, co-develop innovative ideas, and partner together to stand stronger against those who seek to victimize, undermine, disrupt, and destroy our digital world.
Thank you to everyone who has followed, commented, challenged my perspectives, shared my content, and contributed to the conversations. The quality and engagement of this community is what makes it so valuable. Your expertise, collaboration, and willingness to exchange ideas have continually expanded my own thinking.
I look forward with hope (and an enormous cup of coffee) that our #cybersecurity community will continue to rise in pursuit of optimal risk management, keep pace with our adversaries, and relish in the continual challenges that come with protecting our digital world.
Thanks for being on this journey with me.
