AI security review tools may miss vulnerabilities and create new risks, researchers warn

Researchers at Checkmarx have raised concerns about the security review process in Anthropic’s Claude Code, a tool designed to automate vulnerability detection in code. While the tool successfully detects some vulnerabilities like XSS and authorization bypass issues, it struggles with more complex security flaws, such as remote code execution vulnerabilities using popular libraries like pandas. Additionally, researchers highlighted how Claude Code can be tricked by misleading code crafted to bypass its detection. In one example, a function labeled ‘sanitize’ passed the security review despite running an unsafe process. The tool’s approach of generating and executing its own test cases also poses a risk, as simply running code for review could inadvertently introduce security flaws. The researchers advised caution, urging developers to use the tool only with trusted code and to keep human oversight in the loop. They also warned about the dangers of prompt injection and the potential for AI to make mistakes during security reviews, emphasizing that relying on AI alone without human verification may not ensure secure applications.