AI Security Matrix

A curated list of AI-enabled security testing tools.

mmaitre314/picklescan

Licence
MIT
Maintainers
8 contributors · top author wrote 68%
Size
3,160 lines · 9 code files · tests, CI, Dockerfile
Bundled tools
none found
Shares files with
no overlap found

Before running it

Your right to use, modify or redistribute it is unclear no
The install script is fetched when you run it, so what runs is not the version anyone reviewed no
It sends data to a third party that is not the model provider no
Part of what ships cannot be reviewed by reading the source no
Its container is given access back to the host, so the boundary does not hold no

scanner

Last updatedCategoryToolScopeStarsFlags
★ today scanner microsoft/PyRIT
A library for assembling your own attack rather than a fixed suite: you compose an orchestrator from targets, converters and scorers, and it runs the conversation for you.
llm 4.9k remotecalloutescape
★ today scanner Tencent/AI-Infra-Guard
Gathers several separate AI security scans behind one console: the agent, the MCP servers, the skills, the serving infrastructure, and a jailbreak pass against the model itself.
llmagentic 7.5k remotecallout
★ today scanner cisco-ai-defense/mcp-scanner
Judges an MCP server's tool definitions three ways, by YARA rule, by hosted inspection API and by asking a model, so a poisoned tool description has to clear more than one kind of reader.
agentic 1.1k none
★ today scanner NVIDIA/SkillSpector
Checks an agent skill before you install it, following the files it pulls in and asking a model whether any of it is trying to misdirect the agent. Each verdict carries the provenance behind it.
agentic 27k remotecallout
★ 1d scanner promptfoo/promptfoo
Describes a test suite for an LLM app in a config file and runs it from the CLI or CI, either as a plain eval or as a red-team pass. The declarative file is the point: the same suite reruns on every change.
llmagentic 27k remotecalloutopaque
★ 1d scanner NVIDIA/garak
Fires a catalogue of named probes at a model and reports which ones got through, covering jailbreaks, leakage, injection and toxicity. Probes are the unit you pick, the way you would pick scripts in a port scanner.
llm 10k none
★ 1d scanner snyk/agent-scan
Finds the agent components already installed on your machine, the MCP servers, skills and agent configs, and scans each for prompt injection and known vulnerabilities.
agentic 3.3k none
★ 1d scanner msoedov/agentic_security
Probes across text, images and audio rather than prompts alone, and uses reinforcement learning to adapt a multi-step attack as the model's defences respond.
llmagentic 2.0k callout
★ 1d scanner promptfoo/modelaudit
Covers the widest range of model formats of any of these, emits SARIF for a CI gate, and says plainly where its own analysis was bounded rather than reporting clean.
modfile 86 callout
★ 3d scanner cisco-ai-defense/skill-scanner
Rules on whether an agent skill is malicious, with a model judge whose catch and false-positive rates are published against a labelled test set. A team picks a policy and suppresses known findings, so the same skill gets the same verdict on every run.
agentic 2.7k none
★ 4d scanner Lab700xOrg/aisbom
Produces the compliance artefact as well as the verdict: a CycloneDX or SPDX bill of materials for a model file, alongside the Keras, GGUF and ONNX payloads it found while reading it.
modfile 84 none
★ 6d scanner splx-ai/agentic-radar
Reads the source of an agentic workflow, CrewAI or LangGraph or the OpenAI Agents SDK, and maps what the agents can actually reach before anything runs.
agentic 1.1k none
★ 6d scanner adithyan-ak/AgentHound
Runs from a host you have already compromised: it collects the agent credentials and configuration it finds there, reuses them against whatever AI services are reachable, and records the path it proved.
agentic 464 none
★ 7d scanner trailofbits/fickling
Decompiles pickle bytecode back into readable Python, and will rewrite or build one too. The same tool that tells you a model file is malicious is the one that shows you how such a file is made.
modfile 679 none
★ 17d scanner mmaitre314/picklescan
Scans a pickle for the dangerous imports it would execute on load, and will fetch one straight from Hugging Face, a URL or an archive rather than making you download it first.
modfile 432 none
★ 27d scanner confident-ai/deepteam
Simulates jailbreaks and injection against an application you have already built, then ships the guardrail that blocks what it found, so the test and the defence come from the same vocabulary.
llmagentic 3.1k callout
52d scanner protectai/modelscan
Reads a saved model file and reports the unsafe calls hiding in it, across pickle, HDF5 and SavedModel, so a download from a public registry can be checked before anything loads it.
modfile 799 none
281d scanner anthropics/claude-code-security-review
A GitHub Action that has a model review only the diff of a pull request, comments the security issues inline, then filters its own findings to cut the false positives a diff-only view invites.
code 7.0k none
286d scanner cyberark/FuzzyAI
Fuzzes a model endpoint with a catalogue of jailbreak techniques, mutating prompts until one gets past the guardrail, and works against a local model as readily as a hosted one.
llm 1.6k none
333d scanner mbrg/power-pwn
Works the Microsoft 365 side of the problem: Power Platform flows, Copilot Studio bots left open to anonymous users, and the agents a tenant has published without noticing.
cloudllm 1.3k none
353d scanner utkusen/promptmap
Runs a second model whose only job is to read the target's replies and rule on whether an injection actually worked. Point it at your own system prompt, or at an HTTP endpoint you know nothing about.
llm 1.3k none
archived scanner Azure/counterfit
Brings the established adversarial machine-learning frameworks under one CLI and points them at a model you nominate. Its subject is classic ML, not language models.
llm 943 none
1.8y scanner protectai/vulnhuntr
Reads a Python codebase and asks a model to trace user input through to dangerous sinks, reporting the call chain behind each bug it claims. It applies no rules or patterns; the model is the whole analysis.
code 2.8k none