garak: LLM vulnerability scanner
@garak_llm
Followers
870
Following
11
Media
6
Statuses
145
LLM vulnerability scanning & security audit code: https://t.co/T6VrDoBKKT 🐍 discord: https://t.co/bmllkcnenx 🕶️ pip install garak
Latent space
Joined May 2023
New garak release: v0.10.0! Highlights:. * detector: llm as a judge .* generator: update openai for o1 models.* generator: Ollama .* generator: support of azure openai .* generator: vision nims .* probe: add whois injection to latentinjection . Details:
1
2
17
garak has moved to NVIDIA!. New repo link:
3
39
211
New garak release: v0.9.0.16!. New plugins:. * probe: topic pushing .*probe: ruby package hallucination.*probe: Latent prompt injection (aimed at RAGs) .* probe: npm package hallucination.* probe: rust crate hallucination.* generator: Groq API . New features:. * extract.
0
1
7
RT @LeonDerczynski: Will be giving a US Congressional briefing on garak, LLM vulnerability scanner on Sept 5th. We've come a long way. @gar….
0
2
0
new version: v0.9.0.15!. * check HF model repo source for potentially malicious files.* report comparative info for scores, so you know if your scores are "good".* standardize output and caching directories.* make HF models even more configurable.
0
1
5
RT @LeonDerczynski: Oh cool, Fortune/MSN picked up garak (@garak_llm) in their DEF CON highlights!. .
0
3
0
RT @NannaInie: “You’re welcome!” .@LeonDerczynski and Erick Galinkin presenting @garak_llm at @defcon AI Village. #defcon32 .
0
2
0
RT @LeonDerczynski: All the contributing authors of the @garak_llm paper in one place. And where else but the @defcon @aivillage_dc stage!….
0
3
0
RT @LeonDerczynski: At DEF CON? Say Hi at one of our presentations on garak: LLM vulnerability scanner! @garak_llm. 1. garak at DEF CON Dem….
0
1
0
new version: garak 0.9.0.14!. * huge amounts of stability and bug fixes, the core is in great shape.* BEAST attack added.* llama 3.1 support.* plugin cache for super fast operation.* grandma attack added. read full release description here:
0
2
6
Using garak to do an LLM security scan on a web service - great video tutorial by @EmbraceTheRed23 .
0
6
13
Would you like to scan an LLM for vulnerabilities, but garak doesn't support it? No worries! Now you can write your own generator, using our contributing guide:.
0
1
2
🎂 birthday feature: NVIDIA Inference Microservices support!. NIMs contain any LLM or AI service, can run locally or hosted, and are supported by garak :).
0
0
1
🎂 birthday feature: Hugging Face scan speedup!. Add calls to Hugging Face generators to reduce GPU overhead.
1
0
1
🎂 birthday feature: LiteLLM support!. LiteLLM handles loadbalancing, fallbacks and spend tracking across 100+ LLMs. All in OpenAI format.
1
0
0
🎂 birthday feature: text-to-text&image scanning!. garak now supports vulnerability scanning of text-to-text&image models, using the FigStep/SafeBench library. live in today's version. PR here:
1
0
0
Happy birthday garak, LLM vulnerability scanner! One year since first commit 🍰 Many features announced today!.
1
3
12
today's dependency-fixing PR done from none other than: ☺️
0
0
0
you could pay for jailbreak prompts - - or you could just use garak's DanInTheWild probe to run a set of 666 known good jailbreaks against your LLM, and see if the model blocks them. 395 of these jailbreaks aren't mitigated by gpt-3.5-turbo 😬
0
3
3
garak v0.9.0.12 out:. Plugins:.* Tree of Attacks🌴 by @ErickGalinkin .* New encoding probes by @ZanderMackie .* Add Rasa generator by @nworksgreg.* Low Resource Languages Buff by @erickgalinkin. Functionality improvements:.* support multiple buffs.* autosearch in configs/ subdir.
0
0
4