Giskard-AI /giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
★ 5,953⑂ 573👁 40⑃ 2,222● 637▤ 146
+2
日增长
5,953
stars
573
forks
40
watchers
2,222
prs
637
issues
146
releases
README
Evals, Red Teaming and Test Generation for Agentic Systems Modular, Lightweight, Dynamic and Async-first Docs • Website • Community [!IMPORTANT] Giskard v3 is a fresh rewrite designed for dynamic, multi-turn testing of AI agents. This release drops heavy dependencies for better efficiency while introducing a more powerful AI vulnerability scanner and enhanced RAG evaluation — both now shipping natively in giskard-scan, with no dependency on v2. Only the legacy scan for tabular/ML models remains v2-only. Giskard v2 remains available but is no longer actively maintained. Follow progress → Read the v3 Announcement · Roadmap Install sh pip install giskard checks (+ agents, llm, core) pip install "giskard[scan]" + vulnerability / quality scan pip install "giskard[openai]" provider SDK for LLM judges / generators Requires Python 3.12+. | Extra | Adds | | --- | --- | | (none) | giskard-checks and dependencies | | scan | giskard-scan | | openai / anthropic / … | provider SDKs (see pyproject.toml optional deps) | Telemetry: optional aggregated analytics via giskard-core. No prompts or outputs are sent. Opt out with export DO_NOT_TRACK=1 or export GISKARD_TELEMETRY_DISABLED=1 (or the same keys in a .env file in the working directory). Set them before import to skip creating ~/.giskard/id; setting them later still stops further sends. Details: giskard-core README. --- Giskard is an open-source Python library for testing and evaluating agentic systems. The v3 architecture is a modular set of focused packages — each carrying only the dependencies it needs — built from scratch to wrap anything: an LLM, a black-box agent, or a multi-step pipeline. | Status | Package | Description | | -------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ✅ Stable | giskard-checks | Testing & evaluation — scenario API, built-in checks, LLM-as-judge | | ✅ Stable | giskard-scan | Agent vulnerability scanner + RAG/quality evaluation — red teaming, prompt injection, jailbreaks & harmful content (vulnerability_scan, successor of v2 Scan), plus knowledge-base quality eval (quality_scan, successor of v2 RAGET) | These build on three foundational libraries — giskard-core (shared utilities & telemetry), giskard-llm (provider-agnostic LLM routing), and giskard-agents (agent & workflow orchestration) — which are pulled in automatically and rarely used directly. Giskard Checks — create and apply evals for testing agents sh pip install giskard-checks Giskard Checks is a lightweight library
Top issues
#2454Feature request: OWASP LLM02 output-handling detectors pack (XSS / SQLi / Shell / Path)7 comments · 2026-05-15#2876ChatWorkflow may leave tasks running after stream closure or cancellation4 comments · 2026-10-12#2421Feature request: optional verbosity during Suite.run (per check / per generation)4 comments · 2026-04-20#2864RAG evaluation example: external source-rights evidence checks per retrieved document3 comments · 2026-10-06#2802GenerationParams silently drops unsupported fields (e.g. api_base) instead of raising2 comments · 2026-08-26#2736last_interaction_index misattributes steps that add no interactions2 comments · 2026-08-12#2735detect-secrets is not enforced in CI and the baseline is unaudited2 comments · 2026-08-12#2712Feature request: EvalPort import/export for Suites and Scenarios2 comments · 2026-08-09
Discussions
#2868Introducing self-service Giskard: open to beta testers0 comments · 2026-10-08#2825Pinning the LLM calls behind a Giskard scan so a run reproduces exactly0 comments · 2026-09-11#2814Giskard v3 is here!0 comments · 2026-09-02#2537Setting temperature for evaluation1 comments · 2026-06-11#2498Pre-registering eval claims for Giskard scenarios (falsify-giskard): of interest?1 comments · 2026-06-01#2447SymbolicCheck: formally-verified invariants on structured outputs1 comments · 2026-05-11#2330LiteLLM critical security compromise – check your environments0 comments · 2026-03-24#2318about explaining giskard-oss1 comments · 2026-03-20
Latest release
giskard-agents v1.0.6 giskard-agents/v1.0.6 2026-10-25What's Changed fix(deps): bump certifi, filelock and setuptools to clear pip-audit advisories by @kevinmessiaen in https://github.com/Giskard-AI/giskard-oss/pull/2898 [Security] Exclude litellm compromised version by @pierlj in https://github.com/Giskard-AI/giskard-oss/pull/2896 Full Changelog: https://github.com/Giskard-AI/giskard-oss/compare/giskard-agents/v1.0.5...giskard-agents/v1.0.6