How we test

Test levels

Every recipe shows one badge: the highest level the kit actually reached, with the Claude Code version and the date of the test. Claude Code changes often, so a result holds for that version and date, not forever.

Static check

Our lab checked the structure, the plugin manifest and the files, and scanned them for secrets and risky commands. Nothing was run in Claude Code.

Smoke tested

Claude Code actually loaded the kit and ran one scripted task in a capped run, and the result passed our checks.

End-to-end tested

The full workflow ran on a sample repository, and automated checks verified the outcome.

Human tested

A named person installed the kit and used it for the stated outcome.

Smoke runs are capped in time and spend, and they run in a separate test harness on our own machine, not in the automation that drafts pages. A recipe page also shows the latest lab run when we have one, with each check that passed or failed.

What we don't test

• Your repository, your operating system or your settings. A kit that passed on our sample can still fail on yours.
• Every Claude Code version. We test one version and name it; later releases can change behaviour.
• Long-term use. A test is one run on one day, not weeks of daily work.
• Security in depth. Our static check scans for secrets, unknown network calls and destructive commands; it is not a security audit. Read a kit's files before you install it, because hooks run with your permissions.
• Directory listings. Tools, Skills and MCP servers in our directory are not tested unless a recipe says so with a badge.

How AI is used, and who reviews

Our automation uses Claude to draft many pages from public sources, such as directory listings, news summaries and first drafts of guides. Automated checks run before anything is published: required fields, links, the disclaimer, and rules on where external links may point. Not every automated page is read by a person before it goes live.

Recipes are different. Tran Quang Dong (Founder & editor) is the named editor responsible for every recipe and for fixing reported errors. A recipe is marked human-tested only when a named person actually ran it.

Sources, checked dates and source alerts

Prices, limits, versions and commands come from official sources, and pages that state them show the date we last checked them ("Checked" next to the publish date). Some pages also carry a review-by date; when it passes, our build warns us to check the page again.

A watcher job re-reads the official pages we cite, such as the Claude pricing page, the Help Center articles on usage limits and the Claude Code changelog, several times a day. When one of them changes after our checked date, every page that relies on it shows this notice until we have re-checked: "An official source for this page changed on <date>; we're re-checking." The watcher never edits numbers itself; a person does.

Money and testing stay separate

Affiliate links and the kits we sell never carry a test badge, and the plan picker never recommends a paid kit. See the affiliate disclosure for how the site is funded.

Corrections

Found a wrong number, a broken command or a test result that does not match what you see? Email hello@claudekit.ai with the page and what you saw. We check every report against the source, fix confirmed errors and update the page's checked date.

Editor

Tran Quang Dong, Founder & editor. More about the project on the About page.