Skip to contentLockpick
Docs

Allow Lockpick through your firewall

If your firewall or bot protection blocks Lockpick, let its signed requests through by their Signature-Agent header.

Firewalls, bot checks and sign-in walls can stop a scan. When one blocks it before it sees your site, the report is marked Blocked and names what stopped it, such as Cloudflare's or Vercel's bot check. It is not scored, it still checks your public records, and an auto-check pauses and tells you. When bot protection kicks in partway through a Deep scan instead, the report keeps everything read before that, says your bot protection is working, and offers a prompt for your AI agent that adds the rule below. Allowing Lockpick through lets the next scan finish.

How Lockpick introduces itself

Scans send the same headers as an ordinary Chrome or Edge browser, including its user agent, so a scan sees your site the way a visitor does. A user agent proves nothing anyway: anyone can copy one.

Instead, Lockpick signs every request it sends to your site with Web Bot Auth, the open standard for bots to prove who they are. Each request carries this header, a short-lived signature for your domain that only Lockpick's key can make, and the matching Signature-Input header.

Signature-Agent: "https://bots.lockpick.app"

The public keys are published at https://bots.lockpick.app/.well-known/http-message-signatures-directory. Bot protection that checks these signatures can tell a real Lockpick scan from someone borrowing its name, and providers that list Lockpick as a verified bot let it through without a rule of your own. Until yours does, or if a rule of yours blocks or challenges every bot, add the rule below.

Allow it

Add a rule in your firewall or bot protection that skips bot checks and rate limits when the Signature-Agent header contains bots.lockpick.app. Lockpick doesn't scan from a fixed list of IP addresses, so allow it by this header rather than by address.

  • Vercel. In your project's Firewall, add a custom rule whose condition is Request Header signature-agent contains bots.lockpick.app and whose action is Bypass, above any custom rule that challenges or denies. Attack Challenge Mode challenges every visitor except known bots while it is on, and a custom rule doesn't lift it, so check again once it is off.
  • Cloudflare. Add a custom rule with the expression any(http.request.headers["signature-agent"][*] contains "bots.lockpick.app") and the action Skip, skipping the remaining custom rules, rate limiting, managed rules and Super Bot Fight Mode (paid plans), and put it first. The free plan's Bot Fight Mode can't be skipped by any rule, so it has to be turned off for Lockpick to get through.
  • Other firewalls. Look for a rule that skips bot protection by request header. The report names Akamai, Imperva, Sucuri, AWS WAF, Azure Front Door, DataDome or PerimeterX when their block page says who served it, and says "a firewall" when it can't tell which one answered.

Bot checks from Cloudflare and Vercel

A bot check is the page that asks a visitor to wait or prove they are human, sometimes with a normal "200 OK" status. Lockpick recognises Cloudflare's and Vercel's by the markers they send, stops at once and never tries to solve or get past one. It doesn't retry from another address either.

A firewall that blocks one sensitive path is doing its job, so that alone never stops a scan. When Lockpick checks for files that shouldn't be public (such as /.env or /wp-config.php.bak) or for common routes (such as /admin) and your firewall answers with its block or challenge page, the report records that path as protected and the scan carries on. Only when several of those checks are refused in a row does Lockpick read your homepage again. If the firewall refuses the page it served a minute earlier, it is blocking Lockpick itself, and the scan stops. Deep checks for those sensitive files last, after its route and API checks, and in a different order on every scan. It also only checks for files your host could serve: on Vercel, Netlify or a site builder such as Shopify or Webflow, where only your build output is published, it skips .env, .git, database dumps and server files, and checks your own routes instead.

Why it is gentle on your site

  • Every scan has a fixed limit, at most 10 requests for a Pulse and 200 for a Deep scan.
  • Requests are spaced out, not sent all at once.
  • A scan stops early if your site answers "too many requests", starts failing or denies access.
  • It follows your robots.txt rules for every page it discovers, using the rules written for Lockpick or, when there are none, the rules for all crawlers.
  • The page a visitor opens and the files it loads are the exception. Like a visitor's browser, a scan reads your homepage, the scripts it loads and /.well-known/security.txt even when robots.txt asks crawlers to stay away.
  • If robots.txt can't be read, a scan reads only those and skips every other page.
  • It never signs in, submits forms or changes anything.

Was this helpful?

On this page