Momentummomentum

MomentumAuditBot

Public crawler documentation for Cloudflare Verified Bot review and site operators.

Operator

CompanyTall Momentum Limited
Websitehttps://momentum.tall.agency
Contactprivacy@tall.agency

Purpose

MomentumAuditBot is a legitimate SEO and website auditing crawler. It performs:

  • SEO auditing and technical website audits
  • Metadata checks (title, description, canonical, Open Graph)
  • Link validation and broken resource detection
  • Crawl diagnostics and performance-related checks
  • Content discovery and client website health analysis

User Agent

MomentumAuditBot/1.0 (+https://momentum.tall.agency/crawler; contact=privacy@tall.agency)

Expected Crawl Behaviour

  • Crawls public pages only discovered via sitemaps
  • Does not attempt login, checkout, admin, account, cart, payment, or private areas
  • Does not submit forms
  • Does not bypass authentication or security controls
  • Respects robots.txt rules for MomentumAuditBot and wildcard (*) agents
  • Honours Crawl-delay directives where present
  • Backs off when rate limited (429) with Retry-After support
  • Uses a clear, identifiable user agent
  • Signs outbound requests with Cloudflare Web Bot Auth when enabled
  • Identifies itself transparently via this documentation page

robots.txt

MomentumAuditBot fully respects robots.txt. Site owners can allow or block the crawler:

Allow

User-agent: MomentumAuditBot
Allow: /

Block

User-agent: MomentumAuditBot
Disallow: /

The crawler also avoids sensitive paths by default (e.g. /wp-admin, /wp-login.php, /checkout) regardless of robots.txt.

Allowlisting & Blocking

To allow the crawler:

  • Add the robots.txt Allow rule shown above
  • Allowlist User-Agent MomentumAuditBot in your WAF
  • Allowlist IP range 3.40.0.32/27 in your firewall (Trigger.dev egress)

To block the crawler:

  • Add the robots.txt Disallow rule shown above
  • Block User-Agent MomentumAuditBot in your WAF

For questions, abuse reports, or allowlisting assistance, contact privacy@tall.agency.

Cloudflare Web Bot Auth

MomentumAuditBot signs outbound HTTP requests using Cloudflare Web Bot Auth and HTTP Message Signatures (RFC 9421). Signed requests include Signature, Signature-Input, and Signature-Agent headers.

Public key directory

https://momentum.tall.agency/.well-known/http-message-signatures-directory

The public key is used to verify crawler identity — it does not grant access to private content. Site operators and CDNs can verify signed requests against the directory above.

Test verification: crawltest.com/cdn-cgi/web-bot-auth