Gatekeeper - Access Control & Compliance Profiler

This site tests crawler compliance with access restrictions: robots.txt rules, CAPTCHAs, login gates, cookie consent walls, and rate limiting.

Test Areas

robots.txt

Custom rules for different user agents.

View robots.txt

Public Page

Freely accessible page (allowed in robots.txt).

Visit

Private Page

Behind robots.txt Disallow (content still served).

Visit

Slow Page

In the Crawl-delay section of robots.txt.

Visit

Math CAPTCHA

Simple math challenge to reveal content.

Visit

Image CAPTCHA

Image-based CAPTCHA challenge.

Visit

Login

Session-gated content behind a login form.

Visit

Protected Page

Only accessible with a valid session cookie.

Visit

Cookie Wall

Content hidden behind a cookie consent wall.

Visit

Rate Limiting

All routes on this site are rate-limited. Exceeding Gatekeeper - Access Control & Compliance Profiler requests per minute from a single IP will result in a 429 Too Many Requests response.