Skip to content

Entity

Robots.txt

Also known as: Robots Exclusion, Crawl Directive

A text file instructing search engine crawlers which URLs they can or cannot access.

Topic Cluster

SEO & Content Optimization

Parent Entity

Related Entities

Keyword Readability SERP Open Graph Sitemap Alt Text Heading Structure PageRank Canonical URL Hreflang Structured Data Core Web Vitals Web Crawler Search Index Backlink

Related Tools

Testrefy requires JavaScript. Please enable JavaScript in your browser settings.