MIT Custom Robots.txt Rules
- by Meetanshi Technologies LLP
- $2.99/month. Free trial available.
Edit your store's robots.txt file with a simple editor. Control search engine bots and LLM crawlers without coding or file downloads.
Claim app
Verify your developer profile to earn a badge and build trust to your apps
About the MIT Custom Robots.txt Rules
This app lets you modify your store's robots.txt file to control how search engine bots and LLM crawlers access your site. Edit the default robots.txt file directly in the dashboard using a simple structured editor—no coding, copying, or downloading required.
Set custom Allow/Disallow rules and Crawl-delay values for individual user-agents. Add comments to keep your directives organized for future reference. Switch between the structured editor and raw file view as needed, and add user-agent groups whenever required.
The built-in validator checks your robots.txt rules syntax in real-time to prevent errors. Use undo and redo buttons while editing, or easily revert to default settings if needed. Control bot crawling behavior on your Shopify store with confidence.
Key features
- Edit rules using a structured editor or the raw file.
- Add directive and user-agent group as needed.
- Use the undo or redo button while editing.
- Easily revert to default settings while editing the original file.
- In-built validator to check custom robots.txt rules syntax in real-time.
Pricing of MIT Custom Robots.txt Rules
Basic
$2.99 / month
or $29.99/year and save 16%
- Customize the robots.txt rules
