# Robots.txt generator with sitemap

Build a robots.txt file with user-agent rules and your sitemap locations.

Canonical page: https://aismarty.com/tools/sitemap-robots-generator

## Purpose

Write a robots.txt file with user-agent groups, allow and disallow rules and the location of your sitemaps.

## How to use this tool

1. Add a user agent and the paths it should not crawl, plus any paths it may.
2. Add the full URL of each sitemap, then choose Generate robots.txt.
3. Review the warnings, download robots.txt and place it at the root of your site.

## Example inputs


## Limits and what to check

It runs in your browser and nothing is uploaded. Line breaks and comment characters are refused in values so a rule cannot inject another directive. robots.txt is a request to compliant crawlers, not access control, and it does not guarantee indexing or ranking.

## Frequently asked questions

### Does robots.txt hide or protect pages?

No. It is a request that well-behaved crawlers follow. It is not access control, and a blocked URL can still appear in search results if other pages link to it. Use authentication or a noindex tag instead.

### Why am I warned about Disallow: /?

That rule asks every compliant crawler to stay away from your whole site. It is rarely what a public website wants, so check it before you publish.

### Is my robots.txt created on a server?

No. It is written in your browser and nothing is sent to AI Smarty. Copy or download the file when you are happy with it.

### Will a robots.txt file improve my rankings?

No. It only tells compliant crawlers which paths to avoid and where your sitemaps are. Search engines still decide what to index and rank.

## Related tools

- [Sitemap splitter and merger](https://aismarty.com/tools/sitemap-split-merger)
- [Sitemap URL extractor](https://aismarty.com/tools/sitemap-url-extractor)
- [Sitemap URL comparison](https://aismarty.com/tools/sitemap-urls-comparison)
- [All tools](https://aismarty.com/tools)