cdransf/ai.robots.txt
1
1
Fork 0
mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-06-19 18:10:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge 6c3ae6eb20 into eb05f2f527

Browse source
This commit is contained in:
Falko Zurell 2025-06-14 12:26:56 -04:00 • committed by GitHub
commit d99c2bb182
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 38 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -44,6 +44,7 @@ Note that, as stated in the [httpd documentation](https://httpd.apache.org/docs/
middleware plugin for [Traefik](https://traefik.io/traefik/) to automatically add rules of [robots.txt](./robots.txt)
file on-the-fly.
- alternatively you can [manually configure Traefik](./docs/traefik-manual-setup.md) to centrally serve a static `robots.txt`
## Contributing
A note about contributing: updates should be added/made to `robots.json`. A GitHub action will then generate the updated `robots.txt`, `table-of-bot-metrics.md`, `.htaccess` and `nginx-block-ai-bots.conf`.