ai.robots.txt

cdransf/ai.robots.txt

mirror of https://github.com/ai-robots-txt/ai.robots.txt.git synced 2025-06-19 10:00:52 +00:00

Author	SHA1	Message	Date
Cory Dransfeldt	14d68f05ba	chore(robots.json): adds additional SemrushBot user agents	2025-06-11 13:50:53 -07:00
dark-visitors	cf598b6b71	Update from Dark Visitors Some checks failed / ai-robots-txt (push) Has been cancelled Details / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details	2025-06-10 01:00:37 +00:00
ai.robots.txt	3759a6bf14	chore(robots.json): adds EchoboxBot (#148 ) Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details	2025-06-09 15:44:36 +00:00
Cory Dransfeldt	7867c3e26c	chore(robots.json): adds EchoboxBot (#148 )	2025-06-09 16:44:25 +01:00
dark-visitors	e21f6ae1b6	Update from Dark Visitors Some checks failed / ai-robots-txt (push) Has been cancelled Details / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details	2025-06-06 00:59:25 +00:00
ai.robots.txt	ac7ed17e71	Merge pull request #145 from ai-robots-txt/aws-bedrockbot Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): adds bedrockbot	2025-06-05 16:51:17 +00:00
Glyn Normington	81747e6772	Merge pull request #145 from ai-robots-txt/aws-bedrockbot chore(robots.json): adds bedrockbot	2025-06-05 17:51:03 +01:00
Cory Dransfeldt	528d77bf07	chore(robots.json): adds bedrockbot	2025-06-05 09:14:23 -07:00
dark-visitors	77393df5aa	Update from Dark Visitors Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details	2025-06-05 00:59:28 +00:00
ai.robots.txt	75ea75a95b	Merge pull request #143 from ai-robots-txt/panscient Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): adds Panscient	2025-06-04 18:04:06 +00:00
Glyn Normington	2fca1ddcf1	Merge pull request #143 from ai-robots-txt/panscient chore(robots.json): adds Panscient	2025-06-04 19:03:53 +01:00
ai.robots.txt	9c28c63a0c	Merge pull request #142 from ai-robots-txt/quillbot chore(robots.json): adds Quillbot	2025-06-04 17:54:57 +00:00
Cory Dransfeldt	395c013eea	Merge pull request #142 from ai-robots-txt/quillbot chore(robots.json): adds Quillbot	2025-06-04 10:54:46 -07:00
Cory Dransfeldt	4568d69b0e	chore(robots.json): adds Panscient	2025-06-04 10:54:14 -07:00
Cory Dransfeldt	03831a7eb5	chore(robots.json): adds Quillbot	2025-06-04 10:46:58 -07:00
dark-visitors	2b5a59a303	Update from Dark Visitors Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details	2025-06-04 01:00:07 +00:00
ai.robots.txt	3efabc603d	Merge pull request #141 from Ivan-Chupin/patch-1 Add SBIntuitionsBot	2025-06-03 23:28:48 +00:00
Cory Dransfeldt	b35f9a31d7	Merge pull request #141 from Ivan-Chupin/patch-1 Add SBIntuitionsBot	2025-06-03 16:28:36 -07:00
Ivan Chupin	8f75f4a2f5	Add SBIntuitionsBot	2025-06-04 03:48:42 +05:00
ai.robots.txt	080946c360	Merge pull request #140 from ai-robots-txt/yandex-bots Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): adds YandexAdditional crawlers	2025-06-03 19:51:25 +00:00
Glyn Normington	7eec033cad	Merge pull request #140 from ai-robots-txt/yandex-bots chore(robots.json): adds YandexAdditional crawlers	2025-06-03 20:51:14 +01:00
Cory Dransfeldt	3187fd8a32	chore(robots.json): adds YandexAdditional crawlers	2025-06-03 12:41:57 -07:00
ai.robots.txt	d239e7e5ad	Merge pull request #139 from ai-robots-txt/workflow-fix Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(ai_robots_update.yml): correct workflow by revising git flags + adding guard	2025-06-03 01:52:35 +00:00
Glyn Normington	9dbf34010a	Merge pull request #139 from ai-robots-txt/workflow-fix chore(ai_robots_update.yml): correct workflow by revising git flags + adding guard	2025-06-03 02:52:23 +01:00
dark-visitors	87016d1504	Update from Dark Visitors	2025-06-03 01:00:29 +00:00
Cory Dransfeldt	899ce01c55	chore(ai_robots_update.yml): correct workflow by revising git flags + adding guard	2025-06-02 14:56:09 -07:00
Glyn Normington	4af776f0a0	Merge pull request #136 from ai-robots-txt/imgproxy-revert Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): revert "adds imgproxy crawler"	2025-06-02 20:21:10 +01:00
Cory Dransfeldt	1dd66b6969	Revert "chore(robots.json): adds imgproxy crawler" This reverts commit `b65f45e408`.	2025-06-02 11:53:06 -07:00
Cory Dransfeldt	814df6b9a0	Merge pull request #134 from not-not-the-imp/patch-1 Some checks failed / ai-robots-txt (push) Has been cancelled Details / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details Add AndiBot and PhindBot	2025-05-31 16:03:16 -07:00
Cory Dransfeldt	268922f8f2	Update robots.json	2025-05-31 16:02:05 -07:00
Cory Dransfeldt	4259b25ccc	Update robots.json	2025-05-31 16:01:09 -07:00
Cory Dransfeldt	d22b9ec51a	Update robots.json	2025-05-31 16:00:13 -07:00
imp	3e8edd083e	Add AndiBot and PhindBot Fixes #75	2025-05-23 13:03:49 +01:00
ai.robots.txt	093ab81d78	Update from Dark Visitors Some checks failed / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details	2025-05-23 00:58:57 +00:00
dark-visitors	7bf7f9164d	Update from Dark Visitors Some checks failed / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details / ai-robots-txt (push) Has been cancelled Details	2025-05-22 00:58:45 +00:00
ai.robots.txt	fedb658cc0	Merge pull request #133 from ai-robots-txt/wpbot chore(robots.json): adds wpbot	2025-05-21 21:06:05 +00:00
Glyn Normington	851eabe059	Merge pull request #133 from ai-robots-txt/wpbot chore(robots.json): adds wpbot	2025-05-21 22:05:51 +01:00
ai.robots.txt	7c5389f4a0	Merge pull request #98 from kylebuckingham/main Updating Claude Bots	2025-05-21 19:00:23 +00:00
Cory Dransfeldt	af597586b6	Merge pull request #98 from kylebuckingham/main Updating Claude Bots	2025-05-21 12:00:11 -07:00
Cory Dransfeldt	b1d9a60a38	chore(robots.json): adds wpbot	2025-05-21 11:40:33 -07:00
ai.robots.txt	1c2acd75b7	Merge pull request #126 from ai-robots-txt/mistral-bot Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): adds MistralAI-User/1.0 crawler	2025-05-21 15:27:26 +00:00
Glyn Normington	202d3c3b9a	Merge pull request #126 from ai-robots-txt/mistral-bot chore(robots.json): adds MistralAI-User/1.0 crawler	2025-05-21 16:27:14 +01:00
Glyn Normington	0a78fe1e76	Merge pull request #132 from ai-robots-txt/crawler-policy-update chore(README): updates the opening line of our README to clarify the types of agents we block	2025-05-21 15:13:35 +01:00
Cory Dransfeldt	8b151b2cdc	Update README.md Co-authored-by: Glyn Normington <glyn.normington@gmail.com>	2025-05-21 06:52:36 -07:00
Cory Dransfeldt	8a8001cbec	chore(README): updates the opening line of our README to clarify the types of agents we block	2025-05-20 13:55:25 -07:00
Glyn Normington	fe1267e290	Merge pull request #131 from Mihitoko/mention-x-robots-tag-for-bing Some checks failed / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details Mention X-Robots-Tag header as alternative for bing	2025-05-20 07:52:32 +01:00
Mihitoko	9297c7dfa3	Mention X-Robots-Tag header as alternative for bing	2025-05-20 00:10:05 +02:00
dark-visitors	7a2e6cba52	Update from Dark Visitors Some checks failed / ai-robots-txt (push) Has been cancelled Details / run-tests (push) Has been cancelled Details / lint-json (push) Has been cancelled Details	2025-05-17 00:57:28 +00:00
ai.robots.txt	dd1ed174b7	Merge pull request #129 from ai-robots-txt/google-cloudvertexbot Some checks are pending / ai-robots-txt (push) Waiting to run Details / run-tests (push) Waiting to run Details / lint-json (push) Waiting to run Details chore(robots.json): adds Google-CloudVertexBot	2025-05-16 11:35:15 +00:00
Glyn Normington	89c0fbaf86	Merge pull request #129 from ai-robots-txt/google-cloudvertexbot chore(robots.json): adds Google-CloudVertexBot	2025-05-16 12:35:04 +01:00

1 2 3 4 5 ...

468 commits