NoimaBot
Noima reads public web pages on behalf of the companies that use it — a page a customer asked it to watch, or a public source such as a news feed or job board. This is exactly how it behaves, and how to stop it.
It identifies itself on every request
Requests are sent as NoimaBot/1.0 (+https://noima.dev/bot), including the request for robots.txt itself. Noima does not disguise its crawler as a browser.
It obeys robots.txt, and errs towards not fetching
Rules are read per RFC 9309 and cached for a day. If robots.txt cannot be read — a server error, a rate limit, or a network failure — NoimaBot treats the whole site as disallowed rather than assuming permission. A disallowed site is dropped immediately, not retried.
One file is exempt, and only one
If you publish /.well-known/noima-site-verification.txt to prove you control a domain, NoimaBot fetches that one file without consulting robots.txt — because it exists only because you created it at Noima's request and asked for it to be checked. That is how ACME and Google's site-verification files are fetched too. Nothing else is exempt: your pages, feeds and sitemaps are all still governed by robots.txt, so blocking NoimaBot still stops Noima reading your site.
It waits between requests
At least five seconds between requests to the same host, and longer when robots.txt asks for more through Crawl-delay.
It asks whether anything changed
Requests carry If-None-Match and If-Modified-Since from the previous visit, so an unchanged page costs your server a 304 and no body.
It reads a bounded amount
A 15-second timeout, at most 3 MB per page, and at most three redirects — each redirect checked again before it is followed. Only public internet addresses are contacted.
It gives up rather than hammering
Repeated failures back off exponentially, and a source that keeps failing is paused.
It does not pretend to be a person
No JavaScript is executed, no forms are submitted, no cookies or sessions are kept, and it never signs in. NoimaBot reads what an anonymous visitor would be served.
How to block it
Add this to your robots.txt. NoimaBot re-reads robots.txt at least once a day, and stops as soon as it sees the rule.
User-agent: NoimaBot
Disallow: /To allow only part of a site, use ordinary robots.txt rules — NoimaBot follows them. If something looks wrong, write to bot@noima.dev and include the requesting address and the times, and we will look into it.