This reverts commit d30566d8f6.
This is needed by Lighthouse to fetch robots.txt and it's worth making a
harmless exception for it to work properly.
https://github.com/GoogleChrome/lighthouse/issues/4386
1.8 KiB
1.8 KiB