302fa154f4cd364881a4d5ee3696a314fdf0be1f
This reverts commit d30566d8f6.
This is needed by Lighthouse to fetch robots.txt and it's worth making a
harmless exception for it to work properly.
https://github.com/GoogleChrome/lighthouse/issues/4386
Description