Prevent bots & crawlers from accessing Gitlab

Administratorrequested to merge
github/fork/vendruscolo/no-robots into master

Created by: vendruscolo

Since Gitlab is intended to be used for private Git hosting, I think it's reasonably right to prevent bots and crawlers to index Gitlab.

What's your point about this?

Merge request reports