How to Block Robots from Crawling Your Angular Application

If you don't want robots crawling your application you can easily setup a robots.txt file in your Angular CLI Application. First we need to create a txt file to live at /src/robots.txt, this file should contain the below code to block all robots from crawling all of your application.

User-agent: *
Disallow: /

Next we need to tell the Angular CLI to package this file in our /dist folder when we run ng build. Let's add the below snippet to our angular-cli.json file which can be found in the root of our application.

{
  "apps": [
    {
      "assets": [
        "robots.txt"
      ]
    }
  ]
}

It's as easy as that! Now when you run ng build, the Angular CLI, and more specifically Webpack will add the robots.txt file to your /dist folder and block robots from crawling and indexing your application.

Add new comment