The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
0 Replies
vwebdevelopment
-2
Thu 22 Dec, 2016 12:29 am
@thomaswagon,
SEO stand for Search Engine Optimization which is used for getting traffic from the organic or natural search result on search engine.
0 Replies
vondadejesus
-1
Tue 14 Mar, 2017 12:22 am
robots.txt file is used.
0 Replies
jennypitula
1
Wed 28 Jun, 2017 03:07 am
@thomaswagon,
Yes you can do it from robots.txt file, you have to write just two lines code. For example if https://www.xyz.com is your website and you want to disallow abc.html page then you have to write in robots.txt file
User-agent: *
Disallow: /abc.html