Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
259 views
in Technique[技术] by (71.8m points)

Is the following syntax in my robots.txt file, blocking all crawlers from indexing my site?

This is currently in my robots.txt file:

User-agent: * # Disallow everyone else Disallow: /

Question-to me, the novice here, it looks like the "Disallow: /" will prevent Google/MSN/etc from indexing my site. Correct?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

The correct syntax of the robots.txt file for your purpose seems to be as follows:

User-agent: * Disallow: /


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...