What should be in robots txt?

Related questions

1. How does robots txt work?
2. How long does it take for robots txt to take effect?
3. What is http mj12bot com?
4. What is robots txt in SEO?
5. Do I need robots txt on my site?
6. What are search bots?
7. What is Bingbot htm?
8. Should Sitemap be in robots txt?
9. What is crawl delay in robots txt?
10. Does Yoast create a sitemap?

Tags

1. #disallow
2. #agent
3. #their
4. #called
5. #instructions
6. #protocol
7. #nutshell
8. #section
9. #robots
10. #exclusion
11. #pages
12. #applies
13. #about
14. #robot
15. #tells
16. #should
17. #means
18. #owners
19. #visit