robot.txt files

Do you have a robots.txt file set-up on your server? Here's what Google says in their Webmaster Guidelines: 

You need a robots.txt file only if your site includes content that you don't want search engines to index. If you want search engines to index everything in your site, you don't need a robots.txt file. 

Visit http://www.robotstxt.org/faq/what.html or http://www.google.com/support/webmasters/bin/answer.py?answer=40362 to learn more about robots when they visit your site. 

Writing a robots.txt file is, as you have seen, a relatively simple matter. However it is important to bear in mind that robot.txt files are not a security tool. They may be used to stop specified pages from appearing in search engines indices.  There are many hundreds of bots and spiders crawling the Internet now and while most will respect your robot.txt file, some will not and there are even some designed specifically to visit the very pages you are specifying as being out of bounds.