User-agent: crawl-it disallow /menue/ disallow /seitentitel.htm disallow /index.htm