User-agent: * Disallow: /docs/ Disallow: /*.php$ Disallow: /cgi-bin/ Disallow: /flash_demo.html