Con gli strumenti di webmaster di goggle ho trovato:
6 pdf nel mio sito con URL limitato da robots.txt. Perchè?
Questo è il mio robots:
Sitemap:
http://www.meccanicaweb.it/index.php?option=com_xmap&sitemap=1&view=xml&no_html=1User-agent: Googlebot
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/
User-agent: Bullseye/1.0
User-agent: Black Hole
User-agent: bot
User-agent: bot-
User-agent: bot/
User-agent: CopyRightCheck
User-agent: CherryPicker
User-agent: crawl
User-agent: DittoSpyder
User-agent: DISCo Pump 3.1
User-agent: EmailCollector
User-agent: EmailSiphon
User-agent: EmailWolf
User-agent: ExtractorPro
User-agent: hloader
User-agent: humanlinks
User-agent: LexiBot .
User-agent: LinkWalker
User-agent: Microsoft URL
User-agent: Microsoft URL Control - 5.01.4511
User-agent: Microsoft URL Control - 6.00.8169
User-agent: NetAttache
User-agent: NetAttache Light 1.1
User-agent: NetMechanic
User-agent: Offline Explorer
User-agent: robot
User-agent: Spider
User-agent: SuperBot
User-agent: SuperBot/2.6
User-agent: Teleport
User-agent: Titan
User-agent: Voila
User-agent: WebBandit
User-agent: WebCopier
User-agent: webcopy
User-agent: Web Image Collector
User-agent: webmirror
User-agent: WebReaper
User-agent: website extractor
User-agent: WinHTTrack
User-agent: shinchakubin
Disallow: /
E' un problema dei pdf?!
