Search results
Documentation related to python3-robot-detection
Package: python3-robot-detection
Description: Detect web crawlers from HTTP UserAgent (Python 3 version)
This Python module detects if a given HTTP User Agent is a web crawler.
It uses the list of registered robots from the robotstxt.org.
This package contains the Python 3 version of the library.
Homepage: https://github.com/rory/robot-detection
copyright | Debian changelog
Other documents: