| | |
|
s | |
 |
scrapy |
|
|
scrapy.contracts |
|
|
scrapy.contracts.default |
|
|
scrapy.core.scheduler |
|
|
scrapy.crawler |
Scrapy 爬虫 |
|
scrapy.downloadermiddlewares |
|
|
scrapy.downloadermiddlewares.cookies |
Cookies 下载器中间件 |
|
scrapy.downloadermiddlewares.defaultheaders |
默认请求头下载器中间件 |
|
scrapy.downloadermiddlewares.downloadtimeout |
下载超时中间件 |
|
scrapy.downloadermiddlewares.httpauth |
HTTP 认证下载器中间件 |
|
scrapy.downloadermiddlewares.httpcache |
HTTP 缓存下载器中间件 |
|
scrapy.downloadermiddlewares.httpcompression |
HTTP 压缩中间件 |
|
scrapy.downloadermiddlewares.httpproxy |
HTTP 代理中间件 |
|
scrapy.downloadermiddlewares.offsite |
站外中间件 |
|
scrapy.downloadermiddlewares.redirect |
重定向中间件 |
|
scrapy.downloadermiddlewares.retry |
重试中间件 |
|
scrapy.downloadermiddlewares.robotstxt |
robots.txt 中间件 |
|
scrapy.downloadermiddlewares.stats |
下载器统计中间件 |
|
scrapy.downloadermiddlewares.useragent |
用户代理中间件 |
|
scrapy.exceptions |
Scrapy 异常 |
|
scrapy.exporters |
项目导出器 (Item Exporters) |
|
scrapy.extensions.closespider |
关闭爬虫扩展 |
|
scrapy.extensions.corestats |
核心统计收集 |
|
scrapy.extensions.debug |
用于调试 Scrapy 的扩展 |
|
scrapy.extensions.httpcache |
|
|
scrapy.extensions.logcount |
基本统计日志记录 |
|
scrapy.extensions.logstats |
基本统计日志记录 |
|
scrapy.extensions.memdebug |
内存调试器扩展 |
|
scrapy.extensions.memusage |
内存使用扩展 |
|
scrapy.extensions.periodic_log |
定期统计日志记录 |
|
scrapy.extensions.spiderstate |
爬虫状态扩展 |
|
scrapy.extensions.telnet |
Telnet 控制台 |
|
scrapy.http |
Request 和 Response 类 |
|
scrapy.item |
Item 和 Field 类 |
|
scrapy.link |
链接提取器中的链接 |
|
scrapy.linkextractors |
链接提取器类 |
|
scrapy.linkextractors.lxmlhtml |
基于 lxml HTMLParser 的链接提取器 |
|
scrapy.loader |
Item 加载器类 |
|
scrapy.pipelines.files |
文件管道 |
|
scrapy.pipelines.images |
图像管道 |
|
scrapy.robotstxt |
robots.txt 解析器接口和实现 |
|
scrapy.selector |
Selector 类 |
|
scrapy.settings |
设置管理器 |
|
scrapy.signalmanager |
信号管理器 |
|
scrapy.signals |
信号定义 |
|
scrapy.spiderloader |
爬虫加载器 |
|
scrapy.spidermiddlewares |
|
|
scrapy.spidermiddlewares.base |
|
|
scrapy.spidermiddlewares.depth |
深度爬虫中间件 |
|
scrapy.spidermiddlewares.httperror |
HTTP 错误爬虫中间件 |
|
scrapy.spidermiddlewares.referer |
Referer 爬虫中间件 |
|
scrapy.spidermiddlewares.start |
|
|
scrapy.spidermiddlewares.urllength |
URL 长度爬虫中间件 |
|
scrapy.statscollectors |
统计收集器 |
|
scrapy.utils.log |
日志工具 |
|
scrapy.utils.trackref |
跟踪活动对象的引用 |