直接在spider文件的初始化位置加上custom_setting={}这样的字段即可,这个字段是个字典类型,可以选择性的覆盖主setting文件中的某些设置,来达到特殊spider使用特殊设置的要求
class JdMonitorSpider(scrapy.Spider):
name = 'jd_monitor'
allowed_domains = ['jd.com']
setting = get_project_settings()
custom_settings = {
"CONCURRENT_REQUESTS": 3,
"DOWNLOAD_DELAY ": random.randint(10, 20)
}
