Question

scrapy 提取不在标签内的文字

我想抓取的内容为
“”黄念祖：亚运会劫机事件中只有三个人活着，出事时都在念佛。”
可是我用了 response.xpath('//[@id="result"]/div[3]/div[1]/h2/a/text()').extract()只能抓到
”黄念祖：亚运会，事件中只有三个人活着，出事时都在念佛“
劫机没了，还多了个不想要的逗号，求问如何改进
http://search.sina.com.cn/?by=title&q=%BD%D9%BB%FA&c=blog&range=article&col=&source=&from=&country=&size=&time=&a=&sort=time
这是原网址

Answer 1

代码如下

def parse(self,response):
states = {}
list1 = []
list2 = []

for row in response.xpath("//*[@id='info']/*"):
if row.xpath("span[@class='pl']/text()"):
title = row.xpath("span[@class='pl']/text()").extract()[0].strip()
text = row.xpath("a/text()").extract()[0].strip()
states[title]=text
elif row.xpath("text()"):
list1.append(row.xpath("text()").extract()[0].strip()[:-1])

for row in response.xpath("//*[@id='info']/text()").extract():
if row.strip():
list2.append(row.strip())

for i in range(len(list1)):
states[list1[i]]=list2[i]

for n in states:
print n,states[n]