Question

如何用python的urlopen打开自动跳转的网页

我想抓取下http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2009-2403的网页源码，使用了python，urllib2库中的urlopen，但是这个网页会自动跳转，打开网页失败，各位大牛有什么办法能解决这个问题啊？谢啦谢啦

Answer 1

try this:

import sys,re,urllib2,cookielib
def download(url):
____opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()))
____opener.addheaders = [('User-agent', 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)')]
____f = opener.open(url)
____s = f.read()
____f.close()
____return s
s = download("http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2009-2403")
print s