Question

php file_get_contents及其curl 无法抓取网站某一页

php file_get_contents及其curl 无法抓取网站某一页 比如 20页以前可以正常 抓取 大于20页抓取内容就变成20页的内容了 在线等·····

Answer 1

估计是COOKIE在作怪，我在网页打开22页，嗅到的调用是：

http://house.focus.cn/search/0_0_0_0_0_0_0_0_0.html?&page=22&allpage=
----------------------------------------------------------------------
GET /search/0_0_0_0_0_0_0_0_0.html?&page=22&allpage= HTTP/1.1

Accept-Language: zh-CN,zh;q=0.8

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Referer: http://house.focus.cn/search/0_0_0_0_0_0_0_0_0.html?&page=21&allpage=

DNT: 1

User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36

Accept-Encoding: gzip, deflate

Host: house.focus.cn

Connection: Keep-Alive

Cookie: IPLOC=unknown; PHPSESSID=56a959254d81b6274085ebdd567796b3; sohutag=8HsmeSc5NCwmcyc5NCwmYjc5NCwmYSc5NCwmZjc5MCwmZyc5NCwmbjc5NCwmaSc5NCwmdyc5NCwmaCc5NCwmYyc5NCwmZSc5NCwmbSc5NH0; SUV=1311281109462085; __utma=1.150856136.1385608204.1385608204.1385608204.1; __utmb=1.4.10.1385608204; __utmc=1; __utmz=1.1385608204.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); PHPSESSID=56a959254d81b6274085ebdd567796b3

Answer 2

那肯定程序有问题，上代码

追问

就一个php函数 page写死都不行  就能到20也页 以后抓取的全是20页内容  写死为大于20的页码 也为20页内容 不信你抓下试试 我估计他们网站不知道怎么弄得

追答

那就是他们的网站，要么是有缓存，要么是直接不让抓