使用HttpURLConnection抓取google搜索结果,总是报错:
java.net.SocketException: Connection reset抓取百度或者其他网站数据没问题,可是谷歌的总是报错:java.net.
SocketException: Connection reset抓取代码如下:
HttpURLConnection connection = null;
BufferedInputStream in = null;
BufferedReader read = null;
try {
URL url = new URL(strURL);
String cookie = "";
int iii=0;
do {
iii++;
connection = (HttpURLConnection)url.openConnection();
System.out.println("11111111:"+strURL);
Thread.sleep(5000);
if(cookie.length() != 0) {
connection.setRequestProperty("Cookie", cookie);
}
connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0)");
connection.setInstanceFollowRedirects(false);
System.out.println("9999999999"+connection.getHeaderField("Set-Cookie"));
int code = connection.getResponseCode();
if(code == HttpURLConnection.HTTP_MOVED_TEMP) {
cookie += connection.getHeaderField("Set-Cookie") + ";";
}
if((connection.getResponseCode() == HttpURLConnection.HTTP_OK) || iii==4)
break;
}
while(true);
in = new BufferedInputStream(connection.getInputStream());
read = new BufferedReader(new InputStreamReader(in,"GB2312"));
------解决方案--------------------
连接重置,意思是说你的链接设置非法。可能谷歌有特别的设置吧。
------解决方案--------------------貌似要申请一个谷歌的一个账号吧 不然就会有这个问题
------解决方案--------------------
google是做Web服务起家的,你想用程序获取人家服务的结果,google能让你这么轻易拿劳动成果吗?
google map,google translate,google doc...几乎所有的google产品你都可以集成到你自己的web应用中,但是:要给钱!!!!
记得以前集成google map的地图搜索的时候,社区版还是免费的,但是一天内只能搜索20次....
------解决方案--------------------不是吧。是不是你有代理没配什么的。我自己写的抓取的小例子,怎么抓都行啊。。。