日期:2014-05-16  浏览次数:20463 次

通过URL接口定时读取其返回的xml(可存入数据库)
之前都是直接读取xml文件的,可是这次做东西对方只提供一个URL返回xml,我需要定时读取。
解决方案:
1:通过URL和URLConnection来解析改URL,使其返回inputstream
2:用dom4j解析该inputstream,将解析的数据封装成需要的对象,来存入一个List或Map
3:定义一个Controller来执行该过程
4:用crontab来执行定时任务(linux下,执行crontab -e进行编辑)

@Get("/load")
	public String loadDataFromXmlByDate(Invocation inv, @Param("begin") Date datebegin,
			@Param("end") Date dateend) {
		String begin = DateTimeUtil.getFormatDateTime(datebegin, "yyyyMMdd");
		String end = DateTimeUtil.getFormatDateTime(dateend, "yyyyMMdd");
//		String ip = inv.getRequest().getRemoteAddr();
		String URL = WbConstant.URL + WbConstant.URL_DATE_BEGIN + begin
				+ WbConstant.URL_DATE_END + end;
		List<Statistic> stats = parseXML(URL);
		for (int i = 0; i < stats.size(); i++) {
			Statistic st = stats.get(i);
			st.setTime(datebegin);
			//通过service层来执行逻辑并存储到DB
			IResult result = dataService.createStatistic(st);
			if (result.isSuccess()) {
				if (logger.isInfoEnabled()) {
					logger.info("insert data into db from xml success");
				}
			} else {
				if (logger.isInfoEnabled()) {
					logger.info(result.getResult("error").toString());
				}
			}
		}
		return "ok";
	}
	/**
	 * 加载默认日期的xml
	 * @param inv
	 * @return
	 */
	@Get("/loadDefault")
	public String defaultLoadDateFromXml(Invocation inv) {
		Date today = new Date();
		Calendar cal = Calendar.getInstance();
		cal.add(Calendar.DATE, -1);
		Date yesterday = cal.getTime();
		return loadDataFromXmlByDate(inv, yesterday, today);
	}

	private List<Statistic> parseXML(String URL) {
		long beginTime = System.currentTimeMillis();
		List<Statistic> xmlList = new ArrayList<Statistic>();
		SAXReader reader = new SAXReader();
		URL url = null;
		try {
			url = new URL(URL);
		} catch (MalformedURLException e) {
			logger.error("给定的URL生成错误", e);
		}
		try {
			URLConnection urlConnection = url.openConnection();
			InputStream urlIn = urlConnection.getInputStream();
			doc = reader.read(urlIn);
		} catch (IOException e) {
			logger.error("IOException:通过URL-openConnection-getInputStream时,错误",
					e);
		} catch (DocumentException e) {
			logger.error("DocumentExcepion:SAXReader解析inputsream为Document时,错误",
					e);
		}
		Element root = doc.getRootElement();
		for (Iterator<Element> it = root.elementIterator(); it.hasNext();) {
			Element e = (Element) it.next();
			String date = e.elementText("date");
			String pid = e.elementText("pid");
			String data = e.elementText("data");
			//封装数据为对象
			Statistic stat = new Statistic();
			stat.setHomeCount(Integer.parseInt(data));
			stat.setPid(pid);
			stat.setTime(DateTimeUtil.getFormatDate(date));
			xmlList.add(stat);
		}
		long endTime = System.currentTimeMillis();
		System.out.println("共花费时间为" + (endTime - beginTime) + "ms");
		return xmlList;
	}


上面的controller写完了,就可以用crontab来执行定时任务了,该controllerURL为,xxx.com/loadDefault则,定为每天下午2点钟执行一次如下
00 14 * * * wget -a /data/logs/wb/xml.log http://xxx.com/loadDefault
OK了,可以了