java剔除(替换)不可见的unicode/utf-8字符（主要是html显示不了的字符）
日期：2014-05-17 浏览次数：20876 次
java删除(替换)不可见的unicode/utf-8字符（主要是html显示不了的字符）
今天遇到一个问题，由于编辑人员从excel等7788的地方copy内容过来，其中有不可见的字符，导致输出内容看上去是对的，其实是多了一个零长度的字符（比如：0000200B ZERO WIDTH SPACE），下面的代码基本解决了以上问题。
?
?
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<script>
window.onload=function(){
var zero_len_charcode=[]
var div = document.getElementById('ttt')
for(var i=0;i<=0xffff;i++){//
var c = String.fromCharCode(i);
if (!c) continue;
div.innerHTML = "|"+c+"|";
if (div.offsetWidth == 16) //这里的16是我机器的2个'||'的显示长度，请自己调节
zero_len_charcode.push(i.toString(16));
}
console.log('0x'+zero_len_charcode.join(",0x"))
}
</script>
</head>
<body>
<span id=ttt></span>
</body>
</html>
?上面的js代码用于提取出看不见的字符（主要是在html上看不见，也就是前面说的0长度的），copy上面console输出的内容，然后去java里做服务器端处理（为什么这么麻烦，因为swing我不熟悉，而且也不一定可以做这样的长度适应判断）
?
private static String zerolize(String s) {
if (s.length() < 4) {
s = "000".substring(0, 4 - s.length()) + s;
}
return s;
}
public static void main(String[] args) throws IOException {
int[] input = new int[] { 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a,
0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99,
0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, 0xad, 0x483, 0x484, 0x485, 0x486, 0x487, 0x488, 0x489,
0x559, 0x55a, 0x58a, 0x591, 0x592, 0x593, 0x594, 0x595, 0x596, 0x597, 0x598, 0x599, 0x59a,
0x59b, 0x59c, 0x59d, 0x59e, 0x59f, 0x5a0, 0x5a1, 0x5a2, 0x5a3, 0x5a4, 0x5a5, 0x5a6, 0x5a7,
0x5a8, 0x5a9, 0x5aa, 0x5ab, 0x5ac, 0x5ad, 0x5ae, 0x5af, 0x5b0, 0x5b1, 0x5b2, 0x5b3, 0x5b4,
0x5b5, 0x5b6, 0x5b7, 0x5b8, 0x5b9, 0x5ba, 0x5bb, 0x5bc, 0x5bd, 0x5bf, 0x5c1, 0x5c2, 0x5c4,
0x5c5, 0x5c6, 0x5c7, 0x606, 0x607, 0x608, 0x609, 0x60a, 0x63b, 0x63c, 0x63d, 0x63e, 0x63f,
0x674, 0x6e5, 0x6e6, 0x70f, 0x76e, 0x76f, 0x770, 0x771, 0x772, 0x773, 0x774, 0x775, 0x776,
0x777, 0x778, 0x779, 0x77a, 0x77b, 0x77c, 0x77d, 0x77e, 0x77f, 0xa51, 0xa75, 0xb44, 0xb62,
0xb63, 0xc62, 0xc63, 0xce2, 0xce3, 0xd62, 0xd63, 0x135f, 0x200b, 0x200c, 0x200d, 0x200e,
0x200f, 0x2028, 0x2029, 0x202a, 0x202b, 0x202c, 0x202d, 0x202e, 0x2044, 0x2071, 0xf701,
0xf702, 0xf703, 0xf704, 0xf705, 0xf706, 0xf707, 0xf708, 0xf709, 0xf70a, 0xf70b, 0xf70c,
0xf70d, 0xf70e, 0xf710, 0xf711, 0xf712, 0xf713, 0xf714, 0xf715, 0xf716, 0xf717, 0xf718,
0xf719, 0xf71a, 0xfb1e, 0xfc5e, 0xfc5f, 0xfc60, 0xfc61, 0xfc62, 0xfeff, 0xfffc };
StringBuilder b = new StringBuilder();
int lastContinuous = -1;
int span = 0;
for (int i = 0; i < input.length; i++) {
if (lastContinuous == -1 && i < input.length - 1 && input[i] + 1 == input[i + 1]) {
lastContinuous = input[i];
span = 1;
} else {
if (input[i] == lastContinuous + span) {
span++;
} else if (lastContinuous != -1) {
if (b.length() > 0)
b.append("|");
b.append(String.format("[\\u%s-\\u%s]", zerolize(Integer.toHexString(lastContinuous)),
zerolize(Integer.toHexString(lastContinuous + span - 1))));
span = 0;
lastContinuous = -1;
i--;
} else {
b.append("|\\u" + zerolize(Integer.toHexString(input[i])));
}
}
}
if (lastContinuous != -1) {
if (b.length() > 0)
b.append("|");
b.append(String.format("[\\u%s-\\u%s]", zerolize(Integer.toHexString(lastContinuous)),
zerolize(Integer.toHexString(lastContinuous + span - 1))));
}
System.out.println(b.toString());
Pattern pattern = Pattern.compile(b.toString());
Matcher matcher = pattern.matcher("helloworld\ufffc撒范德萨分的阿萨德");
System.out.println(matcher.replaceAll(""));
}
?
最后组成的正则如下：
[\u007f-\u009f]|\u00ad|[\u0483-\u0489]|[\u0559-\u055a]|\u058a|[\u0591-\u05bd]|\u0
免责声明： 本文仅代表作者个人观点，与爱易网无关。其原创性以及文中陈述文字和内容未经本站证实，对本文以及其中全部或者部分内容、文字的真实性、完整性、及时性本站不作任何保证或承诺，请读者仅作参考，并请自行核实相关内容。
