如果您从 Oracle 网站下载或使用 OpenJDK,您的 JRE/JDK 随附的 Oracle 实现 Pattern 类的字符类解析代码中会出现一些奇怪的巫术。我没有检查其他 JVM(尤其是 GNU Classpath)实现如何解析问题中的正则表达式。
从现在开始,任何对Pattern 类及其内部工作的引用都严格限制在Oracle 的实现(参考实现)中。
阅读和理解Pattern 类如何解析嵌套否定需要一些时间,如问题所示。但是,我编写了一个程序1 从Pattern 对象(带有Reflection API)中提取信息以查看编译结果。下面的输出来自于在 Java HotSpot Client VM 版本 1.7.0_51 上运行我的程序。
1:目前,程序是一团糟。当我完成并重构它时,我会用一个链接更新这篇文章。
[^0-9]
Start. Start unanchored match (minLength=1)
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
这没什么好奇怪的。
[^[^0-9]]
Start. Start unanchored match (minLength=1)
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
[^[^[^0-9]]]
Start. Start unanchored match (minLength=1)
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
上面接下来的 2 个案例被编译成与[^0-9] 相同的程序,这是反直觉。
[[^0-9]2]
Start. Start unanchored match (minLength=1)
Pattern.union (character class union). Match any character matched by either character classes below:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
[\D2]
Start. Start unanchored match (minLength=1)
Pattern.union (character class union). Match any character matched by either character classes below:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Ctype. Match POSIX character class DIGIT (US-ASCII)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
如问题所述,上述两种情况都没有什么奇怪的。
[013-9]
Start. Start unanchored match (minLength=1)
Pattern.union (character class union). Match any character matched by either character classes below:
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 2 character(s):
[U+0030][U+0031]
01
Pattern.rangeFor (character range). Match any character within the range from code point U+0033 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
[^\D2]
Start. Start unanchored match (minLength=1)
Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Ctype. Match POSIX character class DIGIT (US-ASCII)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
如问题中所述,这两种情况按预期工作。但是,请注意引擎如何对第一个字符类 (\D) 进行补码,并将集合差异应用于由剩余字符组成的字符类。
[^[^0-9]2]
Start. Start unanchored match (minLength=1)
Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
[^[^[^0-9]]2]
Start. Start unanchored match (minLength=1)
Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
[^[^[^[^0-9]]]2]
Start. Start unanchored match (minLength=1)
Pattern.setDifference (character class subtraction). Match any character matched by the 1st character class, but NOT the 2nd character class:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
LastNode
Node. Accept match
Keppil 在评论中通过测试确认,上面的输出显示上面的所有 3 个正则表达式都编译到同一个程序中!
[^2[^0-9]]
Start. Start unanchored match (minLength=1)
Pattern.union (character class union). Match any character matched by either character classes below:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
我们得到UNION(NOT(2), NOT(0-9)),而不是NOT(UNION(2, NOT(0-9)),也就是0-13-9,相当于NOT(2)。
[^2[^[^0-9]]]
Start. Start unanchored match (minLength=1)
Pattern.union (character class union). Match any character matched by either character classes below:
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
BitClass. Optimized character class with boolean[] to match characters in Latin-1 (code point <= 255). Match the following 1 character(s):
[U+0032]
2
CharProperty.complement (character class negation). Match any character NOT matched by the following character class:
Pattern.rangeFor (character range). Match any character within the range from code point U+0030 to code point U+0039 (both ends inclusive)
LastNode
Node. Accept match
由于相同的错误,正则表达式[^2[^[^0-9]]] 编译为与[^2[^0-9]] 相同的程序。
有一个似乎具有相同性质的未解决错误:JDK-6609854。
说明
初步
下面是Pattern类的实现细节,在进一步阅读之前应该知道:
-
Pattern 类将String 编译成一个节点链,每个节点负责一个小而明确的职责,并将工作委托给链中的下一个节点。 Node 类是所有节点的基类。
-
CharProperty 类是所有与字符类相关的Nodes 的基类。
-
BitClass 类是 CharProperty 类的子类,它使用 boolean[] 数组来加速匹配 Latin-1 字符(代码点 add 方法,允许在编译期间添加字符。
-
CharProperty.complement、Pattern.union、Pattern.intersection 是对应于集合操作的方法。他们所做的事情是不言自明的。
-
Pattern.setDifference 是 asymmetric set difference。
乍一看解析字符类
在看CharProperty clazz(boolean consume)方法的完整代码之前,它是负责解析一个字符类的方法,让我们看一个极其简化的代码版本来理解代码的流程:
private CharProperty clazz(boolean consume) {
// [Declaration and initialization of local variables - OMITTED]
BitClass bits = new BitClass();
int ch = next();
for (;;) {
switch (ch) {
case '^':
// Negates if first char in a class, otherwise literal
if (firstInClass) {
// [CODE OMITTED]
ch = next();
continue;
} else {
// ^ not first in class, treat as literal
break;
}
case '[':
// [CODE OMITTED]
ch = peek();
continue;
case '&':
// [CODE OMITTED]
continue;
case 0:
// [CODE OMITTED]
// Unclosed character class is checked here
break;
case ']':
// [CODE OMITTED]
// The only return statement in this method
// is in this case
break;
default:
// [CODE OMITTED]
break;
}
node = range(bits);
// [CODE OMITTED]
ch = peek();
}
}
代码基本上读取输入(输入String 转换为null-terminated int[] 的代码点)直到它到达] 或字符串的结尾(未封闭的字符类)。
代码有点混淆continue 和break 在switch 块内混合在一起。不过,只要你意识到continue属于外部for循环,break属于switch块,代码就很容易理解了:
- 以
continue 结尾的案例将永远不会执行switch 语句之后的代码。
- 以
break 结尾的情况可以执行switch 语句之后的代码(如果它还没有return)。
通过上面的观察,我们可以看到,每当发现一个字符是非特殊的并且应该包含在字符类中时,我们都会执行switch语句之后的代码,其中node = range(bits); 是第一条语句。
如果检查source code,方法CharProperty range(BitClass bits) 会解析“字符类中的单个字符或字符范围”。该方法要么返回传入的相同BitClass 对象(添加了新字符),要么返回CharProperty 类的新实例。
血淋淋的细节
接下来我们看完整版代码(省略部分解析字符类交集&&):
private CharProperty clazz(boolean consume) {
CharProperty prev = null;
CharProperty node = null;
BitClass bits = new BitClass();
boolean include = true;
boolean firstInClass = true;
int ch = next();
for (;;) {
switch (ch) {
case '^':
// Negates if first char in a class, otherwise literal
if (firstInClass) {
if (temp[cursor-1] != '[')
break;
ch = next();
include = !include;
continue;
} else {
// ^ not first in class, treat as literal
break;
}
case '[':
firstInClass = false;
node = clazz(true);
if (prev == null)
prev = node;
else
prev = union(prev, node);
ch = peek();
continue;
case '&':
// [CODE OMITTED]
// There are interesting things (bugs) here,
// but it is not relevant to the discussion.
continue;
case 0:
firstInClass = false;
if (cursor >= patternLength)
throw error("Unclosed character class");
break;
case ']':
firstInClass = false;
if (prev != null) {
if (consume)
next();
return prev;
}
break;
default:
firstInClass = false;
break;
}
node = range(bits);
if (include) {
if (prev == null) {
prev = node;
} else {
if (prev != node)
prev = union(prev, node);
}
} else {
if (prev == null) {
prev = node.complement();
} else {
if (prev != node)
prev = setDifference(prev, node);
}
}
ch = peek();
}
}
查看switch语句的case '[':中的代码和switch语句之后的代码:
-
node 变量存储解析 unit(独立字符、字符范围、速记字符类、POSIX/Unicode 字符类或嵌套字符类)的结果李>
-
prev 变量存储了到目前为止的编译结果,并且总是在我们编译 node 中的 unit 后立即更新。
由于记录字符类是否取反的局部变量boolean include,从来没有传递给任何方法调用,所以只能在这个方法中单独作用。唯一读取和处理include 的位置是在switch 语句之后。
正在建设中