PHP内核的op_array由编译时转化为运行时

PHP内核在pass_two这个函数里面,会对op_array进行一个编译时到运行时的转化,其中一个就是重新分配literals。让literalsopcodes由原来分散存储的内存合并为连续的一块内存。这么做的其中一个好处是,在执行opline的时候,直接通过偏移量就可以拿到对应的字面量了,不需要传递op_array,相当于少传递了一个参数(之前需要通过op_array->literals的方式来获取)。

PHP内核如何确定一个opcode有几个操作数

首先,PHP内核包含的所有zend_ast节点类型在文件zend_ast.h里面:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
#define ZEND_AST_SPECIAL_SHIFT      6
#define ZEND_AST_IS_LIST_SHIFT 7
#define ZEND_AST_NUM_CHILDREN_SHIFT 8

enum _zend_ast_kind {
// 省略其他的节点类型

/* 0 child nodes */
ZEND_AST_MAGIC_CONST = 0 << ZEND_AST_NUM_CHILDREN_SHIFT,
ZEND_AST_TYPE,
ZEND_AST_CONSTANT_CLASS,

/* 1 child node */
ZEND_AST_VAR = 1 << ZEND_AST_NUM_CHILDREN_SHIFT,
ZEND_AST_CONST,
ZEND_AST_UNPACK,
ZEND_AST_UNARY_PLUS,
ZEND_AST_UNARY_MINUS,
ZEND_AST_CAST,
ZEND_AST_EMPTY,
ZEND_AST_ISSET,
ZEND_AST_SILENCE,
ZEND_AST_SHELL_EXEC,
ZEND_AST_CLONE,
ZEND_AST_EXIT,
ZEND_AST_PRINT,
ZEND_AST_INCLUDE_OR_EVAL,
ZEND_AST_UNARY_OP,
ZEND_AST_PRE_INC,
ZEND_AST_PRE_DEC,
ZEND_AST_POST_INC,
ZEND_AST_POST_DEC,
ZEND_AST_YIELD_FROM,
ZEND_AST_CLASS_NAME,

ZEND_AST_GLOBAL,
ZEND_AST_UNSET,
ZEND_AST_RETURN,
ZEND_AST_LABEL,
ZEND_AST_REF,
ZEND_AST_HALT_COMPILER,
ZEND_AST_ECHO,
ZEND_AST_THROW,
ZEND_AST_GOTO,
ZEND_AST_BREAK,
ZEND_AST_CONTINUE,

/* 2 child nodes */
ZEND_AST_DIM = 2 << ZEND_AST_NUM_CHILDREN_SHIFT,
ZEND_AST_PROP,
ZEND_AST_NULLSAFE_PROP,
ZEND_AST_STATIC_PROP,
ZEND_AST_CALL,
ZEND_AST_CLASS_CONST,
ZEND_AST_ASSIGN,
ZEND_AST_ASSIGN_REF,
ZEND_AST_ASSIGN_OP,
ZEND_AST_BINARY_OP,
ZEND_AST_GREATER,
ZEND_AST_GREATER_EQUAL,
ZEND_AST_AND,
ZEND_AST_OR,
ZEND_AST_ARRAY_ELEM,
ZEND_AST_NEW,
ZEND_AST_INSTANCEOF,
ZEND_AST_YIELD,
ZEND_AST_COALESCE,
ZEND_AST_ASSIGN_COALESCE,

ZEND_AST_STATIC,
ZEND_AST_WHILE,
ZEND_AST_DO_WHILE,
ZEND_AST_IF_ELEM,
ZEND_AST_SWITCH,
ZEND_AST_SWITCH_CASE,
ZEND_AST_DECLARE,
ZEND_AST_USE_TRAIT,
ZEND_AST_TRAIT_PRECEDENCE,
ZEND_AST_METHOD_REFERENCE,
ZEND_AST_NAMESPACE,
ZEND_AST_USE_ELEM,
ZEND_AST_TRAIT_ALIAS,
ZEND_AST_GROUP_USE,
ZEND_AST_CLASS_CONST_GROUP,
ZEND_AST_ATTRIBUTE,
ZEND_AST_MATCH,
ZEND_AST_MATCH_ARM,
ZEND_AST_NAMED_ARG,

/* 3 child nodes */
ZEND_AST_METHOD_CALL = 3 << ZEND_AST_NUM_CHILDREN_SHIFT,
ZEND_AST_NULLSAFE_METHOD_CALL,
ZEND_AST_STATIC_CALL,
ZEND_AST_CONDITIONAL,

ZEND_AST_TRY,
ZEND_AST_CATCH,
ZEND_AST_PROP_GROUP,
ZEND_AST_PROP_ELEM,
ZEND_AST_CONST_ELEM,

/* 4 child nodes */
ZEND_AST_FOR = 4 << ZEND_AST_NUM_CHILDREN_SHIFT,
ZEND_AST_FOREACH,

/* 5 child nodes */
ZEND_AST_PARAM = 5 << ZEND_AST_NUM_CHILDREN_SHIFT,
};

我们发现,有几个子节点,那么就从子节点个数 << ZEND_AST_NUM_CHILDREN_SHIFT开始。所以,对应的,我们可以通过反过来拿到子节点的个数:

1
zend_ast_kind >> ZEND_AST_NUM_CHILDREN_SHIFT

移进规约冲突

今天写了一个移进规约冲突的文法。文法规则如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
%{
#include <stdio.h>
#include <string.h>
#include "zend_compile.h"
#include "zend_opcode.h"
#include "zend_vm.h"

#define YYDEBUG 1

#define zendparse yyparse

extern int yylex(void);
extern int yyparse(void);
extern FILE *yyin;
extern int yylineno;

int yywrap()
{
return 1;
}

void yyerror(const char *s)
{
printf("[error] %s, in line %d\n", s, yylineno);
}

int main(int argc, char const *argv[])
{
return 0;
}
%}

%left '+' '-'
%left '*' '/'
%left '(' ')'

%token <ident> T_ECHO "'echo'"
%token <ast> T_LNUMBER "integer"
%token <ast> T_VARIABLE "variable"

%union {
zend_ast *ast;
}

%type <ast> top_statement statement
%type <ast> expr
%type <ast> echo_expr
%type <ast> scalar
%type <ast> top_statement_list
%type <ast> variable

%%

start:
top_statement_list { CG(ast) = $1; }
;

top_statement_list:
top_statement_list top_statement { $$ = zend_ast_list_add($1, $2); }
| %empty { $$ = zend_ast_create_list(0, ZEND_AST_STMT_LIST); }
;

top_statement:
statement { $$ = $1; }
;

statement:
T_ECHO echo_expr ';' { $$ = $2; }
| expr ';' { $$ = $1; }
;

echo_expr:
expr {
std::cout << "create echo zend_ast" << std::endl;
$$ = zend_ast_create_1(ZEND_AST_ECHO, 0, $1);
}
;

expr:
variable '=' expr {
std::cout << "create assign zend_ast" << std::endl;
$$ = zend_ast_create_2(ZEND_AST_ASSIGN, 0, $1, $3);
}
|
expr '+' expr {
std::cout << "create + zend_ast" << std::endl;
$$ = zend_ast_create_binary_op(ZEND_ADD, $1, $3);
}
| expr '-' expr {
std::cout << "create - zend_ast" << std::endl;
$$ = zend_ast_create_binary_op(ZEND_SUB, $1, $3);
}
| expr '*' expr {
std::cout << "create * zend_ast" << std::endl;
$$ = zend_ast_create_binary_op(ZEND_MUL, $1, $3);
}
| expr '/' expr {
std::cout << "create / zend_ast" << std::endl;
$$ = zend_ast_create_binary_op(ZEND_DIV, $1, $3);
}
| '(' expr ')' { $$ = $2; }
| scalar { $$ = $1; }
;

scalar:
T_LNUMBER { $$ = $1; }
;

variable:
T_VARIABLE { $$ = $1; }

%%

执行生成代码的命令:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
bison -d -Wcounterexamples zend_language_parser.y

zend_language_parser.y: warning: 4 shift/reduce conflicts [-Wconflicts-sr]
zend_language_parser.y: warning: shift/reduce conflict on token '+' [-Wcounterexamples]
Example: variable '=' expr • '+' expr
Shift derivation
expr
↳ variable '=' expr
↳ expr • '+' expr
Reduce derivation
expr
↳ expr '+' expr
↳ variable '=' expr •

# 省略其他的警告

可以看到,警告说是有4个地方有移进规约的冲突。

那么,什么是移进规约冲突呢?意思就是说,当我们预读了词素的时候,既可以对分析栈里面已有的词素进行规约也可以对预读的词素进行移进,这就是已经规约冲突。

OK,我们来看看上面的报错。可以看到,当有字符串:

1
$a = 1 + 1

输入时,会发生移进规约的冲突。

我们来看看如果是优先移进的话,状态是如何变化的:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
+----------------------------------+   +-----+   +----------------------------------+
| | |init | | $a = 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+




+----------------------------------+ +-----+ +----------------------------------+
| $a | |shift| | = 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+



+----------------------------------+ +-----+ +----------------------------------+
| $a = | |shift| | 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+


+----------------------------------+ +-----+ +----------------------------------+
| $a = 1 | |shift| | + 1 |
+----------------------------------+ +-----+ +----------------------------------+



+----------------------------------+ +-----+ +----------------------------------+
| $a = 1 + | |shift| | 1 |
+----------------------------------+ +-----+ +----------------------------------+


+----------------------------------+ +-----+ +----------------------------------+
| $a = 1 + 1 | |shift| | |
+----------------------------------+ +-----+ +----------------------------------+


+----------------------------------+ +------+ +----------------------------------+
| $a = expr | |reduce| | |
+----------------------------------+ +------+ +----------------------------------+



+----------------------------------+ +------+ +----------------------------------+
| expr | |reduce| | |
+----------------------------------+ +------+ +----------------------------------+

对应的AST如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
               +------------+                             
|ZEND_ASSIGN |
| |
+------------+
|
|
+--------------+--------------+
| |
| |
v v
+------------+ +------------+
| $a | | ZEND_ADD |
| | | |
+------------+ +------------+
|
|
|
+---------------+-------------+
| |
| |
v v
+------------+ +------------+
| 1 | | 1 |
| | | |
+------------+ +------------+

计算这个AST,我们会得到$a的最终值为2。这是符合主流语言的预期的。

我们来看看如果是优先规约的话,状态是如何变化的:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
+----------------------------------+   +-----+   +----------------------------------+
| | |init | | $a = 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+




+----------------------------------+ +-----+ +----------------------------------+
| $a | |shift| | = 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+



+----------------------------------+ +-----+ +----------------------------------+
| $a = | |shift| | 1 + 1 |
+----------------------------------+ +-----+ +----------------------------------+


+----------------------------------+ +-----+ +----------------------------------+
| $a = 1 | |shift| | + 1 |
+----------------------------------+ +-----+ +----------------------------------+



+----------------------------------+ +------+ +----------------------------------+
| expr | |reduce| | + 1 |
+----------------------------------+ +------+ +----------------------------------+


+----------------------------------+ +-----+ +----------------------------------+
| expr + | |shift| | 1 |
+----------------------------------+ +-----+ +----------------------------------+


+----------------------------------+ +------+ +----------------------------------+
| expr + 1 | |shift | | |
+----------------------------------+ +------+ +----------------------------------+



+----------------------------------+ +------+ +----------------------------------+
| expr | |reduce| | |
+----------------------------------+ +------+ +----------------------------------+

对应的AST如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
                                  +-----------------+                 
| |
| ZEND_ADD |
| |
+-----------------+
|
|
+---------------+----------------+
| |
| |
v v
+-----------------+ +-----------------+
| | | |
| ZEND_ASSIGN | | 1 |
| | | |
+-----------------+ +-----------------+
|
|
|
+-----------------+----------------+
| |
| |
v v
+-----------------+ +-----------------+
| | | |
| $a | | 1 |
| | | |
+-----------------+ +-----------------+

计算这个AST,我们会得到$a的最终值为1。这和我们想的不太一样。

所以,移进规约的冲突,会导致一些执行的顺序不一致。如果我们学习过bison官方文档经典的if ... else的移进规约冲突问题的话,我们知道,解决它的办法是修改文法,进而避免冲突(因为这个例子有一点绕,所以我没有用那个例子)。那我们这种情况呢,就可以通过设置词素的优先级来解决掉,我们设置=的优先级低于+即可:

1
2
3
4
%left '='
%left '+' '-'
%left '*' '/'
%left '(' ')'

这样的话,当我们的分析栈为如下情况的时候:

1
2
3
+----------------------------------+   +-----+   +----------------------------------+
| $a = 1 | |shift| | + 1 |
+----------------------------------+ +-----+ +----------------------------------+

我们预读一个+,因为+号的优先级更高一点,所以,此时不会选择规约,而是把+移进。这样,我们可以保证在后续规约的时候,先规约1 + 1,进而也保证了运算符的优先级。

《手把手教你编写PHP编译器》-执行opcode

上一篇文章,我们成功的把AST翻译成了opcode,这样有一个好处,就是它是线性的,连续的,这和我们的CPU去一条一条的执行机器指令是保持一致的,非常便于人类理解。但是,我们还没有去设置这些opcode对应的handler

这篇文章,我们来实现对这些opcode的执行,这一节还是比较难的。

首先,我们来捋一捋opcodehandler的关系。我们参考PHP的实现。首先是我们的_zend_op

1
2
3
4
5
6
7
8
9
struct _zend_op {
znode_op op1;
znode_op op2;
znode_op result;
unsigned char opcode;
char op1_type;
char op2_type;
char result_type;
};

这种结构实际上是一种三地址码的组织形式,这种结构可以方便我们后续进行数据流分析。

我们知道,变量和字面量等等是有类型的,既然有类型,我们的操作数1和操作数2就可能多种组合。所以,这实际上就是一种笛卡尔积的表现形式了。再加上opcode的种类也不止一种,所以,我们有如下笛卡尔积:

1
opcode × op1 × op2

举个例子画个图:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
+----------------------+              +----------------------+        +----------------------+
| | | | | |
| ZEND_ADD | | IS_CONST | | IS_CONST |
| | | | | |
+----------------------+ +----------------------+ +----------------------+




+----------------------+ +----------------------+ +----------------------+
| | | | | |
| ZEND_SUB | | IS_TMP_VAR | | IS_TMP_VAR |
| | | | | |
+----------------------+ +----------------------+ +----------------------+





+----------------------+
| |
| ZEND_MUL |
| |
+----------------------+




+----------------------+
| |
| ZEND_DIV |
| |
+----------------------+

那么,我们就会有4 * 2 * 2spec handler

1
2
3
4
5
ZEND_ADD_IS_CONST_IS_CONST
ZEND_ADD_IS_CONST_IS_TMP_VAR
ZEND_ADD_IS_TMP_VAR_IS_CONST
ZEND_ADD_IS_TMP_VAR_IS_TMP_VAR
# 以此类推

假设,我们的opcode是按照顺序从0开始编号的,并且操作数的类型也是从0开始进行编号,并且,我们的spec handler也是严格按照顺序在内存中进行排序的。那我,我们就可以通过opcodeop1_typeop2_type找到spec handler的位置了,这个有点像一个三维的数组。对应的算法如下:

1
opcode * op1_type的数量 * op2_type的数量 + opt_type的编号 * op2_type的数量 + op2_type的编号

我们的实现都是围绕着这个算法来进行的。

首先,我们来定义一下操作数的类型:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#define OP_TYPE_MAP(XX)                                                                                                \
XX(IS_UNUSED, 0) \
XX(IS_CONST, 1 << 0) \
XX(IS_TMP_VAR, 1 << 1) \
XX(IS_VAR, 1 << 2) \
XX(IS_CV, 1 << 3)

enum op_type_e {
#define OP_TYPE_GEN(name, value) name = value,
OP_TYPE_MAP(OP_TYPE_GEN)
#undef OP_TYPE_GEN
};

enum op_type_code_e {
#define OP_TYPE_CODE_GEN(name, value) _##name##_CODE,
OP_TYPE_MAP(OP_TYPE_CODE_GEN)
#undef OP_TYPE_CODE_GEN
};

接着,我们可以来编写我们的spec handler了。从上面可以看出,我们的操作数有好几个。但是,实际上,对于同一个opcode,它要执行的动作是一样的,只不过操作数的类型不同,获取操作数的方式需要改变。如果我们手写每一种opcode对应的所有handler,那么这个维护成本是非常的高的,所以,我们应该是有一个代码生成的机制,写好通用的模板代码,然后直接生成即可。

下面,我们来给出模板代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
ZEND_VM_HANDLER(0, ZEND_NOP, CONST|TMPVAR, CONST|TMPVAR)
{
return 0;
}

ZEND_VM_HANDLER(1, ZEND_ADD, CONST|TMPVAR, CONST|TMPVAR)
{
int64_t op1, op2;

op1 = GET_OP1();
op2 = GET_OP2();
op_array->literals[opline->result.var] = op1 + op2;
return 0;
}

ZEND_VM_HANDLER(2, ZEND_SUB, CONST|TMPVAR, CONST|TMPVAR)
{
int64_t op1, op2;

op1 = GET_OP1();
op2 = GET_OP2();
op_array->literals[opline->result.var] = op1 - op2;
return 0;
}

ZEND_VM_HANDLER(3, ZEND_MUL, CONST|TMPVAR, CONST|TMPVAR)
{
int64_t op1, op2;

op1 = GET_OP1();
op2 = GET_OP2();
op_array->literals[opline->result.var] = op1 * op2;
return 0;
}

ZEND_VM_HANDLER(4, ZEND_DIV, CONST|TMPVAR, CONST|TMPVAR)
{
int64_t op1, op2;

op1 = GET_OP1();
op2 = GET_OP2();
op_array->literals[opline->result.var] = op1 / op2;
return 0;
}

ZEND_VM_HANDLER(136, ZEND_ECHO, CONST|TMPVAR, UNUSED)
{
int64_t op1;

op1 = GET_OP1();
printf("%lld", op1);
return 0;
}

可以看到,非常的简单。其中,这里的数字0, 1, 2, 3, 4, 136是这个opcode的编号。

接着,我们来用PHP代码来完成这个代码生成的脚本:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
<?php

#define IS_UNUSED 0 /* Unused operand */
#define IS_CONST (1 << 0)
#define IS_TMP_VAR (1 << 1)
#define IS_VAR (1 << 2)
#define IS_CV (1 << 3) /* Compiled variable */

define('ZEND_VM_OP_UNUSED', 1 << 0);
define('ZEND_VM_OP_CONST', 1 << 1);
define('ZEND_VM_OP_TMPVAR', 1 << 2);
define('ZEND_VM_OP_VAR', 1 << 3);
define('ZEND_VM_OP_CV', 1 << 4);

$op_types_map = array(
"UNUSED" => ZEND_VM_OP_UNUSED,
"CONST" => ZEND_VM_OP_CONST,
"TMPVAR" => ZEND_VM_OP_TMPVAR,
"VAR" => ZEND_VM_OP_VAR,
"CV" => ZEND_VM_OP_CV,
);

$op1_get = array(
"UNUSED" => "nullptr",
"CONST" => "opline->op1.num",
"TMPVAR" => "op_array->literals[opline->op1.var]",
"VAR" => "nullptr",
"CV" => "nullptr",
);

$op2_get = array(
"UNUSED" => "nullptr",
"CONST" => "opline->op2.num",
"TMPVAR" => "op_array->literals[opline->op2.var]",
"VAR" => "nullptr",
"CV" => "nullptr",
);

$opcodes = [];
$max_opcode = 0;
$spec_names = [];

function parse_operand_spec($def, $lineno, $str, &$flags)
{
global $op_types_map;

$flags = 0;
$a = explode("|", $str);
foreach ($a as $val) {
if (isset($op_types_map[$val])) {
$flags |= $op_types_map[$val];
} else {
die("ERROR ($def:$lineno): Wrong operand type '$str'\n");
}
}

return array_flip($a);
}

function gen_handler($f, $opcode)
{
global $op1_get, $op2_get, $spec_names, $op_types_map;

$opTypes = array_keys($op_types_map);

foreach ($opTypes as $op1Type) {
foreach ($opTypes as $op2Type) {
if (isset($opcode['op1'][$op1Type]) && isset($opcode['op2'][$op2Type])) {
$specialized_replacements = [
"/GET_OP1\(([^)]*)\)/" => $op1_get[$op1Type],
"/GET_OP2\(([^)]*)\)/" => $op2_get[$op2Type],
];

$name = $opcode['op'];
$templateCode = $opcode['code'];

$spec_name = $name."_SPEC"."_".$op1Type."_".$op2Type;
$spec_names[] = $spec_name;
fputs($f, "static int $spec_name(zend_op_array *op_array, zend_op *opline) ");
$code = preg_replace(array_keys($specialized_replacements), array_values($specialized_replacements), $templateCode);
fputs($f, $code);
} else {
$spec_names[] = 'nullptr';
}
}
}
}

function gen_spec_handlers($f)
{
global $spec_names;

fputs($f, "\tstatic const void * const spec_handlers[] = {\n");
foreach ($spec_names as $spec_name) {
fputs($f, "\t\t(void *) $spec_name,\n");
}
fputs($f, "\t};\n");

fputs($f, "\tzend_spec_handlers = spec_handlers;\n");
}

function gen_vm_execute_code($f)
{
fputs($f, "void zend_execute(zend_op_array *op_array) {\n");
fputs($f, "\tfor (size_t i = 0; i < op_array->last; i++) {\n");
fputs($f, "\t\tzend_op *opline = &(op_array->opcodes[i]);\n");
fputs($f, "\t\t((opcode_handler_t)opline->handler)(op_array, opline);\n");
fputs($f, "\t}\n");
fputs($f, "}\n\n");
}

function gen_vm_init_code($f)
{
fputs($f, "void zend_vm_init() {\n");

gen_spec_handlers($f);

fputs($f, "}\n");
}

function gen_executor_code($f)
{
global $opcodes, $max_opcode;

// define
fputs($f, "const void * const *zend_spec_handlers;\n");
fputs($f, "typedef int (*opcode_handler_t) (zend_op_array *op_array, const zend_op *opline);\n\n");

// Generate zend_vm_get_opcode_handler() function

fputs($f, "static uint32_t zend_vm_get_opcode_handler_idx(const zend_op *opline)\n");
fputs($f, "{\n");
fputs($f, "\tstatic int zend_vm_decode[IS_CV + 1] = {0};\n\n");
fputs($f, "\t#define OP_TYPE_CODE_GEN(name, value) zend_vm_decode[name] = _##name##_CODE;\n");
fputs($f, "\t\tOP_TYPE_MAP(OP_TYPE_CODE_GEN)\n");
fputs($f, "\t#undef OP_TYPE_CODE_GEN\n\n");
fputs($f, "\tuint32_t offset = 0;\n");
fputs($f, "\toffset += opline->opcode * 5 * 5;\n");
fputs($f, "\toffset += zend_vm_decode[(int) opline->op1_type] * 5;\n");
fputs($f, "\toffset += zend_vm_decode[(int) opline->op2_type];\n");
fputs($f, "\treturn offset;\n");
fputs($f, "}\n\n");

fputs($f, "const void *zend_vm_get_opcode_handler(const zend_op *opline)\n");
fputs($f, "{\n");
fputs($f, "\tuint32_t offset = zend_vm_get_opcode_handler_idx(opline);\n");
fputs($f, "\treturn zend_spec_handlers[offset];\n");
fputs($f, "}\n\n");

fputs($f, "void zend_vm_set_opcode_handler(zend_op *opline)\n");
fputs($f, "{\n");
fputs($f, "\topline->handler = zend_vm_get_opcode_handler(opline);\n");
fputs($f, "}\n\n");

$num = 0;

for ($i = 0; $i <= $max_opcode; $i++) {
if (isset($opcodes[$num])) {
gen_handler($f, $opcodes[$num], $num);
} else {
gen_handler($f, [], $num);
}
$num++;
}

gen_vm_execute_code($f);

gen_vm_init_code($f);
}

function gen_vm(string $def)
{
global $opcodes, $max_opcode;

$in = file($def);

$lineno = 0;
$handler = 0;

foreach ($in as $line) {
if (strpos($line, "ZEND_VM_HANDLER(") === 0) {
if (preg_match(
"/^ZEND_VM_HANDLER\(\s*([0-9]+)\s*,\s*([A-Z_]+)\s*,\s*([A-Z_|]+)\s*,\s*([A-Z_|]+)\s*(,\s*([A-Z_|]+)\s*)?(,\s*SPEC\(([A-Z_|=,]+)\)\s*)?\)/",
$line,
$m
) == 0) {
die("ERROR ($def:$lineno): Invalid ZEND_VM_HANDLER definition.\n");
}

$code = (int)$m[1];
$op = $m[2];
$op1 = parse_operand_spec($def, $lineno, $m[3], $flags1);
$op2 = parse_operand_spec($def, $lineno, $m[4], $flags2);
$flags = $flags1 | ($flags2 << 8);

if ($code > $max_opcode) {
$max_opcode = $code;
}

if (isset($opcodes[$code])) {
die("ERROR ($def:$lineno): Opcode with code '$code' is already defined.\n");
}
if (isset($opnames[$op])) {
die("ERROR ($def:$lineno): Opcode with name '$op' is already defined.\n");
}
$handler = $code;

$opcodes[$code] = array("op"=>$op,"op1"=>$op1,"op2"=>$op2,"code"=>"","flags"=>$flags);
} else {
$opcodes[$handler]['code'] .= $line;
}
}

ksort($opcodes);

$f = fopen(__DIR__ . "/zend_vm_opcodes.h", "w+") or die("ERROR: Cannot create zend_vm_opcodes.h\n");
fputs($f, "#pragma once\n\n");

foreach ($opcodes as $code => $dsc) {
$op = str_pad($dsc["op"], 20);
fputs($f, "#define $op $code\n");
}
fclose($f);
echo "zend_vm_opcodes.h generated successfully.\n";

$f = fopen(__DIR__ . "/zend_vm_opcodes.cc", "w+") or die("ERROR: Cannot create zend_vm_opcodes.c\n");
fputs($f, "#include \"zend_vm_opcodes.h\"\n\n");

fputs($f, "static const char *zend_vm_opcodes_names[".($max_opcode + 1)."] = {\n");
for ($i = 0; $i <= $max_opcode; $i++) {
fputs($f, "\t".(isset($opcodes[$i]["op"])?'"'.$opcodes[$i]["op"].'"':"nullptr").",\n");
}
fputs($f, "};\n\n");

fputs($f, "const char* zend_get_opcode_name(char opcode) {\n");
fputs($f, "\treturn zend_vm_opcodes_names[opcode];\n");
fputs($f, "}\n");

fclose($f);
echo "zend_vm_opcodes.cc generated successfully.\n";

$f = fopen(__DIR__ . "/zend_vm_execute.h", "w+") or die("ERROR: Cannot create zend_vm_execute.h\n");
fputs($f, "#pragma once\n\n");
fputs($f, "#include <stdint.h>\n");
fputs($f, "#include <stddef.h>\n");
fputs($f, "#include \"zend_compile.h\"\n\n");

gen_executor_code($f);
echo "zend_vm_execute.h generated successfully.\n";
}

gen_vm(__DIR__ . "/zend_vm_def.h");

接着,我们执行这个脚本,就会生成文件zend_vm_opcodes.hzend_vm_opcodes.cczend_vm_execute.h

这里面有两个核心的函数zend_vm_initzend_execute

其中zend_vm_init会用一块内存来存放我们的spec handler的地址,这样,我们就可以通过上面所说的算法,来找到spec handler了。

zend_execute就非常的简单了,执行opline就好了。

接下来,我们只需要设置好每一个opline对应的handler即可。代码如下:

1
2
3
4
5
6
7
// set opcode spec handler
void pass_two(zend_op_array *op_array) {
for (size_t i = 0; i < op_array->last; i++) {
zend_op *opline = &(op_array->opcodes[i]);
zend_vm_set_opcode_handler(opline);
}
}

最后,我们在文件zend_language_parser.y里面调用zend_vm_initpass_twozend_execute即可。

如何准确的查看opline对应的handler名字

我们在分析opcode对应的handler的时候,往往会根据opcode的命名规则来推断具体的handler。然而,如果我们使用PHP8的话,我们可以利用jitdebug功能来快速的看到opcode对应的handler。我举个例子:

有如下代码:

1
2
$a = [1, 2, 3];
$a[2];

像这个$a[2]对应的handler还是非常的长的,我们很难一口气推断出来。我们只需要配置一下php.ini就可以方便的拿到handler

1
2
3
4
5
6
7
zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1

opcache.jit=1201
opcache.jit_buffer_size=64M
opcache.jit_debug=0x01

执行结果如下:

1
2
3
4
5
6
JIT$/Users/hantaohuang/codeDir/cCode/php-src/test.php: ; (/Users/hantaohuang/codeDir/cCode/php-src/test.php)
# 省略其他的汇编代码
mov $ZEND_ASSIGN_SPEC_CV_CONST_RETVAL_UNUSED_HANDLER, %rax
# 省略其他的汇编代码
mov $ZEND_FETCH_DIM_R_INDEX_SPEC_CV_CONST_HANDLER, %rax
# 省略其他的汇编代码

可以看到,handlerZEND_FETCH_DIM_R_INDEX_SPEC_CV_CONST_HANDLER

PHP内核生成zend_vm_opcodes.h

本文基于的PHP8 commit为:14806e0824ecd598df74cac855868422e44aea53

首先,zend_vm_opcodes.h这个文件是通过脚本Zend/zend_vm_gen.php来生成的。而zend_vm_gen.php这个脚本依赖zend_vm_def.hzend_vm_execute.skl来生成文件zend_vm_execute.hzend_vm_opcodes.h

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
 +--------------------+                +--------------------+ 
| | | |
| zend_vm_def.h | |zend_vm_execute.skl |
| | | |
+--------------------+ +--------------------+
| |
+------------------+------------------+
|
v
+--------------------+
| |
| zend_vm_gen.php |
| |
+--------------------+
|
+-------------------+-------------------+
| |
v v
+--------------------+ +--------------------+
| | | |
| zend_vm_opcodes.h | | zend_vm_execute.h |
| | | |
+--------------------+ +--------------------+

我们以文件zend_vm_gen.php分析的起点,来看看生成zend_vm_opcodes.hzend_vm_execute.h的关键步骤。

首先,是函数gen_vm。这个函数会逐行扫描zend_vm_def.h里面的代码。

当扫描到ZEND_VM_HELPER的时候,就会执行下面的代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
if (strpos($line,"ZEND_VM_HELPER(") === 0 ||
strpos($line,"ZEND_VM_INLINE_HELPER(") === 0 ||
strpos($line,"ZEND_VM_COLD_HELPER(") === 0 ||
strpos($line,"ZEND_VM_HOT_HELPER(") === 0) {
// Parsing helper's definition
if (preg_match(
"/^ZEND_VM(_INLINE|_COLD|_HOT)?_HELPER\(\s*([A-Za-z_]+)\s*,\s*([A-Z_|]+)\s*,\s*([A-Z_|]+)\s*(?:,\s*SPEC\(([A-Z_|=,]+)\)\s*)?(?:,\s*([^)]*)\s*)?\)/",
$line,
$m) == 0) {
die("ERROR ($def:$lineno): Invalid ZEND_VM_HELPER definition.\n");
}
$inline = !empty($m[1]) && $m[1] === "_INLINE";
$cold = !empty($m[1]) && $m[1] === "_COLD";
$hot = !empty($m[1]) && $m[1] === "_HOT";
$helper = $m[2];
$op1 = parse_operand_spec($def, $lineno, $m[3], $flags1);
$op2 = parse_operand_spec($def, $lineno, $m[4], $flags2);
$param = isset($m[6]) ? $m[6] : null;
if (isset($helpers[$helper])) {
die("ERROR ($def:$lineno): Helper with name '$helper' is already defined.\n");
}

// Store parameters
if (ZEND_VM_KIND == ZEND_VM_KIND_GOTO
|| ZEND_VM_KIND == ZEND_VM_KIND_SWITCH
|| (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID && $hot)) {
foreach (explode(",", $param) as $p) {
$p = trim($p);
if ($p !== "") {
$params[$p] = 1;
}
}
}

$helpers[$helper] = array("op1"=>$op1,"op2"=>$op2,"param"=>$param,"code"=>"","inline"=>$inline,"cold"=>$cold,"hot"=>$hot);

if (!empty($m[5])) {
$helpers[$helper]["spec"] = parse_spec_rules($def, $lineno, $m[5]);
}

$handler = null;
$list[$lineno] = array("helper"=>$helper);

这段代码具体的细节我们不去深究,总结起来就是去正则匹配zend_vm_def.h里面当前行的ZEND_VM_HELPER,然后把相关的信息存在全局变量$helpers里面。例如:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
ZEND_VM_HELPER(zend_add_helper, ANY, ANY, zval *op_1, zval *op_2)
=>
[
"zend_add_helper" =>
[
"op1" => [
ANY:0
],
"op2" => [
ANY:0
],
"param" => "zval *op_1, zval *op_2",
"code" => "",
"inline" => false,
"cold" => false,
"hot" => false,
]
]

然后

1
2
3
4
5
6
7
else if ($handler !== null) {
// Add line of code to current opcode handler
$opcodes[$handler]["code"] .= $line;
} else if ($helper !== null) {
// Add line of code to current helper
$helpers[$helper]["code"] .= $line;
}

就是去拼接zend_vm_def.h里面的代码。如果是ZEND_VM_HELPER类型的代码,就执行$helpers[$helper]["code"] .= $line;。例如,当拼接完毕的时候,就会得到下面的信息:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
ZEND_VM_HELPER(zend_add_helper, ANY, ANY, zval *op_1, zval *op_2)
{
USE_OPLINE

SAVE_OPLINE();
if (UNEXPECTED(Z_TYPE_INFO_P(op_1) == IS_UNDEF)) {
op_1 = ZVAL_UNDEFINED_OP1();
}
if (UNEXPECTED(Z_TYPE_INFO_P(op_2) == IS_UNDEF)) {
op_2 = ZVAL_UNDEFINED_OP2();
}
add_function(EX_VAR(opline->result.var), op_1, op_2);
if (OP1_TYPE & (IS_TMP_VAR|IS_VAR)) {
zval_ptr_dtor_nogc(op_1);
}
if (OP2_TYPE & (IS_TMP_VAR|IS_VAR)) {
zval_ptr_dtor_nogc(op_2);
}
ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION();
}
=>
[
"zend_add_helper" =>
[
"op1" => [
ANY:0
],
"op2" => [
ANY:0
],
"param" => "zval *op_1, zval *op_2",
"code" => " USE_OPLINE
SAVE_OPLINE();
if (UNEXPECTED(Z_TYPE_INFO_P(op_1) == IS_UNDEF)) {
op_1 = ZVAL_UNDEFINED_OP1();
}
if (UNEXPECTED(Z_TYPE_INFO_P(op_2) == IS_UNDEF)) {
op_2 = ZVAL_UNDEFINED_OP2();
}
add_function(EX_VAR(opline->result.var), op_1, op_2);
if (OP1_TYPE & (IS_TMP_VAR|IS_VAR)) {
zval_ptr_dtor_nogc(op_1);
}
if (OP2_TYPE & (IS_TMP_VAR|IS_VAR)) {
zval_ptr_dtor_nogc(op_2);
}
ZEND_VM_NEXT_OPCODE_CHECK_EXCEPTION();",
"inline" => false,
"cold" => false,
"hot" => false,
]
]

当扫描到ZEND_VM_HANDLER的代码之后,就会执行下面的代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
if (strpos($line,"ZEND_VM_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_INLINE_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_HOT_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_HOT_NOCONST_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_HOT_NOCONSTCONST_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_HOT_SEND_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_HOT_OBJ_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_COLD_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_COLD_CONST_HANDLER(") === 0 ||
strpos($line,"ZEND_VM_COLD_CONSTCONST_HANDLER(") === 0) {
// Parsing opcode handler's definition
if (preg_match(
"/^ZEND_VM_(HOT_|INLINE_|HOT_OBJ_|HOT_SEND_|HOT_NOCONST_|HOT_NOCONSTCONST_|COLD_|COLD_CONST_|COLD_CONSTCONST_)?HANDLER\(\s*([0-9]+)\s*,\s*([A-Z_]+)\s*,\s*([A-Z_|]+)\s*,\s*([A-Z_|]+)\s*(,\s*([A-Z_|]+)\s*)?(,\s*SPEC\(([A-Z_|=,]+)\)\s*)?\)/",
$line,
$m) == 0) {
die("ERROR ($def:$lineno): Invalid ZEND_VM_HANDLER definition.\n");
}
$hot = !empty($m[1]) ? $m[1] : false;
$code = (int)$m[2];
$op = $m[3];
$len = strlen($op);
$op1 = parse_operand_spec($def, $lineno, $m[4], $flags1);
$op2 = parse_operand_spec($def, $lineno, $m[5], $flags2);
$flags = $flags1 | ($flags2 << 8);
if (!empty($m[7])) {
$flags |= parse_ext_spec($def, $lineno, $m[7]);
}

if ($len > $max_opcode_len) {
$max_opcode_len = $len;
}
if ($code > $max_opcode) {
$max_opcode = $code;
}
if (isset($opcodes[$code])) {
die("ERROR ($def:$lineno): Opcode with code '$code' is already defined.\n");
}
if (isset($opnames[$op])) {
die("ERROR ($def:$lineno): Opcode with name '$op' is already defined.\n");
}
$opcodes[$code] = array("op"=>$op,"op1"=>$op1,"op2"=>$op2,"code"=>"","flags"=>$flags,"hot"=>$hot);
if (isset($m[9])) {
$opcodes[$code]["spec"] = parse_spec_rules($def, $lineno, $m[9]);
if (isset($opcodes[$code]["spec"]["NO_CONST_CONST"])) {
$opcodes[$code]["flags"] |= $vm_op_flags["ZEND_VM_NO_CONST_CONST"];
}
if (isset($opcodes[$code]["spec"]["COMMUTATIVE"])) {
$opcodes[$code]["flags"] |= $vm_op_flags["ZEND_VM_COMMUTATIVE"];
}
}
$opnames[$op] = $code;
$handler = $code;
$helper = null;
$list[$lineno] = array("handler"=>$handler);
}

这段代码具体的细节我们不去深究,总结起来就是去正则匹配zend_vm_def.h里面当前行的ZEND_VM_HANDLER,然后把相关的信息存在全局变量$opcodes里面。例如:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ZEND_VM_HOT_NOCONSTCONST_HANDLER(1, ZEND_ADD, CONST|TMPVARCV, CONST|TMPVARCV)
=>
[
1 =>
[
"op" => "ZEND_ADD",
"op1" => [
"CONST" => 0,
"TMPVARCV" => 1
],
"op2" => [
"CONST" => 0,
"TMPVARCV" => 1
],
"code" => "",
"flags" => 2827,
"hot" => "HOT_NOCONSTCONST_"
]
]

其中

1
2
3
1 => [
"op" => "ZEND_ADD"
]

实际上就是ZEND_VM_HOT_NOCONSTCONST_HANDLER(1, ZEND_ADD, CONST|TMPVARCV, CONST|TMPVARCV)里面的1ZEND_ADD,这会用来定义opcode,对应zend_vm_opcodes.h文件里面的:

1
#define ZEND_ADD 1
1
2
"CONST" => 0,
"TMPVARCV" => 1

代表CONST|TMPVARCV的序号。实际上就是:

1
array_flip(explode("|", CONST|TMPVARCV))

之后的结果。

1
"flags" => 2827

计算方法是(CONST|TMPVARCV) | ((CONST|TMPVARCV) << 8)。至于CONSTTMPVARCV的值,我们可以在文件zend_vm_gen.php的变量$vm_op_decode里面找到。

接着,对于ZEND_VM_HANDLER就会执行$opcodes[$handler]["code"] .= $line;了,和ZEND_VM_HELPER的类似。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Generate opcode #defines (zend_vm_opcodes.h)
$code_len = strlen((string)$max_opcode);
$f = fopen(__DIR__ . "/zend_vm_opcodes.h", "w+") or die("ERROR: Cannot create zend_vm_opcodes.h\n");

// Insert header
out($f, HEADER_TEXT);
fputs($f, "#ifndef ZEND_VM_OPCODES_H\n#define ZEND_VM_OPCODES_H\n\n");
fputs($f, "#define ZEND_VM_SPEC\t\t" . ZEND_VM_SPEC . "\n");
fputs($f, "#define ZEND_VM_LINES\t\t" . ZEND_VM_LINES . "\n");
fputs($f, "#define ZEND_VM_KIND_CALL\t" . ZEND_VM_KIND_CALL . "\n");
fputs($f, "#define ZEND_VM_KIND_SWITCH\t" . ZEND_VM_KIND_SWITCH . "\n");
fputs($f, "#define ZEND_VM_KIND_GOTO\t" . ZEND_VM_KIND_GOTO . "\n");
fputs($f, "#define ZEND_VM_KIND_HYBRID\t" . ZEND_VM_KIND_HYBRID . "\n");
if ($GLOBALS["vm_kind_name"][ZEND_VM_KIND] === "ZEND_VM_KIND_HYBRID") {
fputs($f, "/* HYBRID requires support for computed GOTO and global register variables*/\n");
fputs($f, "#if (defined(__GNUC__) && defined(HAVE_GCC_GLOBAL_REGS))\n");
fputs($f, "# define ZEND_VM_KIND\t\tZEND_VM_KIND_HYBRID\n");
fputs($f, "#else\n");
fputs($f, "# define ZEND_VM_KIND\t\tZEND_VM_KIND_CALL\n");
fputs($f, "#endif\n");
} else {
fputs($f, "#define ZEND_VM_KIND\t\t" . $GLOBALS["vm_kind_name"][ZEND_VM_KIND] . "\n");
}
fputs($f, "\n");

这段代码就很简单了,直接往zend_vm_opcodes.h文件里面写这些内容。

1
2
3
foreach($vm_op_flags as $name => $val) {
fprintf($f, "#define %-24s 0x%08x\n", $name, $val);
}

这段代码是把zend_vm_gen.php文件里面的$vm_op_flags内容以16进制的格式写在zend_vm_opcodes.h文件里面:

1
2
3
4
5
6
7
8
9
10
11
$vm_op_flags = array(
"ZEND_VM_OP_SPEC" => 1<<0,
"ZEND_VM_OP_CONST" => 1<<1,
// 省略其他的
);

=>

#define ZEND_VM_OP_SPEC 0x00000001
#define ZEND_VM_OP_CONST 0x00000002
// 省略其他的

接着

1
2
3
4
5
6
7
foreach ($opcodes as $code => $dsc) {
$code = str_pad((string)$code,$code_len," ",STR_PAD_LEFT);
$op = str_pad($dsc["op"],$max_opcode_len);
if ($code <= $max_opcode) {
fputs($f,"#define $op $code\n");
}
}

会去用我们上面搜集好的$opcodes来定义我们的opcode,例如:

1
2
3
#define ZEND_NOP                          0
#define ZEND_ADD 1
// 省略其他的

接着

1
2
3
4
5
$code = str_pad((string)$max_opcode,$code_len," ",STR_PAD_LEFT);
$op = str_pad("ZEND_VM_LAST_OPCODE",$max_opcode_len);
fputs($f,"\n#define $op $code\n");

fputs($f, "\n#endif\n");

会去定义PHP内核一共有多少个opcode,例如:

1
#define ZEND_VM_LAST_OPCODE             199

至此,我们的zend_vm_opcodes.h文件生成完毕了。接着,开始生成zend_vm_opcodes.c文件。

其中:

1
2
3
4
5
fputs($f,"static const char *zend_vm_opcodes_names[".($max_opcode + 1)."] = {\n");
for ($i = 0; $i <= $max_opcode; $i++) {
fputs($f,"\t".(isset($opcodes[$i]["op"])?'"'.$opcodes[$i]["op"].'"':"NULL").",\n");
}
fputs($f, "};\n\n");

用来定义我们所有opcode对应的名字,例如:

1
2
3
4
5
static const char *zend_vm_opcodes_names[200] = {
"ZEND_NOP",
"ZEND_ADD",
// 省略其他的
};

这个zend_vm_opcodes_names数组的索引实际上就是opcode对应的id。所以,如果我们要得到一个opcode的名字,那么可以通过以下方式拿到:

1
2
3
zend_vm_opcodes_names[ZEND_ADD]
=>
"ZEND_ADD"

接着

1
2
3
4
5
fputs($f,"static uint32_t zend_vm_opcodes_flags[".($max_opcode + 1)."] = {\n");
for ($i = 0; $i <= $max_opcode; $i++) {
fprintf($f, "\t0x%08x,\n", isset($opcodes[$i]["flags"]) ? $opcodes[$i]["flags"] : 0);
}
fputs($f, "};\n\n");

用来定义opcode对应的flags。例如:

1
2
3
4
5
static uint32_t zend_vm_opcodes_flags[200] = {
0x00000000,
0x00000b0b,
// 省略其他的
};

flags的值的算法我们已经在上面介绍过了,这里再总结下:

1
$flags = $flags1 | ($flags2 << 8);

接着:

1
2
3
4
5
6
fputs($f, "ZEND_API const char* ZEND_FASTCALL zend_get_opcode_name(zend_uchar opcode) {\n");
fputs($f, "\tif (UNEXPECTED(opcode > ZEND_VM_LAST_OPCODE)) {\n");
fputs($f, "\t\treturn NULL;\n");
fputs($f, "\t}\n");
fputs($f, "\treturn zend_vm_opcodes_names[opcode];\n");
fputs($f, "}\n");

定义一个获取opcode name的函数。生成的结果如下:

1
2
3
4
5
6
ZEND_API const char* ZEND_FASTCALL zend_get_opcode_name(zend_uchar opcode) {
if (UNEXPECTED(opcode > ZEND_VM_LAST_OPCODE)) {
return NULL;
}
return zend_vm_opcodes_names[opcode];
}

首先是判断一下是否有这个opcode,有的话返回它的name,没有的话返回NULL

接着:

1
2
3
4
5
6
puts($f, "ZEND_API uint32_t ZEND_FASTCALL zend_get_opcode_flags(zend_uchar opcode) {\n");
fputs($f, "\tif (UNEXPECTED(opcode > ZEND_VM_LAST_OPCODE)) {\n");
fputs($f, "\t\topcode = ZEND_NOP;\n");
fputs($f, "\t}\n");
fputs($f, "\treturn zend_vm_opcodes_flags[opcode];\n");
fputs($f, "}\n");

定义一个获取opcode flags的函数。生成的结果如下:

1
2
3
4
5
6
ZEND_API uint32_t ZEND_FASTCALL zend_get_opcode_flags(zend_uchar opcode) {
if (UNEXPECTED(opcode > ZEND_VM_LAST_OPCODE)) {
opcode = ZEND_NOP;
}
return zend_vm_opcodes_flags[opcode];
}

首先是判断一下是否有这个opcode,有的话返回它的flags,没有的话返回ZEND_NOPflags(也就是0)。

至此,我们的zend_vm_opcodes.c文件生成完毕了。接着,开始生成zend_vm_execute.h文件。

1
2
3
4
5
6
// Support for ZEND_USER_OPCODE
out($f, "static user_opcode_handler_t zend_user_opcode_handlers[256] = {\n");
for ($i = 0; $i < 255; ++$i) {
out($f, "\t(user_opcode_handler_t)NULL,\n");
}
out($f, "\t(user_opcode_handler_t)NULL\n};\n\n");

用来定义一个zend_user_opcode_handlers数组,这个数组初始的时候全都是NULL。生成结果如下:

1
2
3
4
5
6
7
static user_opcode_handler_t zend_user_opcode_handlers[256] = {
(user_opcode_handler_t)NULL,
(user_opcode_handler_t)NULL,
(user_opcode_handler_t)NULL,
// 省略其他的
(user_opcode_handler_t)NULL
};

接着:

1
2
3
4
5
6
out($f, "static zend_uchar zend_user_opcodes[256] = {");
for ($i = 0; $i < 255; ++$i) {
if ($i % 16 == 1) out($f, "\n\t");
out($f, "$i,");
}
out($f, "255\n};\n\n");

用来定义我们的zend_user_opcodes,生成结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
static zend_uchar zend_user_opcodes[256] = {0,
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,
17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,
33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,
49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,
65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,
81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,
97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,
113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,
129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,
145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,
161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,
177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,
193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,
209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,
225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,
241,242,243,244,245,246,247,248,249,250,251,252,253,254,255
};

说明一共支持256zend_user_opcodes

接着,开始调用gen_executor来按照模板文件Zend/zend_vm_execute.skl生成代码。这个函数也是逐行扫描zend_vm_execute.skl文件。

其中zend_vm_execute.skl文件的第一行是:

1
{%DEFINES%}

意味着我们在zend_vm_execute.h里面需要生成一些定义。具体的生成过程如下:

1
2
3
4
5
6
7
8
9
10
11
out($f,"#define SPEC_START_MASK        0x0000ffff\n");
out($f,"#define SPEC_EXTRA_MASK 0xfffc0000\n");
out($f,"#define SPEC_RULE_OP1 0x00010000\n");
out($f,"#define SPEC_RULE_OP2 0x00020000\n");
out($f,"#define SPEC_RULE_OP_DATA 0x00040000\n");
out($f,"#define SPEC_RULE_RETVAL 0x00080000\n");
out($f,"#define SPEC_RULE_QUICK_ARG 0x00100000\n");
out($f,"#define SPEC_RULE_SMART_BRANCH 0x00200000\n");
out($f,"#define SPEC_RULE_COMMUTATIVE 0x00800000\n");
out($f,"#define SPEC_RULE_ISSET 0x01000000\n");
out($f,"#define SPEC_RULE_OBSERVER 0x02000000\n");

这是一些opcode对应的操作数的规则,例如SPEC_RULE_OP1意味着需要用到操作数1,并且支持的类型至少是2种。对应的代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
if (isset($dsc["op1"]) && !isset($dsc["op1"]["ANY"])) {
$count = 0;
foreach ($op_types_ex as $t) {
if (isset($dsc["op1"][$t])) {
$def_op1_type = $t;
$count++;
}
}
if ($count > 1) {
$spec_op1 = true;
$specs[$num] .= " | SPEC_RULE_OP1";
$def_op1_type = "ANY";
}
}

接着:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
out($f,"static const uint32_t *zend_spec_handlers;\n");
out($f,"static const void * const *zend_opcode_handlers;\n");
out($f,"static int zend_handlers_count;\n");
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"#if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)\n");
out($f,"static const void * const * zend_opcode_handler_funcs;\n");
out($f,"static zend_op hybrid_halt_op;\n");
out($f,"#endif\n");
}
out($f,"#if (ZEND_VM_KIND != ZEND_VM_KIND_HYBRID) || !ZEND_VM_SPEC\n");
out($f,"static const void *zend_vm_get_opcode_handler(zend_uchar opcode, const zend_op* op);\n");
out($f,"#endif\n\n");
if ($kind == ZEND_VM_KIND_HYBRID) {
out($f,"#if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)\n");
out($f,"static const void *zend_vm_get_opcode_handler_func(zend_uchar opcode, const zend_op* op);\n");
out($f,"#else\n");
out($f,"# define zend_vm_get_opcode_handler_func zend_vm_get_opcode_handler\n");
out($f,"#endif\n\n");
}

这个是根据ZEND_VM_KIND来定义一些变量和函数,生成结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
static const uint32_t *zend_spec_handlers;
static const void * const *zend_opcode_handlers;
static int zend_handlers_count;
#if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)
static const void * const * zend_opcode_handler_funcs;
static zend_op hybrid_halt_op;
#endif
#if (ZEND_VM_KIND != ZEND_VM_KIND_HYBRID) || !ZEND_VM_SPEC
static const void *zend_vm_get_opcode_handler(zend_uchar opcode, const zend_op* op);
#endif

#if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)
static const void *zend_vm_get_opcode_handler_func(zend_uchar opcode, const zend_op* op);
#else
# define zend_vm_get_opcode_handler_func zend_vm_get_opcode_handler
#endif

zend_vm_gen.php默认是ZEND_VM_KIND_HYBRID模式。

接着,会有一大段的代码来定义一些如下宏:

1
2
3
4
5
HYBRID_NEXT()
HYBRID_SWITCH()
HYBRID_CASE(op)
HYBRID_BREAK()
HYBRID_DEFAULT

接着,会调用gen_executor_code来生成opcode的详细handler。例如,我们的操作数有如下类型:

1
2
3
4
5
6
7
8
9
10
$op_types_ex = array(
"ANY",
"CONST",
"TMPVARCV",
"TMPVAR",
"TMP",
"VAR",
"UNUSED",
"CV",
);

那么,就最大就会有op1_type * op1_typehandler。所以,就会有如下代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Produce specialized executor
$op1t = $op_types_ex;
// for each op1.op_type
foreach($op1t as $op1) {
$op2t = $op_types_ex;
// for each op2.op_type
foreach($op2t as $op2) {
// for each handlers in helpers in original order
foreach ($list as $lineno => $dsc) {
if (isset($dsc["handler"])) {
$num = $dsc["handler"];
foreach (extra_spec_handler($opcodes[$num]) as $extra_spec) {
// Check if handler accepts such types of operands (op1 and op2)
if (isset($opcodes[$num]["op1"][$op1]) &&
isset($opcodes[$num]["op2"][$op2])) {
// Generate handler code
gen_handler($f, 1, $kind, $opcodes[$num]["op"], $op1, $op2, isset($opcodes[$num]["use"]), $opcodes[$num]["code"], $lineno, $opcodes[$num], $extra_spec, $switch_labels);
}
}
} else if (isset($dsc["helper"])) {
$num = $dsc["helper"];
foreach (extra_spec_handler($helpers[$num]) as $extra_spec) {
// Check if handler accepts such types of operands (op1 and op2)
if (isset($helpers[$num]["op1"][$op1]) &&
isset($helpers[$num]["op2"][$op2])) {
// Generate helper code
gen_helper($f, 1, $kind, $num, $op1, $op2, $helpers[$num]["param"], $helpers[$num]["code"], $lineno, $helpers[$num]["inline"], $helpers[$num]["cold"], $helpers[$num]["hot"], $extra_spec);
}
}
} else {
var_dump($dsc);
die("??? $kind:$num\n");
}
}
}
}

对于这段代码,$list里面存放了所有的helper的名字和opcode的值,例如:

1
2
3
4
5
"helper" => "zend_add_helper",
"handler" => 1,
"helper" => "zend_sub_helper",
"handler" => 2,
// 省略其他的内容

如果是helper,那么我们从$helpers里面获取到这个helper函数的信息。

如果是handler,那么我们从$opcodes里面获取到这个opcode的信息。

其中:

1
2
$opcodes[$num]["op1"]
$opcodes[$num]["op2"]

里面存放的就是这个opcode对应的操作数1和操作数2支持的所有类型,我们在前面解析的时候就拿到了这些信息。

无论是是helper还是opcode类型的handler,都会调用extra_spec_handler来生成spec函数。在生成spec的时候,会将zend_vm_def.h里面对应的handlercode进行替换,替换的规则在函数gen_code里面。

生成了handler对应的specs之后,就完成了模板文件里面{%DEFINES%}的替换了。

接着,开始替换模板文件里面的{%EXECUTOR_NAME%},也就是开始生成我们的zend_execute函数了:

1
2
3
case "EXECUTOR_NAME":
out($f, $m[1].$executor_name.$m[3]."\n");
break;

这里是名字是execute

接着替换模板文件的{%HELPER_VARS%}

1
2
3
case "HELPER_VARS":
// 省略代码
break;

生成结果如下:

1
2
3
4
5
6
7
8
9
#ifdef ZEND_VM_IP_GLOBAL_REG
const zend_op *orig_opline = opline;
#endif
#ifdef ZEND_VM_FP_GLOBAL_REG
zend_execute_data *orig_execute_data = execute_data;
execute_data = ex;
#else
zend_execute_data *execute_data = ex;
#endif

接着替换模板文件的{%INTERNAL_LABELS%}

1
2
3
4
out($f,$prolog."if (UNEXPECTED(execute_data == NULL)) {\n");
out($f,$prolog."\tstatic const void * const labels[] = {\n");
gen_labels($f, $spec, ($kind == ZEND_VM_KIND_HYBRID) ? ZEND_VM_KIND_GOTO : $kind, $prolog."\t\t", $specs);
out($f,$prolog."\t};\n");

这里定义了一个名字叫做labels的静态变量,也就意味着每次调用zend_execute是共享的。生成的代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
if (UNEXPECTED(execute_data == NULL)) {
static const void * const labels[] = {
(void*)&&ZEND_NOP_SPEC_LABEL,
(void*)&&ZEND_ADD_SPEC_CONST_CONST_LABEL,
(void*)&&ZEND_ADD_SPEC_CONST_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_CONST_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_ADD_SPEC_CONST_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_CONST_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_CONST_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_CONST_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL,
(void*)&&ZEND_ADD_SPEC_TMPVARCV_TMPVARCV_LABEL,
(void*)&&ZEND_NULL_LABEL
// 省略其他的内容
};

也就意味着,当第一次调用zend_execute的时候,会初始化这个labels变量。

接着,我们会生成一堆的HYBRID_SWITCHHYBRID_CASE。这个和labels变量里面的指针是对应的,并且和我们生成的handler是对应的。我们后面会写一个小demo来解释下这个switch ... case的原理。

接着,会生成$specs

1
2
3
4
5
6
7
static const uint32_t specs[] = {
0,
1 | SPEC_RULE_OP1 | SPEC_RULE_OP2,
26 | SPEC_RULE_OP1 | SPEC_RULE_OP2,
51 | SPEC_RULE_OP1 | SPEC_RULE_OP2 | SPEC_RULE_COMMUTATIVE,
// 省略其他的
};

其中,SPEC_RULE_OP1SPEC_RULE_OP2解释过了。那么它们前面的数字是什么呢?实际上,前面的数字是第一个当前opcode的第一个spec handlerlabels变量的索引。这么说比较抽象,我用下面的图来解释一下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
  +-------------+                                   +-------------+          
| specs | | labels |
+-------------+ +-------------+


+------+----+------+ +--------------------------------+
| | 0 | |---------------------->| ZEND_NOP_SPEC_LABEL |
+------+----+------+ +--------------------------------+
| | 1 | |---------------------->|ZEND_ADD_SPEC_CONST_CONST_LABEL |
+------+----+------+ +--------------------------------+
| | 26 | |-----------+ |ZEND_ADD_SPEC_CONST_TMPVARCV_LAB|
+------+----+------+ | +--------------------------------+
| | 51 | | | | ... |
+------+----+------+ | +--------------------------------+
| | +---------->|ZEND_SUB_SPEC_CONST_CONST_LABEL |
| | +--------------------------------+
| ... | |ZEND_SUB_SPEC_CONST_TMPVARCV_LAB|
| | +--------------------------------+
| | | ... |
| | | |
+------------------+ +--------------------------------+

至此,zend_vm_gen.php生成代码的过程结束了。

PHP内核pass_two源码分析

本文基于的PHP8 commit为:14806e0824ecd598df74cac855868422e44aea53

我们先来看一下PHP脚本到opcode的生成流程,在函数zend_compile里面:

1
2
3
4
5
6
7
8
9
10
11
12
// 删除了部分代码
static zend_op_array *zend_compile(int type)
{
if (!zendparse()) {
init_op_array(op_array, type, INITIAL_OP_ARRAY_SIZE);

zend_compile_top_stmt(CG(ast));
pass_two(op_array);
}

return op_array;
}

总结起来如下:

1
2
3
1. 调用zendparse完成词法分析、语法分析从而生成AST。
2. 调用init_op_array, zend_compile_top_stmt来完成AST到opcode的转化,此时还没有设置opcode对应的handler,以及有一部分东西是编译时的。
3. 调用pass_two完成编译时到运行时信息的转化、设置opcode对应的handler。

我们来看看具体的代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
ZEND_API void pass_two(zend_op_array *op_array)
{
zend_op *opline, *end;

if (!ZEND_USER_CODE(op_array->type)) {
return;
}

#if ZEND_USE_ABS_CONST_ADDR
if (CG(context).opcodes_size != op_array->last) {
op_array->opcodes = (zend_op *) erealloc(op_array->opcodes, sizeof(zend_op)*op_array->last);
CG(context).opcodes_size = op_array->last;
}
if (CG(context).literals_size != op_array->last_literal) {
op_array->literals = (zval*)erealloc(op_array->literals, sizeof(zval) * op_array->last_literal);
CG(context).literals_size = op_array->last_literal;
}
#else
op_array->opcodes = (zend_op *) erealloc(op_array->opcodes,
ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16) +
sizeof(zval) * op_array->last_literal);
if (op_array->literals) {
memcpy(((char*)op_array->opcodes) + ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16),
op_array->literals, sizeof(zval) * op_array->last_literal);
efree(op_array->literals);
op_array->literals = (zval*)(((char*)op_array->opcodes) + ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16));
}
CG(context).opcodes_size = op_array->last;
CG(context).literals_size = op_array->last_literal;
#endif

/* Needs to be set directly after the opcode/literal reallocation, to ensure destruction
* happens correctly if any of the following fixups generate a fatal error. */
op_array->fn_flags |= ZEND_ACC_DONE_PASS_TWO;

opline = op_array->opcodes;
end = opline + op_array->last;
while (opline < end) {
if (opline->op1_type == IS_CONST) {
ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, opline->op1);
} else if (opline->op1_type & (IS_VAR|IS_TMP_VAR)) {
opline->op1.var = EX_NUM_TO_VAR(op_array->last_var + opline->op1.var);
}
if (opline->op2_type == IS_CONST) {
ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, opline->op2);
} else if (opline->op2_type & (IS_VAR|IS_TMP_VAR)) {
opline->op2.var = EX_NUM_TO_VAR(op_array->last_var + opline->op2.var);
}
if (opline->result_type & (IS_VAR|IS_TMP_VAR)) {
opline->result.var = EX_NUM_TO_VAR(op_array->last_var + opline->result.var);
}
ZEND_VM_SET_OPCODE_HANDLER(opline);
opline++;
}

return;
}

其中:

1
2
3
4
5
6
7
8
9
10
#if ZEND_USE_ABS_CONST_ADDR
if (CG(context).opcodes_size != op_array->last) {
op_array->opcodes = (zend_op *) erealloc(op_array->opcodes, sizeof(zend_op)*op_array->last);
CG(context).opcodes_size = op_array->last;
}
if (CG(context).literals_size != op_array->last_literal) {
op_array->literals = (zval*)erealloc(op_array->literals, sizeof(zval) * op_array->last_literal);
CG(context).literals_size = op_array->last_literal;
}
#else

是在32位的机器上面进行设置的,此时,会重新分配opcodesliterals,可以避免内存的浪费。

1
2
3
4
5
6
7
8
9
10
11
12
    op_array->opcodes = (zend_op *) erealloc(op_array->opcodes,
ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16) +
sizeof(zval) * op_array->last_literal);
if (op_array->literals) {
memcpy(((char*)op_array->opcodes) + ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16),
op_array->literals, sizeof(zval) * op_array->last_literal);
efree(op_array->literals);
op_array->literals = (zval*)(((char*)op_array->opcodes) + ZEND_MM_ALIGNED_SIZE_EX(sizeof(zend_op) * op_array->last, 16));
}
CG(context).opcodes_size = op_array->last;
CG(context).literals_size = op_array->last_literal;
#endif

是在64位的机器上面进行设置的,此时,会重新分配opcodes,大小是opline的条数加上字面量的个数,然后把literals拷贝到opcodes的最后面。这样,使得opcodesliterals是在一块连续的内存上面。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
while (opline < end) {
if (opline->op1_type == IS_CONST) {
c(op_array, opline, opline->op1);
} else if (opline->op1_type & (IS_VAR|IS_TMP_VAR)) {
opline->op1.var = EX_NUM_TO_VAR(op_array->last_var + opline->op1.var);
}
if (opline->op2_type == IS_CONST) {
ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, opline->op2);
} else if (opline->op2_type & (IS_VAR|IS_TMP_VAR)) {
opline->op2.var = EX_NUM_TO_VAR(op_array->last_var + opline->op2.var);
}
if (opline->result_type & (IS_VAR|IS_TMP_VAR)) {
opline->result.var = EX_NUM_TO_VAR(op_array->last_var + opline->result.var);
}
ZEND_VM_SET_OPCODE_HANDLER(opline);
opline++;
}

调用ZEND_PASS_TWO_UPDATE_CONSTANT来完成常量编译时到运行时的转换。我们来看看这个宏:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
/* constant-time constant */
# define CT_CONSTANT_EX(op_array, num) \
((op_array)->literals + (num))

# define CT_CONSTANT(node) \
CT_CONSTANT_EX(CG(active_op_array), (node).constant)

#if ZEND_USE_ABS_CONST_ADDR

/* run-time constant */
# define RT_CONSTANT(opline, node) \
(node).zv

/* convert constant from compile-time to run-time */
# define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
(node).zv = CT_CONSTANT_EX(op_array, (node).constant); \
} while (0)

#else

/* At run-time, constants are allocated together with op_array->opcodes
* and addressed relatively to current opline.
*/

/* run-time constant */
# define RT_CONSTANT(opline, node) \
((zval*)(((char*)(opline)) + (int32_t)(node).constant))

/* convert constant from compile-time to run-time */
# define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
(node).constant = \
(((char*)CT_CONSTANT_EX(op_array, (node).constant)) - \
((char*)opline)); \
} while (0)

#endif

32位的机器上,走的逻辑是:

1
2
3
4
5
6
7
8
9
10
11
# define CT_CONSTANT_EX(op_array, num) \
((op_array)->literals + (num))

/* run-time constant */
# define RT_CONSTANT(opline, node) \
(node).zv

/* convert constant from compile-time to run-time */
# define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
(node).zv = CT_CONSTANT_EX(op_array, (node).constant); \
} while (0)

我们知道,在编译的时候,(node).constant存的是字面量在(op_array)->literals的索引,也就是123等等。

而进行编译时到运行时的转换后,(node).constant存的就是字面量在(op_array)->literals的绝对地址了。

我们再来看看64位的机器上,走的逻辑是:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# define CT_CONSTANT_EX(op_array, num) \
((op_array)->literals + (num))

/* run-time constant */
# define RT_CONSTANT(opline, node) \
((zval*)(((char*)(opline)) + (int32_t)(node).constant))

/* convert constant from compile-time to run-time */
# define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
(node).constant = \
(((char*)CT_CONSTANT_EX(op_array, (node).constant)) - \
((char*)opline)); \
} while (0)

#endif

我们发现,进行编译时到运行时的转换后,(node).constant存的就是字面量相对当前opline的相对地址了。因为在64位的机器上,opcodesliterals是在一块连续的内存上面,所以可以存一个相对地址。如下图:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
+----------------------------+   
| opcodes |
+----------------------------+

+----------------------------+
| opline1 |--+
+----------------------------+ |
| opline2 | |
+----------------------------+ |
| opline3 | |
+----------------------------+ |
| | |
| ...... | |
| Continuous memory | |
| | |
| | |
+----------------------------+ |
| literal1 |<-+
+----------------------------+
| literal2 |
+----------------------------+
| literal3 |
+----------------------------+
| |
| ...... |
| |
+----------------------------+
1
ZEND_VM_SET_OPCODE_HANDLER(opline);

这一步就是设置我们opcode对应的handler了。

《手把手教你编写PHP编译器》-echo_expr生成opcode

上篇文章,我们讲解了如何从yaphp源代码生成AST。这篇文章,我们来讲解如何从AST生成opcode

一句话总结起来就是,我们对AST进行深度遍历,然后以每一个子AST来生成一条opline,最终得到op_array。我们来看一下具体如何去实现它。

首先,我们需要定义一下和op_array相关的数据结构:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#define IS_UNUSED 0 /* Unused operand */
#define IS_CONST (1 << 0)
#define IS_TMP_VAR (1 << 1)
#define IS_VAR (1 << 2)
#define IS_CV (1 << 3) /* Compiled variable */

#define INITIAL_OP_ARRAY_SIZE 64

typedef struct _zend_op_array zend_op_array;
typedef struct _zend_oparray_context zend_oparray_context;
typedef struct _zend_op zend_op;
typedef union _znode_op znode_op;
typedef struct _znode znode;

union _znode_op {
uint32_t num;
uint32_t var;
};

struct _znode {
char op_type;
znode_op op;
};

struct _zend_op {
znode_op op1;
znode_op op2;
znode_op result;
unsigned char opcode;
char op1_type;
char op2_type;
char result_type;
};

struct _zend_op_array {
zend_op *opcodes;
uint32_t last; /* number of opcodes */
uint32_t T; /* number of temporary variables */
};

struct _zend_oparray_context {
uint32_t opcodes_size;
};

其中,_zend_op_array是核心结构,其他的结构都是以它为中心展开的。这些结构的具体含义我们可以在很多PHP源码分析的文章里面找到,所以我们不过多介绍。

这里,我们来看看_zend_op结构,我们发现,这个结构本质上是一种三地址码格式。有一个指令类型的opcode,有两个操作数 op1op2和一个结果result

现在,让我们来实现一下对AST的处理流程,在文件zend_language_parser.y里面:

1
2
3
4
5
6
op_array = (zend_op_array *) malloc(sizeof(zend_op_array));
init_op_array(op_array, INITIAL_OP_ARRAY_SIZE);
CG(active_op_array) = op_array;

zend_oparray_context_begin();
zend_compile_top_stmt(CG(ast));

这就是我们的核心了,这里的步骤可以总结为:

1
2
3
1. 初始化op_array,并且赋值给CG(active_op_array)
2. 调用zend_oparray_context_begin初始化CG(context)
3. 调用zend_compile_top_stmt生成opcode

OK,以这个为思路,我们来看看具体如何实现的。

首先是函数init_op_array

1
2
3
4
5
void init_op_array(zend_op_array *op_array, int initial_ops_size) {
op_array->opcodes = (zend_op *) malloc(initial_ops_size * sizeof(zend_op));
op_array->last = 0;
op_array->T = 0;
}

这个函数非常的简单,首先为op_array里面保存的opcodes分配初始化的内存;然后设置last(即opline的条数)为0;然后设置T(即临时变量的个数为0,后面,我们的临时变量的名字是按照T来递增命名的,例如第一个临时变量叫做1,第二个临时变量叫做2,依次类推)为0。

然后是zend_oparray_context_begin

1
2
3
void zend_oparray_context_begin() {
CG(context).opcodes_size = INITIAL_OP_ARRAY_SIZE;
}

这个CG(context)会与我们的CG(active_op_array)挂钩,例如这里的CG(context).opcodes_size表示,我们的CG(active_op_array)总容量。可想而知,当我们编译的opline条数达到CG(context).opcodes_size的时候,需要进行CG(active_op_array)的扩容。

最后就是我们的核心函数zend_compile_top_stmt

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
void zend_compile_top_stmt(zend_ast *ast) {
if (!ast) {
return;
}

if (ast->kind == ZEND_AST_STMT_LIST) {
zend_ast_list *list = zend_ast_get_list(ast);
for (uint32_t i = 0; i < list->children; ++i) {
zend_compile_top_stmt(list->child[i]);
}
return;
}

zend_compile_stmt(ast);
}

这个函数是用来编译我们的ZEND_AST_STMT_LIST的。然后,每一条语句,我们会调用zend_compile_stmt来进行编译。

我们来看看zend_compile_stmt

1
2
3
4
5
6
7
8
9
10
11
12
13
void zend_compile_stmt(zend_ast *ast) {
if (!ast) {
return;
}

switch (ast->kind) {
case ZEND_AST_ECHO:
zend_compile_echo(ast);
break;
default:
break;
}
}

因为我们目前只实现了echo语句,所以,我们这里只有一个ZEND_AST_ECHO的类型。我们来看看我们是如何编译echo语句的:

1
2
3
4
5
6
7
8
void zend_compile_echo(zend_ast *ast) {
zend_ast *expr_ast = ast->child[0];

znode expr_node;
zend_compile_expr(&expr_node, expr_ast);

zend_emit_op(ZEND_ECHO, &expr_node, nullptr);
}

这个函数做了两件事情,第一件事情是编译echo节点使用的expr ast,第二件事情为echo ast(也就是我们的echo语句)生成一条opline

我们来看看我们是如何编译expr_ast的:

1
2
3
4
5
6
7
8
9
10
11
12
13
void zend_compile_expr(znode *result, zend_ast *ast) {
switch (ast->kind) {
case ZEND_AST_LNUM:
result->op.num = zend_ast_get_lnum(ast)->lnum;
result->op_type = IS_CONST;
break;
case ZEND_AST_BINARY_OP:
zend_compile_binary_op(result, ast);
return;
default:
break;
}
}

可以看到,如果是一个ZEND_AST_LNUM类型的节点(也就是一个数字),那么我们直接返回它作为编译ast后的结果;如果是一个ZEND_AST_BINARY_OP类型的节点(也就是操作符),那么我们需要继续编译。

1
2
3
4
5
6
7
8
9
10
11
12
void zend_compile_binary_op(znode *result, zend_ast *ast) {
zend_ast *left_ast = ast->child[0];
zend_ast *right_ast = ast->child[1];
uint32_t opcode = ast->attr;

znode left_node, right_node;

zend_compile_expr(&left_node, left_ast);
zend_compile_expr(&right_node, right_ast);

zend_emit_op_tmp(result, opcode, &left_node, &right_node);
}

可以看到,对ZEND_AST_BINARY_OP类型的编译实际上就是一个递归的函数。我们发现,只有在AST的子节点都是终结符的时候,我们才会调用zend_emit_op_tmp生成一条opline。我们看看zend_emit_op_tmp的实现:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
/**
* generate an opline
*/
static zend_op *zend_emit_op(unsigned char opcode, znode *op1, znode *op2) {
zend_op *opline = get_next_op();
opline->opcode = opcode;

if (op1 != nullptr) {
opline->op1_type = op1->op_type;
opline->op1 = op1->op;
}

if (op2 != nullptr) {
opline->op2_type = op2->op_type;
opline->op2 = op2->op;
}

return opline;
}

static inline uint32_t get_temporary_variable(void) {
return ++CG(active_op_array)->T;
}

static zend_op *zend_emit_op_tmp(znode *result, unsigned char opcode, znode *op1, znode *op2) {
zend_op *opline = zend_emit_op(opcode, op1, op2);

if (result) {
zend_make_tmp_result(result, opline);
}

return opline;
}

这几个函数就非常的简单了,设置oplineop1op2,然后oplineresult我们都用一个临时变量。因为我们的_zend_op是一个三地址码,所以,我们一条表达式里面,如果出现了多个操作符和多个操作数,那么我们就需要拆解出来,因此,我们就需要临时变量来作为中间结果了。例如1 + 2 + 3对应的过程如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
+----------------------+
| 1 + 2 + 3 |
+----------------------+
|
|
v
+----------------------+
| T1 = 1 + 2 |
+----------------------+
|
|
|
v
+----------------------+
| T2 = T1 + 3 |
+----------------------+

这样,我们就实现了ASTopcode的转化,我们来编写一个测试脚本:

1
2
echo 1 + 2 * 3;
echo 1 + 2 + 3;

执行结果如下:

1
2
3
4
5
6
7
*********************gennerate opcode*********************
#0 ZEND_MUL 2 3 ~1
#1 ZEND_ADD 1 ~1 ~2
#2 ZEND_ECHO ~2
#3 ZEND_ADD 1 2 ~3
#4 ZEND_ADD ~3 3 ~4
#5 ZEND_ECHO ~4

可以看到,是符合我们的预期的。

OPcache优化opcode

这篇文章,我们会通过一些例子来介绍一下Opcache对于opcode的一些优化。

简单的本地优化(pass1)

这个pass会开启一些简单的优化,例如优化常数条件JMP

对应的opcache配置如下:

1
2
3
4
5
zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1
opcache.opt_debug_level=0x30000
opcache.optimization_level=0x1

我们有如下代码:

1
2
3
4
5
6
7
8
9
10
<?php

function foo()
{
if (1) {
echo 1;
} else {
echo 2;
}
}

执行结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
foo:
; (lines=5, args=0, vars=0, tmps=0)
; (before optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-10
0000 JMPZ int(1) 0003
0001 ECHO int(1)
0002 JMP 0004
0003 ECHO int(2)
0004 RETURN null

foo:
; (lines=5, args=0, vars=0, tmps=0)
; (after optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-10
0000 NOP
0001 ECHO int(1)
0002 JMP 0004
0003 ECHO int(2)
0004 RETURN null

我们发现,优化前,我们需要执行JMPZ,并且按照条件来执行0003或者0004。但是优化后的代码,我们只需要执行00010004即可。我们发现,优化后的代码00020003实际上不会被执行,但是却生成了opcode,实际上是因为我们没有开启对应的优化,我们后面会有例子来讲解。

常数传播优化(pass8)

对应的opcache配置如下:

1
2
3
4
5
zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1
opcache.opt_debug_level=0x30000
opcache.optimization_level=0xe0

我们有如下代码:

1
2
3
4
5
6
7
<?php

function foo()
{
$a = 1;
echo $a + 2 + 3;
}

执行结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
foo:
; (lines=5, args=0, vars=1, tmps=3)
; (before optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-7
; return [] RANGE[0..0]
0000 ASSIGN CV0($a) int(1)
0001 T2 = ADD CV0($a) int(2)
0002 T3 = ADD T2 int(3)
0003 ECHO T3
0004 RETURN null

foo:
; (lines=3, args=0, vars=1, tmps=3)
; (after optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-7
0000 CV0($a) = QM_ASSIGN int(1)
0001 ECHO string("6")
0002 RETURN null

我们发现,因为a是常量1,所以在优化opcode的时候,会直接用1替换掉a

死代码消除(pass14)

对应的opcache配置如下:

1
2
3
4
5
zend_extension=opcache.so
opcache.enable=1
opcache.enable_cli=1
opcache.opt_debug_level=0x30000
opcache.optimization_level=0x2061

我们有如下代码:

1
2
3
4
5
6
7
8
9
10
<?php

function foo()
{
if (1) {
echo 1;
} else {
echo 2;
}
}

执行结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
foo:
; (lines=5, args=0, vars=0, tmps=0)
; (before optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-10
; return [] RANGE[0..0]
0000 JMPZ int(1) 0003
0001 ECHO int(1)
0002 JMP 0004
0003 ECHO int(2)
0004 RETURN null

foo:
; (lines=2, args=0, vars=0, tmps=0)
; (after optimizer)
; /Users/hantaohuang/codeDir/cCode/php-src/test.php:3-10
0000 ECHO int(1)
0001 RETURN null

我们发现,优化前的0000, 0002, 0003都被删除了,因为它们不会被执行。

为什么类方法中创建Swoole协程之后可以使用$this

前几天在写公司代码的时候,使用Hyperf写了大概这么一段代码:

1
2
3
4
5
6
7
8
9
class IndexController
{
public function test1()
{
Coroutine::create(function () {
$this->request->getHeaders();
});
}
}

然后,在执行$this->request->getHeaders();这一行报错了,说是某某某接口没有实现。但是,如果我把这一行代码直接放在创建协程的外面,也就是这么写:

1
2
3
4
5
6
7
class IndexController
{
public function test1()
{
$this->request->getHeaders();
}
}

就不会报错了。

具体要怎么解决这个问题不是我们这篇文章讨论的重点。这个问题一开始让我产生了一个疑问,以为不能在创建的子协程里面使用$this

然后,我写了这么一段代码来进行测试:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?php

class Foo
{
public function test1()
{
var_dump('foo1');
go(function () {
$this->test2();
});
}

public function test2()
{
var_dump('foo2');
}
}

$foo = new Foo;
$foo->test1();

输出结果如下:

1
2
string(4) "foo1"
string(4) "foo2"

发现,在子协程里面使用$this完全没有问题。

然后,回想到了编译原理的面向对象的语义特征里面作用域角度(我在我的这篇博客有讲解),我看了看Swoole的实现,果然发现有一部分代码是把$this复制到了子协程栈里面,实际上对应的就是EX(This)

1
2
call = zend_vm_stack_push_call_frame(
ZEND_CALL_TOP_FUNCTION | ZEND_CALL_ALLOCATED, func, argc, fci_cache.called_scope, fci_cache.object);

如果fci_cache.object改成传nullptr的话,执行我们的脚本,就会报这个错误:

1
2
string(4) "foo1"
[1] 60865 segmentation fault php test.php

说明,我们获取$this的时候出了问题。

实际上,PHP$this->test1()的生成的opcodeINIT_METHOD_CALL,对应的handlerZEND_INIT_METHOD_CALL_SPEC_UNUSED_CONST_HANDLER。我们一定会在这个handler里面看到从作用域里面去获取$this的过程。

如果我们传递的fci_cache.objectnullptr,意味着子协程作用域的EX(This)nullptr,那么EX(This)foo1的调用必然会段错误了。