2020 新春流行的RPC框架性能大比拼

自人类文明以来,人类交流的方式就在不断的变化,从最早的结绳记事、到烽火传消息,从飞鸽传书到驿站飞马,从电报电话到互联网传送,交流的速度越来越快,信息传输量越来越大。通过交流,人们编写程序的时候就可以实现远程方法调用,就像调用本地方法一样便捷,所以RPC技术也在发展,尤其近几年的微服务的大力推广,RPC技术应用的越来越广泛。

本质上讲,鸿雁传书也是一种RPC调用,只不过速度比较慢,可靠性不是那么高,现在RPC远程方法调用一般直接使用TCP或者HTTP实现。 HTTP的服务暴露方式比较简单,可以采用RESTful的方式提供通用的API, 客户端的调用也比较简单。直接TCP上实现的RPC远程方法调用性能优良,可以用在高吞吐低延迟的场景上。

近日江湖百晓生兄提供了一份最新的2020 RPC框架性能比较文档,我整理一下发表在这里,相关的测试代码可以在rpcx-benchmark找到。

历史benchmark比较:

当然,作为一份榜单,必然会有争议,争议无外乎以下几种:

  • 利益相关者争议: 因为我是rpcx作者,会不会特意给rpcx开小灶。答案是没有,因为测试代码明明白白的开源出来了,可以供大家review
  • 测试不全面:这是对的,没有一份测试满足所有的测试场景。有些场景是CPU敏感的、有些场景是IO敏感的,有些是内存敏感的业务,测试不太可能测试的各个方面,所以这一份测试只是针对一个场景的测试:通过序列化/反序列化一个比较大的protobuf,来模拟业务的处理。(代码中有delay参数,可以用来模拟耗时较长的服务,但是看这次的测试结果,这个场景没有覆盖。CPU耗时大家可扩充以下,通过计算阶乘或者挖矿的方式模拟CPU的消耗)
  • 测试结果不全面:这次测试只采集了并发数和耗时(latency), 并没有采集服务器和客户端的CPU/内存占用等指标。

这次测试的只是框架的性能,在评估一个框架的时候,还需要考虑框架的功能、易用性、活跃度等各个方面。

以上是测试背景。

测试环境

  • CPU: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz,两颗, 共20个物理core, 开超线程 40个逻辑core
  • 内存: 32G
  • 磁盘: SSD

测试数据

payload

所有框架传输的数据都是在客户端序列化好的protobuf对象,并使用一些测试数据填充这个对象。序列化后的payload大小是581个字节,每个框架都会增加一些额外的数据,大小不等。收到的数据类型,并且需要反序列化。

服务端收到这个数据后会反序列化,设置几个字段后再序列化返回客户端。

可以看出业务处理主要是对象的序列化和反序列化,这些操作对每一个框架都是一样的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
syntax = "proto2";
package proto;
option optimize_for = SPEED;
message BenchmarkMessage {
required string field1 = 1;
optional string field9 = 9;
optional string field18 = 18;
optional bool field80 = 80 [default=false];
optional bool field81 = 81 [default=true];
required int32 field2 = 2;
required int32 field3 = 3;
optional int32 field280 = 280;
optional int32 field6 = 6 [default=0];
optional int64 field22 = 22;
optional string field4 = 4;
repeated fixed64 field5 = 5;
optional bool field59 = 59 [default=false];
optional string field7 = 7;
optional int32 field16 = 16;
optional int32 field130 = 130 [default=0];
optional bool field12 = 12 [default=true];
optional bool field17 = 17 [default=true];
optional bool field13 = 13 [default=true];
optional bool field14 = 14 [default=true];
optional int32 field104 = 104 [default=0];
optional int32 field100 = 100 [default=0];
optional int32 field101 = 101 [default=0];
optional string field102 = 102;
optional string field103 = 103;
optional int32 field29 = 29 [default=0];
optional bool field30 = 30 [default=false];
optional int32 field60 = 60 [default=-1];
optional int32 field271 = 271 [default=-1];
optional int32 field272 = 272 [default=-1];
optional int32 field150 = 150;
optional int32 field23 = 23 [default=0];
optional bool field24 = 24 [default=false];
optional int32 field25 = 25 [default=0];
optional bool field78 = 78;
optional int32 field67 = 67 [default=0];
optional int32 field68 = 68;
optional int32 field128 = 128 [default=0];
optional string field129 = 129 [default="xxxxxxxxxxxxxxxxxxxxx"];
optional int32 field131 = 131 [default=0];
}

测试结果

吞吐率比较

耗时比较

完整的测试数据

dubbo

服务端:

1
2
ulimit -n 1000000
nohup java -jar dubbo-bench-provider-2.7.5.jar zookeeper://10.222.77.227:2181 &

客户端:

1
2
ulimit -n 1000000
nohup java -jar dubbo-bench-consumer-2.7.5.jar 100 10000000 zookeeper://10.222.77.227:2181 > result.log 2>&1 &

c=100, n=10000000

1
2
3
4
5
6
throughput (TPS) : 10590
mean: 1.323284
median: 1.000000
max: 3105.000000
min: 0.000000
99P: 2.000000

c=1000, n=10000000

1
2
3
4
5
6
throughput (TPS) : 9577
mean: 14.678306
median: 12.000000
max: 3249.000000
min: 0.000000
99P: 14.000000

go 标准库

c=100,n=10000000

1
2
3
4
5
6
7
took 64733.274973 ms for 10000000 requests
2020/01/18 15:27:09 sent requests : 10000000
2020/01/18 15:27:09 received requests : 10000000
2020/01/18 15:27:09 received requests_OK : 10000000
2020/01/18 15:27:09 throughput (TPS) : 154480
2020/01/18 15:27:09 mean: 611249 ns, median: 511101 ns, max: 8589768 ns, min: 75924 ns, p99.9: 2464773 ns
2020/01/18 15:27:09 mean: 0 ms, median: 0 ms, max: 8 ms, min: 0 ms, p99: 2 ms

c=1000,n=10000000

1
2
3
4
5
6
7
took 55479.341019 ms for 10000000 requests
2020/01/18 15:30:50 sent requests : 10000000
2020/01/18 15:30:50 received requests : 10000000
2020/01/18 15:30:50 received requests_OK : 10000000
2020/01/18 15:30:50 throughput (TPS) : 180247
2020/01/18 15:30:50 mean: 5206044 ns, median: 5163272 ns, max: 29196701 ns, min: 86510 ns, p99.9: 13032492 ns
2020/01/18 15:30:50 mean: 5 ms, median: 5 ms, max: 29 ms, min: 0 ms, p99: 13 ms

grpc

c=100,n=10000000

1
2
3
4
5
6
7
took 91803 ms for 10000000 requests
2020/01/18 15:45:00 grpc_mclient.go:109: INFO : sent requests : 10000000
2020/01/18 15:45:00 grpc_mclient.go:110: INFO : received requests : 10000000
2020/01/18 15:45:00 grpc_mclient.go:111: INFO : received requests_OK : 10000000
2020/01/18 15:45:00 grpc_mclient.go:112: INFO : throughput (TPS) : 108928
2020/01/18 15:45:00 grpc_mclient.go:113: INFO : mean: 857796 ns, median: 672012 ns, max: 43451467 ns, min: 125892 ns, p99: 6719222 ns
2020/01/18 15:45:00 grpc_mclient.go:114: INFO : mean: 0 ms, median: 0 ms, max: 43 ms, min: 0 ms, p99: 6 ms

c=1000,n=10000000

1
2
3
4
5
6
7
took 75736 ms for 10000000 requests
2020/01/18 15:46:44 grpc_mclient.go:109: INFO : sent requests : 10000000
2020/01/18 15:46:44 grpc_mclient.go:110: INFO : received requests : 10000000
2020/01/18 15:46:44 grpc_mclient.go:111: INFO : received requests_OK : 10000000
2020/01/18 15:46:44 grpc_mclient.go:112: INFO : throughput (TPS) : 132037
2020/01/18 15:46:44 grpc_mclient.go:113: INFO : mean: 7148157 ns, median: 5975982 ns, max: 133300722 ns, min: 137300 ns, p99: 58021151 ns
2020/01/18 15:46:44 grpc_mclient.go:114: INFO : mean: 7 ms, median: 5 ms, max: 133 ms, min: 0 ms, p99: 58 ms

rpcx

c=100,n=10000000

1
2
3
4
5
6
7
took 99082 ms for 10000000 requests
2020/01/18 16:02:14 rpcx_mclient.go:121: INFO : sent requests : 10000000
2020/01/18 16:02:14 rpcx_mclient.go:122: INFO : received requests : 10000000
2020/01/18 16:02:14 rpcx_mclient.go:123: INFO : received requests_OK : 10000000
2020/01/18 16:02:14 rpcx_mclient.go:124: INFO : throughput (TPS) : 100926
2020/01/18 16:02:14 rpcx_mclient.go:125: INFO : mean: 954794 ns, median: 776113 ns, max: 13546797 ns, min: 80234 ns, p99: 4047876 ns
2020/01/18 16:02:14 rpcx_mclient.go:126: INFO : mean: 0 ms, median: 0 ms, max: 13 ms, min: 0 ms, p99: 4 ms

c=1000,n=10000000

1
2
3
4
5
6
7
took 58275 ms for 10000000 requests
2020/01/18 16:03:45 rpcx_mclient.go:121: INFO : sent requests : 10000000
2020/01/18 16:03:45 rpcx_mclient.go:122: INFO : received requests : 10000000
2020/01/18 16:03:45 rpcx_mclient.go:123: INFO : received requests_OK : 10000000
2020/01/18 16:03:45 rpcx_mclient.go:124: INFO : throughput (TPS) : 171600
2020/01/18 16:03:45 rpcx_mclient.go:125: INFO : mean: 5474568 ns, median: 5369917 ns, max: 33885454 ns, min: 78216 ns, p99: 16806552 ns
2020/01/18 16:03:45 rpcx_mclient.go:126: INFO : mean: 5 ms, median: 5 ms, max: 33 ms, min: 0 ms, p99: 16 ms

async-rpcx

c=100,n=10000000

1
2
3
4
5
6
7
took 54582 ms for 10000000 requests
2020/01/18 16:10:14 rpcx_mclient.go:145: INFO : sent requests : 10000000
2020/01/18 16:10:14 rpcx_mclient.go:146: INFO : received requests : 10000000
2020/01/18 16:10:14 rpcx_mclient.go:147: INFO : received requests_OK : 10000000
2020/01/18 16:10:14 rpcx_mclient.go:148: INFO : throughput (TPS) : 183210
2020/01/18 16:10:14 rpcx_mclient.go:149: INFO : mean: 235075061 ns, median: 238628422 ns, max: 713519584 ns, min: 1853214 ns, p99: 591043364 ns
2020/01/18 16:10:14 rpcx_mclient.go:150: INFO : mean: 235 ms, median: 238 ms, max: 713 ms, min: 1 ms, p99: 591 ms

c=1000,n=10000000

1
2
3
4
5
6
7
took 55263 ms for 10000000 requests
2020/01/18 16:07:21 rpcx_mclient.go:145: INFO : sent requests : 10000000
2020/01/18 16:07:21 rpcx_mclient.go:146: INFO : received requests : 10000000
2020/01/18 16:07:21 rpcx_mclient.go:147: INFO : received requests_OK : 10000000
2020/01/18 16:07:21 rpcx_mclient.go:148: INFO : throughput (TPS) : 180952
2020/01/18 16:07:21 rpcx_mclient.go:149: INFO : mean: 843985176 ns, median: 825286868 ns, max: 1645398520 ns, min: 15688776 ns, p99: 1532695207 ns
2020/01/18 16:07:21 rpcx_mclient.go:150: INFO : mean: 843 ms, median: 825 ms, max: 1645 ms, min: 15 ms, p99: 1532 ms

thrift

c=100,n=10000000

1
2
3
4
5
6
7
8
9
10
java -cp thrift-1.0-SNAPSHOT.jar com.colobu.thrift.AppClient 10.41.15.226 100 10000000
sent requests : 10000000
received requests : 10000000
received requests_OK : 10000000
throughput (TPS) : 18798
mean: 0.674689
median: 1.000000
max: 19.000000
min: 0.000000
99P: 4.000000

c=1000,n=10000000

1
2
3
4
5
6
7
8
9
10
java -cp thrift-1.0-SNAPSHOT.jar com.colobu.thrift.AppClient 10.41.15.226 1000 10000000
sent requests : 10000000
received requests : 10000000
received requests_OK : 10000000
throughput (TPS) : 19151
mean: 7.158192
median: 7.000000
max: 181.000000
min: 0.000000
99P: 19.000000

tarsgo

c=100,n=10000000

1
2
3
4
5
6
7
tarsgo_mclient.go:96: INFO : took 146657 ms for 10000000 requests
2020/01/18 16:40:37 tarsgo_mclient.go:113: INFO : sent requests : 10000000
2020/01/18 16:40:37 tarsgo_mclient.go:114: INFO : received requests : 10000000
2020/01/18 16:40:37 tarsgo_mclient.go:115: INFO : received requests_OK : 0
2020/01/18 16:40:37 tarsgo_mclient.go:116: INFO : throughput (TPS) : 68186
2020/01/18 16:40:37 tarsgo_mclient.go:117: INFO : mean: 1463218 ns, median: 897658 ns, max: 833883541 ns, min: 40970 ns, p99: 24386698 ns
2020/01/18 16:40:37 tarsgo_mclient.go:118: INFO : mean: 1 ms, median: 0 ms, max: 833 ms, min: 0 ms, p99: 24 ms

c=1000,n=10000000

1
2
3
4
5
6
7
8
9
10
11
./tars_client -config benchmark.conf
2020/01/18 16:42:19 tarsgo_mclient.go:42: INFO : concurrency: 1000
requests per client: 10000
2020/01/18 16:42:19 tarsgo_mclient.go:47: INFO : message size: 581 bytes
2020/01/18 16:44:33 tarsgo_mclient.go:96: INFO : took 133438 ms for 10000000 requests
2020/01/18 16:44:40 tarsgo_mclient.go:113: INFO : sent requests : 10000000
2020/01/18 16:44:40 tarsgo_mclient.go:114: INFO : received requests : 10000000
2020/01/18 16:44:40 tarsgo_mclient.go:115: INFO : received requests_OK : 0
2020/01/18 16:44:40 tarsgo_mclient.go:116: INFO : throughput (TPS) : 74941
2020/01/18 16:44:40 tarsgo_mclient.go:117: INFO : mean: 13189350 ns, median: 255591 ns, max: 1020977156 ns, min: 41305 ns, p99: 808153981 ns
2020/01/18 16:44:40 tarsgo_mclient.go:118: INFO : mean: 13 ms, median: 0 ms, max: 1020 ms, min: 0 ms, p99: 808 ms

hprose

c=100,n=10000000

1
2
3
4
5
6
7
info took 85780 ms for 10000000 requests
2020/01/18 16:53:47 info sent requests : 10000000
2020/01/18 16:53:47 info received requests : 10000000
2020/01/18 16:53:47 info received requests_OK : 10000000
2020/01/18 16:53:47 info throughput (TPS) : 116577
2020/01/18 16:53:47 info mean: 855396 ns, median: 611286 ns, max: 43307064 ns, min: 74772 ns, p99: 8799176 ns
2020/01/18 16:53:47 info mean: 0 ms, median: 0 ms, max: 43 ms, min: 0 ms, p99: 8 ms

c=1000,n=10000000

1
2
3
4
5
6
7
info took 55890 ms for 10000000 requests
2020/01/18 16:55:23 info sent requests : 10000000
2020/01/18 16:55:23 info received requests : 10000000
2020/01/18 16:55:23 info received requests_OK : 10000000
2020/01/18 16:55:23 info throughput (TPS) : 178922
2020/01/18 16:55:23 info mean: 5574522 ns, median: 5450258 ns, max: 112631418 ns, min: 86944 ns, p99: 15302910 ns
2020/01/18 16:55:23 info mean: 5 ms, median: 5 ms, max: 112 ms, min: 0 ms, p99: 15 ms

go-micro

c=100,n=10000000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
./micro_client
2020/01/18 16:08:32 gomicro_client.go:43: INFO : 192.168.1.226:8972 1000000 100
2020/01/18 16:08:32 gomicro_client.go:47: INFO : concurrency: 100
requests per client: 10000
2020/01/18 16:08:32 gomicro_client.go:52: INFO : message size: 581 bytes
2020/01/18 16:10:30 gomicro_client.go:102: INFO : took 117938 ms for 1000000 requests
2020/01/18 16:10:31 gomicro_client.go:119: INFO : sent requests : 1000000
2020/01/18 16:10:31 gomicro_client.go:120: INFO : received requests : 1000000
2020/01/18 16:10:31 gomicro_client.go:121: INFO : received requests_OK : 0
2020/01/18 16:10:31 gomicro_client.go:122: INFO : throughput (TPS) : 8479
2020/01/18 16:10:31 gomicro_client.go:123: INFO : mean: 11462696 ns, median: 10945519 ns, max: 1015949740 ns, min: 10513193 ns, p99: 17612504 ns
2020/01/18 16:10:31 gomicro_client.go:124: INFO : mean: 11 ms, median: 10 ms, max: 1015 ms, min: 10 ms, p99: 17 ms

c=1000,n=10000000

1
2
3
4
5
6
7
8
9
10
11
12
13
2020/01/18 16:11:09 gomicro_client.go:43: INFO : 192.168.1.226:8972 1000000 1000
2020/01/18 16:11:09 gomicro_client.go:47: INFO : concurrency: 1000
requests per client: 1000
2020/01/18 16:11:09 gomicro_client.go:52: INFO : message size: 581 bytes
2020/01/18 16:12:01 gomicro_client.go:102: INFO : took 51922 ms for 1000000 requests
2020/01/18 16:12:01 gomicro_client.go:119: INFO : sent requests : 1000000
2020/01/18 16:12:01 gomicro_client.go:120: INFO : received requests : 1000000
2020/01/18 16:12:01 gomicro_client.go:121: INFO : received requests_OK : 0
2020/01/18 16:12:01 gomicro_client.go:122: INFO : throughput (TPS) : 19259
2020/01/18 16:12:01 gomicro_client.go:123: INFO : mean: 46115145 ns, median: 22666560 ns, max: 3060928934 ns, min: 10578529 ns, p99: 1050307726 ns
2020/01/18 16:12:01 gomicro_client.go:124: INFO : mean: 46 ms, median: 22 ms, max: 3060 ms, min: 10 ms, p99: 1050 ms