常用web框架性能基准

techempower网站已经做了9轮的常用web框架的测试,它提供了三种硬件环境,几种测试case,得出了一系列的web框架的性能基准。 对于架构师而言,这是一个很好的评估选择web框架的参考资料。
完整的测试套件可以在github上下载。

硬件环境包括三种

  • Peak: Dell R720xd dual Xeon E5-2660 v2 servers with 32 GB memory; database servers equipped with SSDs in RAID; switched 10-gigabit Ethernet
  • i7: Sandy Bridge Core i7-2600K workstations with 8 GB memory (early 2011 vintage); database server equipped with Samsung 840 Pro SSD; switched gigabit Ethernet
  • EC2: Amazon EC2 m1.large instances; switched gigabit Ethernet

在不同的环境下各web框架的表现可能有些变化。
各web框架机及其所需环境可以参看 environment

奇怪没有tomcat的测试,而是使用Resin作为servlet的容器

测试case

测试case的介绍可以参看: code,
包括:

  1. JSON serialization
  2. Single database query
  3. Multiple database queries
  4. Fortunes
  5. Database updates
  6. Plaintext

测试结果

针对每种硬件和测试case有一个排名。这里列出了在Peak硬件环境下返回纯文本Plaintext的性能基准:

FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
cpoll_cppsp6,738,911
100.0%
McrC++CplNonLinRea0
undertow4,162,894
61.8%
PltJavUtwNonLinRea95
undertow edge4,151,495
61.6%
PltJavUndNonLinRea4
netty2,756,821
40.9%
PltJavNtyNonLinRea0
plain2,158,968
32.0%
FulScaPlaNonLinRea0
grizzly2,120,141
31.5%
McrJavSvtGrzLinRea0
servlet1,776,915
26.4%
PltJavSvtResLinRea0
plain-servlet-linux1,576,626
23.4%
FulScaPlaNonLinRea0
openresty1,561,596
23.2%
PltLuaOpRngxLinRea6
plain-windows1,483,529
22.0%
FulScaPlaNonWinRea0
gemini1,357,894
20.2%
FulJavSvtResLinRea0
jetty-servlet1,167,791
17.3%
PltJavJtyJtyLinRea408
ur/web712,499
10.6%
FulUrUr/NonLinRea0
go490,728
7.3%
PltGoGoNonLinRea0
rack-jruby460,183
6.8%
PltRbyJRbTorLinRea186
wicket408,004
6.1%
FulJavSvtResLinRea0
revel394,470
5.9%
FulGoGoNonLinRea0
lapis365,942
5.4%
FulLuaOpRngxLinRea65
activeweb339,627
5.0%
PltJavActResLinRea0
falcore333,658
5.0%
McrGoGoNonLinRea0
treefrog-thread312,573
4.6%
FulC++TreNonLinRea260
nodejs289,578
4.3%
PltJSnjsNonLinRea0
wsgi251,501
3.7%
PltPyGunNonLinRea0
falcon-pypy205,929
3.1%
McrPyGunNonLinRea0
falcon203,963
3.0%
McrPyGunNonLinRea0
bottle-pypy199,254
3.0%
McrPyGunNonLinRea0
php195,923
2.9%
PltPHPFPMngxLinRea0
tornado-pypy192,795
2.9%
PltPyTorNonLinRea0
compojure176,734
2.6%
McrCljSvtResLinRea0
bottle172,662
2.6%
McrPyGunNonLinRea0
falcon-py3169,519
2.5%
McrPyGunNonLinRea0
bottle-py3143,681
2.1%
McrPyGunNonLinRea0
rack-ruby131,659
2.0%
PltRbyRacUniLinRea5
ringojs123,575
1.8%
PltJSRinJtyLinRea54
aspnet-stripped109,136
1.6%
FulC#NetIISWinStr0
evhttp-sharp108,436
1.6%
McrC#MonNonLinRea0
http-listener104,047
1.5%
PltC#NethtsWinRea0
tornado98,211
1.5%
PltPyTorNonLinRea0
ringojs-conv94,583
1.4%
McrJSRinJtyLinRea171
wsgi-nginx-uwsgi92,231
1.4%
PltPyuWSngxLinRea0
sinatra-jruby90,035
1.3%
McrRbyJRbTorLinRea0
spring86,885
1.3%
FulJavSvtTomLinRea0
tornado-py380,487
1.2%
PltPyTorNonLinRea0
luminus72,618
1.1%
McrCljSvtResLinRea0
grails72,348
1.1%
FulGrvSvtResLinRea0
snap72,217
1.1%
McrHklSnpNonLinRea0
aspnet71,589
1.1%
FulC#NetIISWinRea0
flask66,463
1.0%
McrPyGunNonLinRea0
flask-py364,620
1.0%
McrPyGunNonLinRea0
flask-pypy60,218
0.9%
McrPyGunNonLinRea0
dart57,551
0.9%
PltDarDarngxLinRea0
flask-nginx-uwsgi55,204
0.8%
McrPyuWSngxLinRea85
start54,888
0.8%
McrDarDarngxLinRea0
stream54,160
0.8%
McrDarDarngxLinRea0
bottle-nginx-uwsgi40,347
0.6%
McrPyuWSngxLinRea3
pyramid-py240,071
0.6%
FulPyWsgGunLinRea9
pyramid-py337,585
0.6%
FulPyWsgGunLinRea5
ninja-standalone35,271
0.5%
FulJavJtyJtyLinRea925
ninja-standalone35,177
0.5%
FulJavSvtResLinRea0
phpixie32,876
0.5%
FulPHPFPMngxLinRea0
sinatra-ruby30,541
0.5%
McrRbyRacUniLinRea3,139,167
treefrog17,545
0.3%
FulC++TreNonLinRea171
spark5,769
0.1%
McrJavSvtResLinRea2,397
cake5,207
0.1%
FulPHPFPMngxLinRea0
aspnet-mvc-mono5,192
0.1%
FulC#MonngxLinRea0
phreeze3,419
0.1%
McrPHPFPMngxLinRea931,997
servicestack-nginx-d2,343
0.0%
PltC#MonngxLinRea2
hhvm938
0.0%
PltPHPhhvHPHLinRea2,231
servicestackDid not completePltC#MonXSPLinRea
silexDid not completeMcrPHPFPMngxLinRea
sprayRemoved at request of framework maintainer - outdated versionMcrScaSprNonLinRea
vertxRemoved at request of framework maintainer - outdated versionPltJavvtxNonLinRea

其它的Benchmark

因为我最近关注Scala/spray和netty的性能,所以还搜集了一些其他人所做的Benchmark数据。
jakubkulhan提供了Node.js, Spray, Erlang, Http-kit, Warp, Tornado,和 Puma的benchmark: hit-server-bench
每秒请求数 rps

Environment Req/s (c=1) Req/s (c=10) Req/s (c=50) Req/s (c=100)
Node.js 10215.47 38447.10 51362.60 52722.19
Spray 8139.76 30282.27 69328.26 69629.64
Erlang 24.70 246.98 1240.38 2474.18
Http-kit 8789.63 61355.26 73457.67 73475.39
Warp 12640.29 27749.54 26365.52 23916.69
Tornado 2356.82 4590.33 4422.08 4155.61
Puma 10048.54 19289.60 16280.00 16698.86
Netty 28409.44 116605.01 186109.06 180001.91
Spark (Jetty) 10982.23 44533.38 31624.97 57425.96
Spring Boot (Tomcat) 2652.85 13450.45 26336.17 31288.19
Reactor 12899.17 50113.60 66310.16 65718.33
PHP-FPM 2907.34 9968.78 9004.46 9425.08
HHVM 1323.99 6434.97 9192.72 9743.09
ReactPHP (PHP) 1636.49 9454.17 13676.61 12061.91
ReactPHP (HHVM) 2087.04 12185.55 16151.42 12036.11

平均延迟时间

Environment Latency (c=1) Latency (c=10) Latency (c=50) Latency (c=100)
Node.js 83.59us 253.57us 1.03ms 2.19ms
Spray 109.95us 1.41ms 1.40ms 1.86ms
Erlang 40.60ms 40.58ms 40.35ms 40.43ms
Http-kit 98.01us 770.13us 683.12us 1.36ms
Warp 64.65us 345.92us 1.89ms 4.20ms
Tornado 414.73us 2.28ms 12.21ms 24.95ms
Puma 96.37us 510.70us 2.83ms 5.88ms
Netty 53.17us 113.52us 635.84us 1.18ms
Spark (Jetty) 2.04ms 617.83us 6.23ms 7.94ms
Spring Boot (Tomcat) 7.09ms 815.26us 2.25ms 3.81ms
Reactor 2.19ms 3.93ms 6.72ms 8.49ms
PHP-FPM 355.04us 1.26ms 6.37ms 11.60ms
HHVM 824.73us 1.53ms 5.62ms 10.47ms
ReactPHP (PHP) 626.27us 1.05ms 3.81ms 10.83ms
ReactPHP (HHVM) 481.31us 811.58us 3.21ms 15.36ms

看起来Netty最好

评论