高并发编程
by smallnest

常用web框架性能基准

techempower网站已经做了9轮的常用web框架的测试，它提供了三种硬件环境，几种测试case，得出了一系列的web框架的性能基准。 对于架构师而言，这是一个很好的评估选择web框架的参考资料。
完整的测试套件可以在github上下载。

硬件环境包括三种

  • Peak: Dell R720xd dual Xeon E5-2660 v2 servers with 32 GB memory; database servers equipped with SSDs in RAID; switched 10-gigabit Ethernet
  • i7: Sandy Bridge Core i7-2600K workstations with 8 GB memory (early 2011 vintage); database server equipped with Samsung 840 Pro SSD; switched gigabit Ethernet
  • EC2: Amazon EC2 m1.large instances; switched gigabit Ethernet

在不同的环境下各web框架的表现可能有些变化。
各web框架机及其所需环境可以参看 environment

奇怪没有tomcat的测试，而是使用Resin作为servlet的容器

测试case

测试case的介绍可以参看: code,
包括：

  1. JSON serialization
  2. Single database query
  3. Multiple database queries
  4. Fortunes
  5. Database updates
  6. Plaintext

测试结果

针对每种硬件和测试case有一个排名。这里列出了在Peak硬件环境下返回纯文本Plaintext的性能基准：

FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
cpoll_cppsp6,738,911
100.0%		McrC++CplNonLinRea0
undertow4,162,894
61.8%		PltJavUtwNonLinRea95
undertow edge4,151,495
61.6%		PltJavUndNonLinRea4
netty2,756,821
40.9%		PltJavNtyNonLinRea0
plain2,158,968
32.0%		FulScaPlaNonLinRea0
grizzly2,120,141
31.5%		McrJavSvtGrzLinRea0
servlet1,776,915
26.4%		PltJavSvtResLinRea0
plain-servlet-linux1,576,626
23.4%		FulScaPlaNonLinRea0
openresty1,561,596
23.2%		PltLuaOpRngxLinRea6
plain-windows1,483,529
22.0%		FulScaPlaNonWinRea0
gemini1,357,894
20.2%		FulJavSvtResLinRea0
jetty-servlet1,167,791
17.3%		PltJavJtyJtyLinRea408
ur/web712,499
10.6%		FulUrUr/NonLinRea0
go490,728
7.3%		PltGoGoNonLinRea0
rack-jruby460,183
6.8%		PltRbyJRbTorLinRea186
wicket408,004
6.1%		FulJavSvtResLinRea0
revel394,470
5.9%		FulGoGoNonLinRea0
lapis365,942
5.4%		FulLuaOpRngxLinRea65
activeweb339,627
5.0%		PltJavActResLinRea0
falcore333,658
5.0%		McrGoGoNonLinRea0
treefrog-thread312,573
4.6%		FulC++TreNonLinRea260
nodejs289,578
4.3%		PltJSnjsNonLinRea0
wsgi251,501
3.7%		PltPyGunNonLinRea0
falcon-pypy205,929
3.1%		McrPyGunNonLinRea0
falcon203,963
3.0%		McrPyGunNonLinRea0
bottle-pypy199,254
3.0%		McrPyGunNonLinRea0
php195,923
2.9%		PltPHPFPMngxLinRea0
tornado-pypy192,795
2.9%		PltPyTorNonLinRea0
compojure176,734
2.6%		McrCljSvtResLinRea0
bottle172,662
2.6%		McrPyGunNonLinRea0
falcon-py3169,519
2.5%		McrPyGunNonLinRea0
bottle-py3143,681
2.1%		McrPyGunNonLinRea0
rack-ruby131,659
2.0%		PltRbyRacUniLinRea5
ringojs123,575
1.8%		PltJSRinJtyLinRea54
aspnet-stripped109,136
1.6%		FulC#NetIISWinStr0
evhttp-sharp108,436
1.6%		McrC#MonNonLinRea0
http-listener104,047
1.5%		PltC#NethtsWinRea0
tornado98,211
1.5%		PltPyTorNonLinRea0
ringojs-conv94,583
1.4%		McrJSRinJtyLinRea171
wsgi-nginx-uwsgi92,231
1.4%		PltPyuWSngxLinRea0
sinatra-jruby90,035
1.3%		McrRbyJRbTorLinRea0
spring86,885
1.3%		FulJavSvtTomLinRea0
tornado-py380,487
1.2%		PltPyTorNonLinRea0
luminus72,618
1.1%		McrCljSvtResLinRea0
grails72,348
1.1%		FulGrvSvtResLinRea0
snap72,217
1.1%		McrHklSnpNonLinRea0
aspnet71,589
1.1%		FulC#NetIISWinRea0
flask66,463
1.0%		McrPyGunNonLinRea0
flask-py364,620
1.0%		McrPyGunNonLinRea0
flask-pypy60,218
0.9%		McrPyGunNonLinRea0
dart57,551
0.9%		PltDarDarngxLinRea0
flask-nginx-uwsgi55,204
0.8%		McrPyuWSngxLinRea85
start54,888
0.8%		McrDarDarngxLinRea0
stream54,160
0.8%		McrDarDarngxLinRea0
bottle-nginx-uwsgi40,347
0.6%		McrPyuWSngxLinRea3
pyramid-py240,071
0.6%		FulPyWsgGunLinRea9
pyramid-py337,585
0.6%		FulPyWsgGunLinRea5
ninja-standalone35,271
0.5%		FulJavJtyJtyLinRea925
ninja-standalone35,177
0.5%		FulJavSvtResLinRea0
phpixie32,876
0.5%		FulPHPFPMngxLinRea0
sinatra-ruby30,541
0.5%		McrRbyRacUniLinRea3,139,167
treefrog17,545
0.3%		FulC++TreNonLinRea171
spark5,769
0.1%		McrJavSvtResLinRea2,397
cake5,207
0.1%		FulPHPFPMngxLinRea0
aspnet-mvc-mono5,192
0.1%		FulC#MonngxLinRea0
phreeze3,419
0.1%		McrPHPFPMngxLinRea931,997
servicestack-nginx-d2,343
0.0%		PltC#MonngxLinRea2
hhvm938
0.0%		PltPHPhhvHPHLinRea2,231
servicestack—Did not completePltC#MonXSPLinRea—
silex—Did not completeMcrPHPFPMngxLinRea—
spray—Removed at request of framework maintainer - outdated versionMcrScaSprNonLinRea—
vertx—Removed at request of framework maintainer - outdated versionPltJavvtxNonLinRea—

其它的Benchmark

因为我最近关注Scala/spray和netty的性能，所以还搜集了一些其他人所做的Benchmark数据。
jakubkulhan提供了Node.js, Spray, Erlang, Http-kit, Warp, Tornado,和 Puma的benchmark： hit-server-bench。
每秒请求数 rps

EnvironmentReq/s (c=1)Req/s (c=10)Req/s (c=50)Req/s (c=100)
Node.js10215.4738447.1051362.6052722.19
Spray8139.7630282.2769328.2669629.64
Erlang24.70246.981240.382474.18
Http-kit8789.6361355.2673457.6773475.39
Warp12640.2927749.5426365.5223916.69
Tornado2356.824590.334422.084155.61
Puma10048.5419289.6016280.0016698.86
Netty28409.44116605.01186109.06180001.91
Spark (Jetty)10982.2344533.3831624.9757425.96
Spring Boot (Tomcat)2652.8513450.4526336.1731288.19
Reactor12899.1750113.6066310.1665718.33
PHP-FPM2907.349968.789004.469425.08
HHVM1323.996434.979192.729743.09
ReactPHP (PHP)1636.499454.1713676.6112061.91
ReactPHP (HHVM)2087.0412185.5516151.4212036.11

平均延迟时间

EnvironmentLatency (c=1)Latency (c=10)Latency (c=50)Latency (c=100)
Node.js83.59us253.57us1.03ms2.19ms
Spray109.95us1.41ms1.40ms1.86ms
Erlang40.60ms40.58ms40.35ms40.43ms
Http-kit98.01us770.13us683.12us1.36ms
Warp64.65us345.92us1.89ms4.20ms
Tornado414.73us2.28ms12.21ms24.95ms
Puma96.37us510.70us2.83ms5.88ms
Netty53.17us113.52us635.84us1.18ms
Spark (Jetty)2.04ms617.83us6.23ms7.94ms
Spring Boot (Tomcat)7.09ms815.26us2.25ms3.81ms
Reactor2.19ms3.93ms6.72ms8.49ms
PHP-FPM355.04us1.26ms6.37ms11.60ms
HHVM824.73us1.53ms5.62ms10.47ms
ReactPHP (PHP)626.27us1.05ms3.81ms10.83ms
ReactPHP (HHVM)481.31us811.58us3.21ms15.36ms

看起来Netty最好