by smallnest

常用web框架性能基准

techempower网站已经做了9轮的常用web框架的测试，它提供了三种硬件环境，几种测试case，得出了一系列的web框架的性能基准。对于架构师而言，这是一个很好的评估选择web框架的参考资料。
完整的测试套件可以在github上下载。

硬件环境包括三种

Peak: Dell R720xd dual Xeon E5-2660 v2 servers with 32 GB memory; database servers equipped with SSDs in RAID; switched 10-gigabit Ethernet
i7: Sandy Bridge Core i7-2600K workstations with 8 GB memory (early 2011 vintage); database server equipped with Samsung 840 Pro SSD; switched gigabit Ethernet
EC2: Amazon EC2 m1.large instances; switched gigabit Ethernet

在不同的环境下各web框架的表现可能有些变化。
各web框架机及其所需环境可以参看 environment

奇怪没有tomcat的测试，而是使用Resin作为servlet的容器

测试case

测试case的介绍可以参看: code,
包括：

JSON serialization
Single database query
Multiple database queries
Fortunes
Database updates
Plaintext

测试结果

针对每种硬件和测试case有一个排名。这里列出了在Peak硬件环境下返回纯文本Plaintext的性能基准：

Framework	Best performance (higher is better)		Cls	Lng	Plt	FE	Aos	DB	Dos	Orm	IA	Errors
cpoll_cppsp	6,738,911	100.0%	Mcr	C++	Cpl	Non	Lin	Non	Lin	Raw	Rea	0
undertow	4,162,894	61.8%	Plt	Jav	Utw	Non	Lin	Non	Lin	Raw	Rea	95
undertow edge	4,151,495	61.6%	Plt	Jav	Und	Non	Lin	Non	Lin	Raw	Rea	4
netty	2,756,821	40.9%	Plt	Jav	Nty	Non	Lin	Non	Lin	Raw	Rea	0
plain	2,158,968	32.0%	Ful	Sca	Pla	Non	Lin	My	Lin	Mcr	Rea	0
grizzly	2,120,141	31.5%	Mcr	Jav	Svt	Grz	Lin	Non	Lin	Raw	Rea	0
servlet	1,776,915	26.4%	Plt	Jav	Svt	Res	Lin	Non	Lin	Raw	Rea	0
plain-servlet-linux	1,576,626	23.4%	Ful	Sca	Pla	Non	Lin	My	Lin	Ful	Rea	0
openresty	1,561,596	23.2%	Plt	Lua	OpR	ngx	Lin	My	Lin	Raw	Rea	6
plain-windows	1,483,529	22.0%	Ful	Sca	Pla	Non	Win	My	Lin	Ful	Rea	0
gemini	1,357,894	20.2%	Ful	Jav	Svt	Res	Lin	My	Lin	Mcr	Rea	0
jetty-servlet	1,167,791	17.3%	Plt	Jav	Jty	Jty	Lin	Non	Lin	Raw	Rea	408
ur/web	712,499	10.6%	Ful	Ur	Ur/	Non	Lin	Non	Lin	Mcr	Rea	0
go	490,728	7.3%	Plt	Go	Go	Non	Lin	My	Lin	Raw	Rea	0
rack-jruby	460,183	6.8%	Plt	Rby	JRb	Tor	Lin	Non	Lin	Ful	Rea	186
wicket	408,004	6.1%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
revel	394,470	5.9%	Ful	Go	Go	Non	Lin	My	Lin	Raw	Rea	0
lapis	365,942	5.4%	Ful	Lua	OpR	ngx	Lin	Pg	Lin	Ful	Rea	65
activeweb	339,627	5.0%	Plt	Jav	Act	Res	Lin	Non	Lin	Raw	Rea	0
falcore	333,658	5.0%	Mcr	Go	Go	Non	Lin	My	Lin	Raw	Rea	0
treefrog-thread	312,573	4.6%	Ful	C++	Tre	Non	Lin	My	Lin	Mcr	Rea	260
nodejs	289,578	4.3%	Plt	JS	njs	Non	Lin	Non	Lin	Raw	Rea	0
wsgi	251,501	3.7%	Plt	Py	Gun	Non	Lin	Non	Lin	Raw	Rea	0
falcon-pypy	205,929	3.1%	Mcr	Py	Gun	Non	Lin	Non	Lin	Raw	Rea	0
falcon	203,963	3.0%	Mcr	Py	Gun	Non	Lin	Non	Lin	Raw	Rea	0
bottle-pypy	199,254	3.0%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
php	195,923	2.9%	Plt	PHP	FPM	ngx	Lin	My	Lin	Ful	Rea	0
tornado-pypy	192,795	2.9%	Plt	Py	Tor	Non	Lin	Mo	Lin	Raw	Rea	0
compojure	176,734	2.6%	Mcr	Clj	Svt	Res	Lin	My	Lin	Mcr	Rea	0
bottle	172,662	2.6%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
falcon-py3	169,519	2.5%	Mcr	Py	Gun	Non	Lin	Non	Lin	Raw	Rea	0
bottle-py3	143,681	2.1%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
rack-ruby	131,659	2.0%	Plt	Rby	Rac	Uni	Lin	Non	Lin	Ful	Rea	5
ringojs	123,575	1.8%	Plt	JS	Rin	Jty	Lin	Non	Lin	Raw	Rea	54
aspnet-stripped	109,136	1.6%	Ful	C#	Net	IIS	Win	Non	Lin	Raw	Str	0
evhttp-sharp	108,436	1.6%	Mcr	C#	Mon	Non	Lin	Non	Lin	Raw	Rea	0
http-listener	104,047	1.5%	Plt	C#	Net	hts	Win	Non	Lin	Raw	Rea	0
tornado	98,211	1.5%	Plt	Py	Tor	Non	Lin	Mo	Lin	Raw	Rea	0
ringojs-conv	94,583	1.4%	Mcr	JS	Rin	Jty	Lin	My	Lin	Mcr	Rea	171
wsgi-nginx-uwsgi	92,231	1.4%	Plt	Py	uWS	ngx	Lin	Non	Lin	Raw	Rea	0
sinatra-jruby	90,035	1.3%	Mcr	Rby	JRb	Tor	Lin	My	Lin	Ful	Rea	0
spring	86,885	1.3%	Ful	Jav	Svt	Tom	Lin	My	Lin	Ful	Rea	0
tornado-py3	80,487	1.2%	Plt	Py	Tor	Non	Lin	Mo	Lin	Raw	Rea	0
luminus	72,618	1.1%	Mcr	Clj	Svt	Res	Lin	My	Lin	Ful	Rea	0
grails	72,348	1.1%	Ful	Grv	Svt	Res	Lin	My	Lin	Ful	Rea	0
snap	72,217	1.1%	Mcr	Hkl	Snp	Non	Lin	My	Lin	Ful	Rea	0
aspnet	71,589	1.1%	Ful	C#	Net	IIS	Win	Non	Lin	Raw	Rea	0
flask	66,463	1.0%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
flask-py3	64,620	1.0%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
flask-pypy	60,218	0.9%	Mcr	Py	Gun	Non	Lin	My	Lin	Ful	Rea	0
dart	57,551	0.9%	Plt	Dar	Dar	ngx	Lin	Pg	Lin	Raw	Rea	0
flask-nginx-uwsgi	55,204	0.8%	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	85
start	54,888	0.8%	Mcr	Dar	Dar	ngx	Lin	Pg	Lin	Raw	Rea	0
stream	54,160	0.8%	Mcr	Dar	Dar	ngx	Lin	Pg	Lin	Raw	Rea	0
bottle-nginx-uwsgi	40,347	0.6%	Mcr	Py	uWS	ngx	Lin	My	Lin	Ful	Rea	3
pyramid-py2	40,071	0.6%	Ful	Py	Wsg	Gun	Lin	Pg	Lin	Ful	Rea	9
pyramid-py3	37,585	0.6%	Ful	Py	Wsg	Gun	Lin	Pg	Lin	Ful	Rea	5
ninja-standalone	35,271	0.5%	Ful	Jav	Jty	Jty	Lin	My	Lin	Ful	Rea	925
ninja-standalone	35,177	0.5%	Ful	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	0
phpixie	32,876	0.5%	Ful	PHP	FPM	ngx	Lin	My	Lin	Ful	Rea	0
sinatra-ruby	30,541	0.5%	Mcr	Rby	Rac	Uni	Lin	My	Lin	Ful	Rea	3,139,167
treefrog	17,545	0.3%	Ful	C++	Tre	Non	Lin	My	Lin	Mcr	Rea	171
spark	5,769	0.1%	Mcr	Jav	Svt	Res	Lin	My	Lin	Ful	Rea	2,397
cake	5,207	0.1%	Ful	PHP	FPM	ngx	Lin	My	Lin	Ful	Rea	0
aspnet-mvc-mono	5,192	0.1%	Ful	C#	Mon	ngx	Lin	Non	Lin	Raw	Rea	0
phreeze	3,419	0.1%	Mcr	PHP	FPM	ngx	Lin	My	Lin	Mcr	Rea	931,997
servicestack-nginx-d	2,343	0.0%	Plt	C#	Mon	ngx	Lin	Non	Lin	Raw	Rea	2
hhvm	938	0.0%	Plt	PHP	hhv	HPH	Lin	My	Lin	Raw	Rea	2,231
servicestack	—	Did not complete	Plt	C#	Mon	XSP	Lin	Non	Lin	Raw	Rea	—
silex	—	Did not complete	Mcr	PHP	FPM	ngx	Lin	My	Lin	Raw	Rea	—
spray	—	Removed at request of framework maintainer - outdated version	Mcr	Sca	Spr	Non	Lin	Non	Lin	Raw	Rea	—
vertx	—	Removed at request of framework maintainer - outdated version	Plt	Jav	vtx	Non	Lin	Mo	Lin	Raw	Rea	—

其它的Benchmark

因为我最近关注Scala/spray和netty的性能，所以还搜集了一些其他人所做的Benchmark数据。
jakubkulhan提供了Node.js, Spray, Erlang, Http-kit, Warp, Tornado,和 Puma的benchmark： hit-server-bench。
每秒请求数 rps

Environment	Req/s (c=1)	Req/s (c=10)	Req/s (c=50)	Req/s (c=100)
Node.js	10215.47	38447.10	51362.60	52722.19
Spray	8139.76	30282.27	69328.26	69629.64
Erlang	24.70	246.98	1240.38	2474.18
Http-kit	8789.63	61355.26	73457.67	73475.39
Warp	12640.29	27749.54	26365.52	23916.69
Tornado	2356.82	4590.33	4422.08	4155.61
Puma	10048.54	19289.60	16280.00	16698.86
Netty	28409.44	116605.01	186109.06	180001.91
Spark (Jetty)	10982.23	44533.38	31624.97	57425.96
Spring Boot (Tomcat)	2652.85	13450.45	26336.17	31288.19
Reactor	12899.17	50113.60	66310.16	65718.33
PHP-FPM	2907.34	9968.78	9004.46	9425.08
HHVM	1323.99	6434.97	9192.72	9743.09
ReactPHP (PHP)	1636.49	9454.17	13676.61	12061.91
ReactPHP (HHVM)	2087.04	12185.55	16151.42	12036.11

平均延迟时间

Environment	Latency (c=1)	Latency (c=10)	Latency (c=50)	Latency (c=100)
Node.js	83.59us	253.57us	1.03ms	2.19ms
Spray	109.95us	1.41ms	1.40ms	1.86ms
Erlang	40.60ms	40.58ms	40.35ms	40.43ms
Http-kit	98.01us	770.13us	683.12us	1.36ms
Warp	64.65us	345.92us	1.89ms	4.20ms
Tornado	414.73us	2.28ms	12.21ms	24.95ms
Puma	96.37us	510.70us	2.83ms	5.88ms
Netty	53.17us	113.52us	635.84us	1.18ms
Spark (Jetty)	2.04ms	617.83us	6.23ms	7.94ms
Spring Boot (Tomcat)	7.09ms	815.26us	2.25ms	3.81ms
Reactor	2.19ms	3.93ms	6.72ms	8.49ms
PHP-FPM	355.04us	1.26ms	6.37ms	11.60ms
HHVM	824.73us	1.53ms	5.62ms	10.47ms
ReactPHP (PHP)	626.27us	1.05ms	3.81ms	10.83ms
ReactPHP (HHVM)	481.31us	811.58us	3.21ms	15.36ms