Performance, Benchmark of the Day #
hacking, June 2nd 2009
In reference to The Price of Abstraction and the associated Reddit comments:
On my laptop C++ versions 1 and 2 run in 2.139s and 2.410s respectively (user+sys), and the direct Common Lisp translations using double floats and native complexes run in 3.692s and 4.594s on SBCL. See: mandelbench.lisp
I haven't investigated the source of the performance difference, but given that the SBCL generates — at a quick glance — reasonable looking code for both, I believe the difference comes from something lacking in loop optimization, and/or possibly suboptimal floating point instruction sequences. Given that Paul Khuong is working on using SSE2 for complex arithmetic, it will be interesting to see what kind of effect that has on performance.
I did not bother implementing complexes as user-defined objects — depending on how you view this benchmark it either invalidates my numbers, or focuses on the only real point there is...
Conclusions?
While not hitting the C++ target today, I think these numbers support SBCL's call to fame as a high-performance Common Lisp compiler.
Also, an important reminder for any CL advocates: when comparing the speed of a Common Lisp implementation like SBCL to C or C++, never say Common Lisp is faster — not even in those cases when it is. Everything most performance oriented people know conflicts with that, so that even if you are right, they will not believe you. Instead, say something like: "Even for low level number crunching SBCL can generate code that approaches gcc or g++ in speed.", or whatever the point you are trying to make is. Say almost as fast, not faster — and please, don't lie. Nothing is worse PR then unfounded claims.
As a bonus, some interesting — and related — papers from Didier Verna "On Behaviour and Performance of Common Lisp": Beating C in Scientific Computing Applications, and CLOS Efficiency: Instantiation.