Thursday, November 01, 2012

OrientDB: huge improvement in performance (+9,000%) in many use cases. Thanks RaspberryPi !



Hi all,
today I've a good story to tell you. A couple of days ago Fabrizio Fortino sent to me an email with some metrics and screenshots about the profiling of an in-production instance of OrientDB. Well, a lot of time was spent on open/close of database. 

That was the issue 1145 (http://code.google.com/p/orient/issues/detail?id=1145) but I assigned to it a low priority because it was an improvement, not a real bug...

Well today I'm hacking with a Raspberry PI cheap HW and OrientDB to see if it could be used in production for some limited use cases. Well on this kind of HW everything is much-much slower! "Yeah, it's normal: I have a $35 HW, Java is not so optimized yet on this ARM platform, etc.". This were my firsts thoughts about the initial results.

But after some profiling I was arrived at the same conclusion of Fabrizio, so I decided to spend 2 hours of my life to investigate in deep.

Well, I've just committed a small patch (r7134) that avoids to open a database every time a database is re-used from the pool. In facts this is a quite costly operation, specially if you do many small atomic operation where most of the cost is in open/close that in the operation itself!

This fix improved a lot these scenarios:
  • Usage via HTTP/Rest, because a new connection is acquired every time from the pool at every operation
  • Java Web Applications where at the server side you used the database pool
  • you wrote a Java App that every time creates a new instance of a database. if this is your case I strongly encourage using the database pool that at this time is much faster
  1. in case metadata changes (schema, security, functions) you would need to invoke a reload() to get the changes
The improvement will be minor in the cases:
All these are PROS, what about CONS?
This is a simple load of a tiny document against a database on my pc:

$ ab -n1000 -A admin:admin -k -c10 http://localhost:2480/document/demo/71:1
...
Requests per second:    52.56 [#/sec] (mean)

$ ab -n1000 -A admin:admin -k -c10 http://localhost:2480/document/demo/71:1
...
Requests per second:    4694.57 [#/sec] (mean)

This is 90x faster, namely 9,000%, namely a huge improvement!

Now it's funny that OrientDB on the Raspberry PI, with the new patch, runs at a speed quite close to my PC I used everyday to work before this patch!