OrientDB the fastest GraphDB available today?

Two days ago I've finished the integration of the last part of the Blueprints: the Index. Now OrientDB can be used as for Neo4J with the entire ThinkerPop stack. This include the Gremlin language as well.

First tests shows that OrientDB outperforms Neo4J, the market leader of GraphDB, in all the tests but the iteration (and therefore counting). This is due to the implementation of the Blueprints that need to create a new wrapper object to contain the OrientDB's OGraphVertex and OGraphEdge objects. I've some ideas to improve it, but I need more time for it. Maybe in the next weeks or earlier if some users need it. However using native OrientDB Graph APIs this overhead is removed.

These are the results:

Test name	Description	Time in ms (less is better)		OrientDB vs Neo4J
		OrientDB 0.9.22	Neo4J	+ faster, - slower
testVertexEquality	1 vertex added and retrieved in	4.74	2.33	-203,4%
testRemoveVertexNullId	1000 vertices added in	216.73	2,070.74	+955,4%
	1000 vertices deleted in	1,093.71	1,910.02	+174,6%
testVertexIterator	5000 vertices added in	476.62	8,314.04	+1.744,4%
	5000 vertices counted in	86.94	1.60	-5.433,8% *
testAddManyVertexProperties	750 vertex properties added (with vertices being added too) in	72.34	43,437.29	+60.046,0%
testAddEdges	6 elements added and checked in	0.79	30.70	+3.886,1%
testAddManyEdges	3000 elements added in	2,314.44	8,031.12	+347,0%
	1000 edges counted in	8.45	12.54	+148,4%
	2000 vertices counted in	31.62	0.51	-6.200,0% *
	2000 vertices checked in	98.27	14.05	-699,4% *
testGetEdges	3 edges retrieved in	0.45	0.12	-375,0%
testRemoveManyEdges	200 vertices counted in	167.28	0.14	-119.485,7% *
	100 edges counted in	34.92	0.79	-4.420,3% *
	100 edges removed and graph checked in	20,555.44	332.54	-6.181,3% *
testStringRepresentation	1 graph string representation generated in	0.01	0.01	100,0%
testClear	75 elements added in	45.89	152.87	+333,1%
	75 elements deleted in	30.76	422.05	+1.372,1%
testRemovingEdges	500 vertices added in	133.62	974.37	+729,2%
	1000 edges added in	1,130.64	4,521.90	+399,9%
	1000 edges deleted (with size check on each delete) in	36,773.66	4,411.69	-833,6% *
testRemovingVertices	500 vertices added in	10.48	1,110.03	+10.591,9%
	250 edges added in	132.32	1,089.45	+823,3%
	500 vertices deleted (with size check on each delete) in	70,675.74	2,140.07	-3.302,5% *
testTreeConnectivity	1464 vertices added in a tree structure in	1,506.82	5,832.02	+387,0%
	1464 vertices iterated in	427.67	0.75	-57.022,7% *
	1463 edges iterated in	9.89	5.39	-183,5% *
testTinkerGraphEdges	graph-example-1 loaded in	43.75	541.15	+1.236,9%
testTinkerGraphVertices	graph-example-1 loaded in	6.81	520.29	+7.640,1%
testTinkerGraphSoftwareVertices	graph-example-1 loaded in	5.31	543.02	+10.226,4%
testTinkerGraphVertexAndEdges	graph-example-1 loaded in	5.01	544.17	+10.861,7%

* are the tests with iteration

Yesterday Mike Tallent, via Twitter, made his congratulations to me for my recent "Db4o community award".

Is it a joke? The reason seems to reside in the homepage of TevereFlow that claims to support db4o. After some private messages with a db4o guy I explained that TevereFlow supports db4o since it uses DataNucleus as OR-Mapping and it supports db4o. Therefore TevereFlow can run on top of db4o but the support is indirect. All clear.

But after some days I found my name in the "DB4O MOST VALUES PROFESSIONAL" list:

Thanks a lot, but I never have written a single line of code for/with db4o!

Furthermore I'm the author of OrientDB, a competitor of db4o.

So is it a joke? :-)

Published the Roma presentation for the Open Source Italian Conference in Cagliari, Italy

Check out this SlideShare Presentation:

Roma introduction and concepts

View more presentations from Luca Garulli.

Published the OrientDB presentation for the Open Source Italian Conference in Cagliari, Italy

Check out this SlideShare Presentation:

Orient db introduction

View more presentations from lvca.

Romulus workshop in Madrid

Madrid, February 18th 2010. The day seems to be finished and it has been a pretty long day for me. This morning wake up at 7:30 o'clock, 8:30 breakfast at the Starbucks in front at the hotel and then straightforward to the Universidad Politécnica de Madrid (UPM) for the Romulus meeting and for the Romulus Workshop in the afternoon.

Today I've presented Roma Meta Framework to a bunch of students and some business IT men interested on the fast developing of Web Applications following the DDD approach, the Meta Framework philosophy and obviously Roma. We have had a hands-on session where almost all the attendees tried to follow me step by step on the building of a simple social application. I hope today the Roma community welcomes some new committers.

Below some photos shooted by Marco.



Anyone is in Madrid until Saturday morning?

Orient DB almost ready for the show time!

After some weekends of work I've in my hands the first version of Orient DB! I wrote a lot of Unit Tests that helped me a lot on the heavy refactoring in order to achieve the best performance than ever. The main features of Orient are:

• Full ACID: Optimistic transaction available with recovery of aborted transactions when the engine restarts. Tested with 1 billion of records even with simulated crashes during the commit phase. Pessimistic (with object-level locking) transaction in the next releases.
  
• Extremely light: less than 400kb of jar without any dependency with other libraries
• Super fast: stores 1,000,000 (yes one million) of records in 3.5 seconds on common hardware (my notebook). The KeyValue engine stores 100,000 entries indexed in 9 seconds. Lookups are so fast as memory access.
• tested up to 1 billion of records, but the limits are up to 9.223.372.036 Billions. Actually I think I'll never will be able to include this test in my Unit tests :-)
• A lot of cool features such as Asynchronous Queries, SQL language, Partitioning of data, native sharding support, and more
  
• Requires: Java5 or major and run on all the most used Operative Systems.
  

What about the license? I have no choose yet, maybe LGPL?

http://www.orientechnologies.com.

Released TevereFlow, the first workflow engine with a Web Editor

Starting from December 22nd 2009 is available the new version of TevereFlow. Tevere Flow is a light-weight Workflow engine built using the Java® technology. Just download it and create your process using the Web interface. You can use Tevere as embedded or as external engine by using the provided Web Services API.

Tevere was built using Roma Framework under the Romulus consortium an it's currently used in several production systems.

Main features:

• Open Source Apache 2.0 license
• Web User Interface using the Ajax technology
• Stand-alone application: just download and start it
• Fully Transactional supporting any RDBMS or db4o ODBMS
• Set of built-in commands available (email, web service invocation, etc.)
• Auto-resume of failure activities
• User and profile management
• Activities can be written in Java or using any supported scripting language such as Javascript and Ruby
• Integration via Java APIs or WebService

Search on Freshmeat

After I've posted the last release 2.1.0 of Roma Meta Framework on Source Forge I tried to search the project by typing "Roma" in the search field of Freshmeat home page:

0 search results for: roma

What???? If I type "Roma Framework": the result is always 0 (zero) results but with the result table filled:

Maybe they require an exact search? Not properly, since if I try to search for "framework":

1941 search results for: framework

What kind of search is it?

Terracotta as distributed DBMS? Bad idea!

Some months ago I have had a fucking genius idea about a new application. In order to start working to my idea I need to distributed at large tons of objects in several nodes around the Internet. After a deep research about all the best solutions available now I decided to try Terracotta.

Basically I need a real distributed Object repository so why don't create a simple Object DBMS virtually all in RAM and let to Terracotta most of the hard work?

1. Handle the cache on nodes
2. Manage the object/page fault in a transparent way for the application
   
3. Share the load among nodes
4. And obviously: read/write objects in persistent way

So I've developed a very simple library that uses the JDK 1.6+ TreeMap to collect data and to query them. I was surprised to discover how much terracotta makes a good job in hiding the dirty work and complexity of data distribution.

But before to think to use this piece of code in a production system I need to be sure that the objects became really persistent in ACID way. For this reason I developed some Test Cases as microbenchmarks to see the real throughtput and if data are written in synchronous way to the disk. This was my test using the library I wrote:

UserDatabase db = ServerNode.getInstance().connect("petshop", null, null);
clazz = db.getClazz("Animal");

for( int i = 0; i < 100000; ++i ){
UserObjectTransient object = clazz.createObject();

object.setValue("name", "Gaudi");
object.setValue("description", "My crazy ferret!");
object.setValue("from", "Barcelona, Spain");
object.create();
}

Pretty nice, don't you? The Terracotta server instance was on a remote server with 1GB of heap and enought bandwitch available and the client runned on my laptop. Yes I know, this is a microbenchmark and can't tell to me the performance in all the scenarios. But to go on on my experiments I needed to know if I was totally crazy or if there was a way to build a robust & scalable solution to satisfy my needs.

Well, in this test my library + Terracotta (after some tuning by reading the documentation and the forum) are able to write around 190 objects per second. Not really bad for the first release and in comparison to a RDBMS solution. But I've stored just a dynabean with two properties... I remember in the past (about 8 years ago!) when I tried a real ODBMS (Orient ODBMS) it was able to store 100x of this solution using a 8-years-old hardware!

But the really bad news is another one: When the "for cycle 100.000" was finished and the application seemed to be succesfully ended I noted that the console had no control. The test was still running! After 10 seconds I killed the java process and counted the objects created (using the Terracotta console): 99.345!

Where are the 655 objects missed? Ok, probably I got wrong about ACID features. No, the Terracotta documentation tells ACID! But how it can be ACID if the client sends the objects to the server in asynchronous way?

I'm pretty confident that it would be exist in any point of configuration the real synchronous mode but even though it exists what about the performance? If this ASYNCHRONOUS solution was able for 190 obejcts/sec as throughtput the synchronous one must be much more slow!

These are the reasons why I abandoned this path. Terracotta surely fits well in traditional replication contexts and the product is really good on documentation, presentation, APIs, etc. But for my needs I have to continue in searching...

Remove the persistence layer at all to scale up?

In these days I'm working for a brand new idea that need to scale out (close to the infinite?) and after a lot of thoughts about its architecture I'm seriously evaluating to remove the persistence layer using a distributed in-memory cache, obviously with the "fail-over" feature: Terracotta.

Terracorra, under the hood, records all changes happened to the objects using BerkeleyDB. All is transparent to the application and on crash the server swaps to another one configured without interruption and leaving all objects consistent. Or, at least, it seems to be so.

Does anyone have never used Terracotta in a real-world application without a DBMS to store data?