GraphDB market share

Last week a market analysis agency contacted me to ask some questions about OrientDB saying that OrientDB, following its research, has the second position in the worldwide GraphDB market right after Neo4J. Awesome!

But who are the main players of the GraphDB market?

Since each vendor claims, more or less, to be the market leader, what is the real user base? Seems quite hard to gather real data about users and customers directly from vendors.

So I though that one of the best way is to look into the public groups and forums because users, before or after, will subscribe on it because it's the first hand source of information, help and tricks. They can't lie! This document contains some metrics extracted from public sources. Click on the source to see with your eyes about the source I used.

By reading this data Neo4J is, without any doubts, the GraphDB market leader, followed by OrientDB in rapid grow and after a long distance InfiniteGraph and  DEX. By reading the web site  InfiniteGraph seems to have some real customers, but seems all related to the previous product ObjectivityDB (an ODBMS born more than a decade ago).

Below the metrics:

Products	Neo4J	OrientDB	InfiniteGraph	DEX
Updated on	05/05/2012	05/05/2012	05/05/2012	05/05/2012
Source	Google Group	Google Group	Google Group	Support Forum
created on	April 2011	April 2010	September 2011	May 2011
members	926	620	75	?
threads since the beginning	1,240	1,449	36	33
posts since the beginning	6,752	7,918	233	87
posts in the last month (April 2012)	1,107	439	19	0
posts 2 months ago (March 2012)	1,310	519	13	7 but 100% announcements

Working on large datasets in OrientDB

I was really busy in the last 5 days because I'was working in a huge rewriting of the RB+Tree algorithm that is behind the OrientDB index system. All is started some days ago when in the OrientDB Group the user Mark Harwood was sharing the first experiences in the import of the entire Internet Movie Data Base (IMDB) inside an OrientDB instance. We're talking about 13 millions of Vertices and about 8 millions of Edges.

Well, performance seemed good, specially in comparison with Neo4J, but the insertion time was not-linear and after some millions of records the engine slow down a lot. Furthermore a nasty bug in the management of multiple segments broke the import job after some minutes in his machine while on my PC all went correctly.

This is the reason why I stopped the development of the Cluster feature in favor of a strong review of the RB+Tree algorithm. Indexing has been alway fast, but the Mark Harwood's case stressed the engine as no one did until now. The reason is in the import process itself that executes random reads into the index while the RB+Tree seemed optimized for sequential insertion and retrieval.

I'm fixing all the pending issues but first benchmarks show a speed improvement in Indexing between 100% and 800%! Furthermore times now are really linear. Inserting the first millions of Vertices is the same of the last million. This is a big result to the already super-fast OrientDB engine.

Once finished the test phase (with new Unit tests, of course) I'll commit all the changes in SVN to be reviewed by the community before the next release.

Started the development of OrientDB clustering

Today is a new day, the day after the official release of OrientDB version 0.9.22. I apologize to all the requesters of the issues planned for this release and not yet finished and postponed to the 0.9.24. What is the reason?

---> Replication, Clustering, Fault-Tolerance <---

Seems that the most missed feature in OrientDB is the support for clustering, and therefore high scalability, high availability and high volume of transactions that a single node can't handle. In the last months I studied the whole different architectures of other NoSQL solutions for clustering.

Today is a new day because I'll start the development of clustering for OrientDB with the following features:

• Master-Slaves type, where it can be only one Master and N Slaves. If the Master crashes a Slave is elected to be the new Master
• IP multicast to discover cluster nodes
• Configuration of nodes using TCP/IP, useful for Clouds that don't allow the IP multicast
• Two sync modes: full where all the database is compressed and sent over the network, and partial by sending only the changes happened since the last sync
• New database handled by the Master OrientDB Server instance to store all the pending records until a configurable threshold. Up this threshold the logs are deleted and the node need a full-sync on startup
• New console commands to display nodes, listen clustering messages and elect the master manually

The release 0.9.23 is planned for the October 15th, 2010. Stay tuned and contribute with comments, ideas or critiques.

Released OrientDB 0.9.22

Hi all,

many thanks to all the contributors for this new piece of software. A new version of OrientDB has been released after more than 1 month of work (and holidays).

Starting from this version you can use OrientDB as a pure GraphDB using the custom API (http://code.google.com/p/orient/wiki/GraphDatabase), by the Tinkerpop stack (Blueprints, Gremlin, Pipes, etc.) or again by using the Erlang language with the Cali project (http://github.com/dmitriid/cali).

Direct URL to download: http://orient.googlecode.com/files/orient-database-0.9.22.zip

New features:

- Issue 21: Full text index against schema fields

- Issue 91: SQL UPDATE against collections and maps

- Issue 94: Support for GREMLIN graph language

- Issue 108: Regular expression support in WHERE clause

- Issue 109: Support for memory clusters inside regular persistent database

Bugs:

- Issue 86: Cannot insert record from console

- Issue 87: Cannot query records from OrientDB Studio Command-page

- Issue 100: Transaction does not work as expected

- Issue 101: Error in using orient queries having ' character

- Issue 104: Bulk record creation in cluster with pre-created indices causes ClassCastException

- Issue 105: Error in using UPDATE query

- Fixed other minor bugs

The next release is planned for the end of September 2010 with the first version of Clustering support. Here the details: http://code.google.com/p/orient/issues/list?q=label:v0.9.23

To test the OrientDB Studio go here: http://www.moobilis.com:2480. Users: reader/reader and writer/writer. To manage the server: root/root.

If you'd like to suggest some new feature for the next releases please open a New Issue as "New feature": http://code.google.com/p/orient/issues/entry?template=New%20feature

Published the OrientDB presentation for the Open Source Italian Conference in Cagliari, Italy

Check out this SlideShare Presentation:

Orient db introduction

View more presentations from lvca.

Orient DB almost ready for the show time!

After some weekends of work I've in my hands the first version of Orient DB! I wrote a lot of Unit Tests that helped me a lot on the heavy refactoring in order to achieve the best performance than ever. The main features of Orient are:

• Full ACID: Optimistic transaction available with recovery of aborted transactions when the engine restarts. Tested with 1 billion of records even with simulated crashes during the commit phase. Pessimistic (with object-level locking) transaction in the next releases.
  
• Extremely light: less than 400kb of jar without any dependency with other libraries
• Super fast: stores 1,000,000 (yes one million) of records in 3.5 seconds on common hardware (my notebook). The KeyValue engine stores 100,000 entries indexed in 9 seconds. Lookups are so fast as memory access.
• tested up to 1 billion of records, but the limits are up to 9.223.372.036 Billions. Actually I think I'll never will be able to include this test in my Unit tests :-)
• A lot of cool features such as Asynchronous Queries, SQL language, Partitioning of data, native sharding support, and more
  
• Requires: Java5 or major and run on all the most used Operative Systems.
  

What about the license? I have no choose yet, maybe LGPL?

http://www.orientechnologies.com.