Terracotta as distributed DBMS? Bad idea!

Some months ago I have had a fucking genius idea about a new application. In order to start working to my idea I need to distributed at large tons of objects in several nodes around the Internet. After a deep research about all the best solutions available now I decided to try Terracotta.

Basically I need a real distributed Object repository so why don't create a simple Object DBMS virtually all in RAM and let to Terracotta most of the hard work?

1. Handle the cache on nodes
2. Manage the object/page fault in a transparent way for the application
   
3. Share the load among nodes
4. And obviously: read/write objects in persistent way

So I've developed a very simple library that uses the JDK 1.6+ TreeMap to collect data and to query them. I was surprised to discover how much terracotta makes a good job in hiding the dirty work and complexity of data distribution.

But before to think to use this piece of code in a production system I need to be sure that the objects became really persistent in ACID way. For this reason I developed some Test Cases as microbenchmarks to see the real throughtput and if data are written in synchronous way to the disk. This was my test using the library I wrote:

UserDatabase db = ServerNode.getInstance().connect("petshop", null, null);
clazz = db.getClazz("Animal");

for( int i = 0; i < 100000; ++i ){
UserObjectTransient object = clazz.createObject();

object.setValue("name", "Gaudi");
object.setValue("description", "My crazy ferret!");
object.setValue("from", "Barcelona, Spain");
object.create();
}

Pretty nice, don't you? The Terracotta server instance was on a remote server with 1GB of heap and enought bandwitch available and the client runned on my laptop. Yes I know, this is a microbenchmark and can't tell to me the performance in all the scenarios. But to go on on my experiments I needed to know if I was totally crazy or if there was a way to build a robust & scalable solution to satisfy my needs.

Well, in this test my library + Terracotta (after some tuning by reading the documentation and the forum) are able to write around 190 objects per second. Not really bad for the first release and in comparison to a RDBMS solution. But I've stored just a dynabean with two properties... I remember in the past (about 8 years ago!) when I tried a real ODBMS (Orient ODBMS) it was able to store 100x of this solution using a 8-years-old hardware!

But the really bad news is another one: When the "for cycle 100.000" was finished and the application seemed to be succesfully ended I noted that the console had no control. The test was still running! After 10 seconds I killed the java process and counted the objects created (using the Terracotta console): 99.345!

Where are the 655 objects missed? Ok, probably I got wrong about ACID features. No, the Terracotta documentation tells ACID! But how it can be ACID if the client sends the objects to the server in asynchronous way?

I'm pretty confident that it would be exist in any point of configuration the real synchronous mode but even though it exists what about the performance? If this ASYNCHRONOUS solution was able for 190 obejcts/sec as throughtput the synchronous one must be much more slow!

These are the reasons why I abandoned this path. Terracotta surely fits well in traditional replication contexts and the product is really good on documentation, presentation, APIs, etc. But for my needs I have to continue in searching...

Remove the persistence layer at all to scale up?

In these days I'm working for a brand new idea that need to scale out (close to the infinite?) and after a lot of thoughts about its architecture I'm seriously evaluating to remove the persistence layer using a distributed in-memory cache, obviously with the "fail-over" feature: Terracotta.

Terracorra, under the hood, records all changes happened to the objects using BerkeleyDB. All is transparent to the application and on crash the server swaps to another one configured without interruption and leaving all objects consistent. Or, at least, it seems to be so.

Does anyone have never used Terracotta in a real-world application without a DBMS to store data?

Roma Framework and the Virtual Objects

Yesterday night I've committed in the Roma's SVN (revision: 4479) source repository the first version of Virtual Objects. Virtual Objects are like POJO, but built dinamically just using a descriptor, in this case a XML file. The XSD is the same of Roma XML annotations but now allows to specify the field type and in future declaring inheritance and other stuff.

Action implementation can be written using any scripting language is supported by JSR 223, Javascript in primis.

Roma treats Virtual Objects in the same way of POJO, so you can define an Employee at the fly in this way:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<class xmlns="http://www.romaframework.org/xml/roma" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance"

    xsd:schemaLocation="http://www.romaframework.org/xml/roma http://www.romaframework.org/schema/v2/roma.xsd">

    <aspects>

        <view>

            <form>

                <area name="main" type="column">

                    <area name="fields" size="2" type="grid" />

                    <area name="actions" type="row" />

                </area>

            </form>

        </view>

    </aspects>

    <fields>

        <field name="id" type="Integer" />

        <field name="name" type="String" />

        <field name="surname" type="String" />

        <field name="born" type="Date" />

        <field name="age" type="Integer">

            <aspects>

                <view enabled="false" />

            </aspects>

        </field>

        <field name="salary" type="Float" />

    </fields>

    <actions>

        <action name="refresh">

            <aspects>

                <scripting>

                    org.romaframework.core.Roma.objectChanged(me);

                </scripting>

            </aspects>

        </action>

        <action name="print">

            <aspects>

                <scripting>

                    print(age);

                </scripting>

            </aspects>

        </action>

        <action name="login">

            <aspects>

                <scripting>

                    print('Login...');

                    org.romaframework.frontend.RomaFrontend.flow().forward("ProjectLogin");

                </scripting>

            </aspects>

        </action>

    </actions>

    <events>

        <event name="show">

            <aspects>

                <scripting>

                <![CDATA[

                    age=32;

                    Roma.fieldChanged(me, ["age"]);

                ]]>

                </scripting>

            </aspects>

        </event>

    </events>

</class>

And this is the result:
Note that changes to the XML files are loaded in real-time enabling a real productive development (and production why not) environment.

New Google Wave account

Yesterday I've received my Google Wave account as developer.

I've played for some minutes and it's really awesome even though I don't know no one that own it as well to play together with all the features!

Once returned from holidays I'll try to go deep to the APIs in order to know how to integrate it inside Roma Meta Framework and Romulus.

At leat one good reason to use Linux

With this picture someone convinced me to pass to the Linux OS ;-)

Taken from http://slashweb.org.

Scripting RAD tools such as Ruby on Rails and Grails can be really slow?

Tiago Fernandez in his blog reported some results about benchmarks using Java, JRuby, Groovy and Scala. I know: we can talk about benchmarks for days but in the Internet there are a lot of them and the performance problems of the scripting language is not a news.

Results show that Groovy is very slow in comparison to the pure Java and the Scala scripting language. This could be a serious reason to be worried when your application created with Grails or RoR goes in production...

So, why don't you use a real Java RAD tool like Roma Meta Framework? :-)

You can develop shortly without using a scripting language with the benefits of great refactoring support (Eclipse, Netbeans and IntelliJ are master on it) and performance and scalability!

The source: http://tiagofernandez.blogspot.com/2009/05/java-integration-with-groovy-jruby-and.html

Romulus at the Internet of Services 2009 conference

On June, 10th 2009 I will be in Brussels to present the Romulus demo at the "Internet of Services 2009" conference. Romulus project is scheduled in the afternoon. Below the agenda.

Group 2: June 10, 13:30 - 17:30

• Semantically enhanced SLA Negotiation (BREIN)
• Automated Service Level Agreement Management Framework - Adhoc Demonstrator (SLA@SOI)
• TEAM: Knowledge management in distributed software communities by applying semantic technologies (TEAM)
• Decentralized transactional collaborative drawing (SELFMAN)
• Romulus: your answer to improving productivity in Java Web Applications (ROMULUS)
• Edutain@Grid: A Service-Oriented Infrastructure for Future Real-Time Interactive Internet Applications (edutain@grid)

Luca in the first page of Google searching for "Java JSR"

Today Mara told me that searching "Java JSR" in Google (at least in the Italian version), in the first page compares my face and this blog... I've tried and is real! What nice thing :-)

Transparent DataNucleus enhancement at run-time

Today it has worked! In the last days I was working to avoid the enhancement phase in the building of Roma Framework projects. Yes. Until yesterday you needed to invoke the "persistence-compile" task in your Roma project and all the projects that contained persistent classes.

Furthermore pre-packaged modules such as ADMIN, USERS, MESSAGING and SCHEDULER-QUARTZ needed to self-execute the enhancement in order to put persistence capable classes inside the JAR. This was a mess if you needed to change the ORM binding since anything resided inside the JAR.

I have to check yet if run-time changes to the entity POJOs are auto-enhanced at the fly... If works the productivity would be speedup over and over again.

Thanks to Andy Jefferson of DataNucleus team in supporting my crazy requestes :-) The work tomorrow will be in changing all the wizards to support the new configuration. In few words the change consist in:

1. Let to the PersistenceAspect to invoke the Enhancer at startup
2. Create the file persistence.xml inside the src/META-INF folder.
   

Once everything works I'll commit in to the SVN trunk for the upcoming release 2.0.

Romulus technical meeting in Madrid

Today has finished the 3-days technical meeting of Romulus. In the first day we've worked in the UPM (Universitad Politecnica de Madrid) class room.

For the second day Carlos Iglesias has organized the entire working day in Segovia, a beautiful city at about one hour from Madrid. We've worked inside a agritourism and eaten a lot of home made food. In the afternoon we have had the social roundtrip in the Segovia city. If you're in Madrid you can't not to visit Segovia! There are a lot of monuments in a so small city. The catedral, the Fortress and the Roman waterworks, all in perfect state. It seems they was built in the past year.

This morning we went back to the UPM class room in order to finish the last pending works and give the new action points. Asset Data is envolved in almost all the activities and I know that when I'm back to Italy a huge work expects me and my company for the next months.

Am I Tired? Not at all. Instead, I'm very excited to go back to work to the new Roma Meta Framework version 2.0 and contributing to the growth of Romulus consortium & technology.

Tomorrow is the last day. Luigi will take the flight in the early morning. Giordano and me, instead, will go to the Toledo city for a fast trip. Just the time to see the centre of the city, buy a tipical medieval sword for my house, having lunch with "Pulpo a la Gallega" and go back to Madrid for the our flight.

Bye bye Spain!