VMware vFabric Blog: vFabric SQLFire: A roadmap to cloud-enable your data architecture

VMware vFabric Blog: vFabric SQLFire: A roadmap to cloud-enable your data architecture:
At SpringOne-2012, the “Effective design patterns with NewSQL” session generated a lot of interest. Jags Ramnarayan and I discussed scalability and predictability problems with traditional RDBMS systems, NewSQL driving principles, SQLFire architecture, application design patterns as well as data consistency and reliability.
We went deep into vFabric SQLFire which is a pragmatic solution that addresses these data challenges:

• How do I architect my data tier for very high concurrent workloads?
• How do I achieve predictability both for data access response time and availability?
• How do I distribute data efficiently and real time to multiple data centers (and to external clouds)?
• How do I process these large quantities of data in an efficient manner to allow for better real-time decision-making? 

At VMware, we understand that adopting a new technology can be a daunting task. With SQLFire one can take an evolutionary approach.  You can start with a distributed cache, move (or addd) to a full-fledged OLTP data store, and to a global database… at your own pace.

Embedded, clustered Java database

To get started, you can embed SQLFire into Java applications by including the required SQLFire libraries. When the application initiates a connection to SQLFire, it starts a peer server that joins other peers in the same cluster. Unlike other embedded databases such as H2 or Derby, SQLFire allows several servers to store replicated and partitioned tables, persist data to disk, communicate directly with other servers and participate in distributed queries.

Write-thru distributed caching

When you need to boost application performance and reduce network traffic, you can use SQLFire as a distributed cache to offload your existing DBs. Applications can use a familiar SQL syntax to read from and write to the distributed cache. With the write-through pattern, all SQLFire changes can be synchronously written to the external database before the cache is changed. Because SQLFire can participate in the container transaction, if the write-through succeeds then the data becomes available in the cache.

Distributed caching with asynchronous writes to DB

If synchronous writes to the backend are too costly, your application can asynchronously write to the external database using a cache listener.  For HA purposes, you can designate a primary listener with one or more secondary instances to ensure that the solution continues to run even under failure conditions. Whenever the external database is not available, your application will continue to run as SQLFire queues up events until the external database comes back up.  Once that happens, the listener will forward data changes to keep the external databases up to date.

Scalable OLTP data store

Because of SQLFire’s performance, availability and reliability guarantees, you no longer need databases behind SQLFire.  So, when you need thousands of transaction per second, sub-millisecond response time and linear scalability, you need to use SQLFire as your primary data store. This pattern greatly simplifies your data architecture and reduces maintenance costs as well as operational overhead.

Distributed compute grid with data-aware procedures (real time in-database map reduce)

Using SQLFire stored procedures, you can implement business logic at the server level that runs on the same process space as your data and in parallel on multiple data nodes, thus significantly improving application performance and scalability.
Because the SQLFire clusters consists of multiple servers, the stored procedure execution in SQLFire is parallelized to run on multiple servers, concurrently. SQLFire makes the whole map-reduce process in real-time and transparent to your application.
The ability to write functions and procedures in Java brings the complete set of Java APIs and the power of Spring into your SQL environment as server side logic.

Global and cloud-based database

SQLFire supports global WAN connectivity and gives you the option of replicating data across data centers and cloud providers. This multi-site topology makes geographically-distributed clusters appear as one global system. If one of the SQLFire clusters fails or is taken offline for maintenance, clients in that geographical region can failover to the next available cluster and continue to operate.
This makes SQLFire the ideal solution for disaster recovery and business continuity purposes.

One thing is clear. If data is locked up in heavyweight and legacy RDBMSs, organizations can’t realize the benefits promised by cloud computing.
Now you have a very good idea of how to incorporate SQLFire into your architecture at your own pace to continue your journey towards cloud computing. Go ahead and test-drive SQLFire. It will be worth your time.
Thank you for reading!