PIVOTAL GemFire Technology
What is GemFire?
GemFire is a distributed in-memory data grid for OLTP applications. It is backed by a robust, shared-nothing persistence architecture that guarantees the performance of in-memory and the reliability of disk. By combining data redundancy, GemFire provides the best-of-breed, high availability for data. It is a transformational piece of technology that was purpose-built to support large volumes of data with a very high degree of data consistency. Data is stored as key-value pairs or as JSON documents in a partitioned/replicated data grid. Every feature of the product is designed to minimize latency, allowing the development of highly responsive applications that never have to be redesigned for scaling.
Familiar Java HashMap API
Data sets in GemFire are internally held in a data structure called a region, which implements the java.util.HashMap interface. Developers familiar with Java will quickly understand GemFire.
GemFire makes optimal use of system resources like CPU, memory, network and disk to ensure that if these resources are doubled, the overall system throughput can also double, without any visible increase in latency for end-user applications. It does this by intelligently managing the placement of data while reducing network round trips. Data gets replicated only to those nodes that need the data, and requests for access are routed intelligently using the most direct path available.
Elastic at Cloud Scale
GemFire’s resource usage is designed to be elastic. If you add more resources, it increases capacity and balances the use of resources across the network. If you take away resources, it responds to the decrease in resources without impacting end-user applications, as long as the remaining elements of the cluster can handle the load. The ability to add and remove nodes in response to changing load conditions makes GemFire cloud ready for private and public cloud deployments.
Highly Available with No Maintenance Windows Required, Even for Upgrade
GemFire runs as a cluster of processes on commodity hardware running on regular TCP/IP networks. In this environment, it takes extraordinary measures to protect data in the face of process failures, machine failures, rack failures and even data center outages. GemFire’s cluster management and resource utilization algorithms use a combination of retries, disk persistence, redundancy zones and network partitioning management techniques to ensure data consistency in a highly available system. GemFire supports the notion of rolling upgrades, which ensures your GemFire administrators do not have to plan for scheduled maintenance windows to upgrade the product or any application code that runs with GemFire.
PIVOTAL GemFire Technology
What Makes GemFire Easy to Use for Developers?
The following capabilities make GemFire an easy-to-use solution for developers:
Deep and Rich Spring Data Integration
As a fully supported Spring project within Pivotal, Spring Data GemFire allows developers to architect applications so that data access and business logic are separated from configuration and operation code. In addition, GemFire APIs and related code samples help make developers productive quickly. Spring Data support in GemFire is enhanced and updated with each product release which allows Spring users to bootstrap applications using very familiar programming techniques to exercise the powerful set of capabilities exposed by GemFire.
Support for Memcached Clients
GemFire servers can be started up as memcached servers with full support for the memcached API. Gemcached extends the reach of GemFire to any memcached client allowing end users an easy way to get started with GemFire.
L2 Hibernate Support and Scalable Session State Management Support
Applications store increasing amounts of data in user sessions and more importantly, user session expiration may be in the order of days, if not months. The ability to store session-state information in a data grid, support multiple topologies for scale (thousands of app servers that can access any session from anywhere), transmit deltas when user sessions are updated, and support for popular application servers such as tcServer, JBoss, WebLogic and WebSphere are common use cases supported by GemFire.
PIVOTAL GemFire Technology
What Legacy Integration Capabilities Does GemFire Have?
GemFire includes the following legacy integration capabilities:
Support for Multiple Deployment Topologies
GemFire supports multiple deployment topologies to provide highly customized application deployments based on their latency, throughput, scale and redundancy requirements. GemFire applications can be deployed as a peer-to-peer cluster (supporting hundreds of nodes), in a client/server cluster (supporting tens of thousands of clients) or in multiple clusters daisy-chained using GemFire’s WAN replication capabilities.
Reliable Publish-Subscribe Framework Allowing Integration with the Rest of the Enterprise
GemFire provides an “eventing” framework that allows end-user applications to receive notifications when data-change events happen in the system. This includes the ability for GemFire clients to configure very customized subscriptions for data change notifications, which is supported by a highly reliable publish-subscribe notification mechanism. These subscriptions can be set up as regular expressions on keys, key sets, or continuous queries using an SQL-like querying language. Event callbacks can be programmed to integrate data coming from GemFire with the rest of the enterprise (including backend databases and exchanges). Subscription management and event distribution is highly optimized to avoid wasting CPU and network cycles and to ensure that events are delivered to the right set of consumers.
Support for Native Clients in Java, C++ and .NET
GemFire is one of the few data grid providers that supports a rich set of APIs and native implementations for those APIs in Java, C++ and .NET. The clients are 64-bit enabled and support full client-side caching with eventual consistency for the cached data which allows you to build applications that are powerful, respond in real time, and can manage large volumes of data without ever going to disk.
Backward Compatibility Support
GemFire clients have been deployed in kiosks—on vendor sites where the customer has no control over the upgrade schedule. As a result, GemFire allows older clients to continue operating with newer servers. Support for rolling upgrades in the cluster ensures that peers from two versions can support data access during an upgrade process.
Support for Reliable Asynchronous Event Queues
GemFire supports callbacks for read-through and write-through operations allowing applications to integrate with legacy data sources. However, when GemFire is being used as the operational data store, it makes sense to use the Async Event Queueing mechanism to update traditional databases using batch operations. As with other aspects of GemFire, high availability is built into the Async Event Queues ensuring data consistency between GemFire and the traditional database being updated.
PIVOTAL GemFire Technology
Which Enterprise-Readiness Capabilities Does GemFire Support?
Support for Global WAN Replication
GemFire provides a globally consistent view of data using highly fault-tolerant asynchronous WAN replication capabilities that are designed to work on high-latency congested networks on the open internet. Using techniques like batch replication, tuned sliding windows, support for conflict detection and failover for both sender and receiver, GemFire WAN replication helps you synchronize data across the globe, ensuring that your users can be served even if a data center goes down for any reason.
Support for Backward- and Forward-Compatible Serialization Protocol
Applications evolve their data models at different rates. When a new application introduces additional attributes to an existing data entity, GemFire’s backward- and forward-compatible serialization protocol ensures that older and newer applications can continue to exchange information in native binary format without having knowledge of the newer attributes to the entity while preserving them in the system.
Scalable Management and Monitoring Framework
GemFire collects statistics within each GemFire server through a highly optimized sampling algorithm, which provides insight into what is happening within each server. GemFire aggregates these statistics within a managing member and makes them available to management and monitoring tools via JMX MBeans. The public statistics API allows your users to plug in their own statistics within application callbacks to see what is happening across the system.
Scalable Behavior Execution Support
GemFire allows your users to write functions in Java that get executed in proximity to the data that the function will use. This is akin to writing distributed stored procedures in Java that get executed in parallel. The function-execution framework in GemFire is generic enough and supports multiple execution models including MapReduce-like executions across the grid.
Support for Transactions, Querying and Indexing
GemFire stores data in tables that are referred to as regions, and there can be any number of regions in the system that store data either as partitioned or replicated. Regions can be co-located, allowing for optimized scalable transactions across co-located data in the cluster. GemFire supports Object Querying Language (OQL), which offers support for indexing and SQL-like querying capabilities over complex object-graph collections that are stored in regions. Some of the capabilities provided to JSON documents include the ability to query within and across JSON documents, federated queries with support for joins across JSON, and key-value regions/tables and transactional support.
Online Monitoring Dashboards and Command Line Utilities
GemFire includes a web-based monitoring dashboard, called Pulse, that provides a birds-eye view into the cluster with relevant summary statistics. The operator can use it to drill down into individual members, look at individual data sets, or dive into the performance characteristics of individual aspects of product behavior that are depicted via Pulse.
Powerful Offline Analysis Capabilities
GemFire supports an offline analysis tool called Visual Statistics Display (VSD). VSD opens stat files created by GemFire to provide powerful correlation capabilities across events and across members. VSD is useful in pinpointing performance or resource bottlenecks in the distributed system that otherwise would be hard to diagnose.
Sophisticated Proactive Resource Management Capabilities
GemFire actively manages CPU, disk, memory and network resources. Using auto-tuning techniques, GemFire ensures that applications have the access to the right quantities of resources when executing both read and write workloads. The distributed resource manager detects critical memory conditions in the server and moves to prevent operations that can destabilize the container while working to remedy the excessive use of memory in the server.