$blogname

Archive for the ‘gemfire’ tag

GemFire DTD

without comments

If you’re writing GemFire cache.xml files you’ll probably want to refer to the GemFire DTD files for valid syntax. These are hosted on gemstone.com but don’t seem to appear in Google searches so I’m putting this blog in place to facilitate finding them.

GemFire 6.6 DTD on gemstone.com (latest version)
Ye Olde GemFire 6.0 DTD (consider an upgrade)

Written by Carter Shanklin

March 23rd, 2012 at 11:14 pm

Posted in GemFire,Uncategorized,vfabric

Tagged with ,

Using GemFire from Maven.

with 2 comments

If you’re using GemFire in your software, there’s a good chance you’d like to build it using Maven. GemFire is hosted in a Maven repository but the details don’t seem to be documented, so this post is a bit of a “Missing Manual” entry.

It’s pretty easy, you need to set up the GemFire repository and add it as a dependency in your pom.xml.

Repository:

  <repositories>
    <repository>
      <id>gemstone</id>
      <url>http://dist.gemstone.com.s3.amazonaws.com/maven/release/</url>
    </repository>
  </repositories>

And dependency:

    <dependency>
        <groupId>com.gemstone.gemfire</groupId>
        <artifactId>gemfire</artifactId>
        <version>6.6</version>
    </dependency>

If you want to see all the versions of GemFire available you can on the repo root page.

I used this in the course of putting together a Continuous Query client for GemFire, which lets you run SQL-like queries against GemFire that returns matches in real-time as data enters GemFire. So you can run something like “SELECT * from /myregion WHERE totalsale > 100″ and any time an object is inserted into GemFire that has a totalsale value greater than 100 you’re immediately notified of it. It’s pretty interesting and the client is a nice way to interactively discover the system’s capabilities.

One other thing that might not be totally obvious is that GemFire is free for development use, there is a non-expiring license to use it for up to 3 connections (clients or members in the distributed system), so feel free to try it out, play around give feedback.

Written by Carter Shanklin

November 1st, 2011 at 6:05 am

Posted in GemFire

Tagged with , , ,

vFabric GemFire — 20 demos in less than 5 minutes.

with 5 comments

vFabric GemFire is a Java-oriented in-memory distributed caching platform. To translate that into English, it’s a way of sharing and processing that’s extremely fast because the data is kept live in memory, and is also extremely scalable because GemFire applications can automatically discover and share data between peers over the network.

GemFire can run peer-to-peer or standalone. When you’re using it peer-to-peer you embed GemFire into your application. When the application launches, it discovers all its peers automatically via the network.

You run GemFire standalone through something called cacheserver. In the standalone case you launch one or a few cacheservers and then a large number of clients can connect to it. This mode resembles what you’d see if you used something like memcache or redis. This mode is good if you need to centralize data, write data to disk or if you want to perform continuous queries.

GemFire includes an extensive example set that gives you a good sense of what’s possible. Unfortunately I found the getting started guide a bit hard to follow. Part of it was just the usual Java BS: any time you’re asked to set environment variables to get software working you know you’re in for a rough ride. But beyond that there areĀ  various different piece parts you have to collect and configure before you can really get started, and the way to run the examples themselves is not very clear, the directory structure of the sample files seems inconsistent with the way the instructions tell you to run the samples. Maybe that was my error but I bet a lot of people would make the same error.

To try to help things a bit I wrote a script that tries to completely automate the demo experience. The script downloads all the necessary code and then presents a menu where you can select any demo you want to run. Some of the examples require a cacheserver, and in those cases the demo script will automatically start and stop one for you. There’s nothing to think about, just run the script and it should work (leave me a note if it doesn’t!)

Download the script or just copy and paste it from the box below. After you download, just run it and you’re on your way (see below for a note on system requirements).

#!/usr/bin/python
 
import string
import time
import urllib
import os.path
 
# If you want to see the commands as they execute, set this to 1.
verbose = 0
 
files = {
  "gemfire" : {
    "name" : "GemFire JAR",
    "file" : "gemfire-6.5.1.jar",
    "url"  : "http://dist.gemstone.com/maven/release/com/gemstone/gemfire/gemfire/6.5.1/",
    "sum"  : "f9b633da145a2afb27d2015ab76010818c0c2ee2"
  },
 
  "antlr" : {
    "name" : "ANTLR JAR",
    "file" : "antlr-3.1.3.jar",
    "url"  : "http://www.antlr.org/download/",
    "sum"  : "acd5253cf1eba7bdb133f14cd77b0ba2fd219f98"
  },
 
  "examples" : {
    "name" : "GemFire Examples",
    "file" : "GemFire_v6.5_SampleCode.zip",
    "url"  : "http://community.gemstone.com/download/attachments/6032137/",
    "sum"  : "6a1d6f5e4a9a4e1236caecac2935b5cb841bcb10"
  }
}
 
demos = {
 'BenchmarkAckConsumer' : {},
 'BenchmarkAckProducer' : {},
 'BenchmarkClient' : { "cacheServerArgs" : "start cache-xml-file=xml/BenchmarkServer.xml" },
 'ClientConsumer'  : { "cacheServerArgs" : "start cache-xml-file=xml/Server.xml" },
 'ClientWorker' : {},
 'CqClient'        : { "cacheServerArgs" : "start cache-xml-file=xml/CqServer.xml" },
 'DataEviction' : {},
 'DataExpiration' : {},
 'DataOverflow' : {},
 'DataPersistence' : {},
 'DeltaPropagationClientFeeder' :
        { "cacheServerArgs" : "start cache-xml-file=xml/DeltaServer.xml" },
 'DeltaPropagationClientReceiver' : {},
 'DeltaPropagationServer' : {},
 'DistributedLocking' : {},
 'DurableClient' : {},
 'DurableServer' : {},
 'FunctionExecutionPeer1' : {},
 'FunctionExecutionPeer2' : {},
 'I18nClient' : {},
 'MultiuserSecurityClient' : {},
 'MultiuserSecurityServer' : {},
 'MyArrayListResultCollector' : {},
 'PartitionedRegionVM1' : {},
 'PartitionedRegionVM2' : {},
 'PushConsumer' : {},
 'PushProducer' : {},
 'Querying' : {},
 'SecurityClient' : {},
 'SecurityServer' : {},
 'Transactions' : {},
}
 
def download(url, file):
        urllib.urlretrieve(url, file)
 
def downloadFile(fileInfo):
        # Check to see if the file exists.
        if os.path.isfile(fileInfo["file"]):
                return
 
        # Otherwise download it.
        url = fileInfo["url"] + fileInfo["file"]
        print "Downloading " + fileInfo["name"]
        download(url, fileInfo["file"])
 
def runDemo(demoName, args=""):
        baseCommand = "java -classpath classes:../gemfire-6.5.1.jar:../antlr-3.1.3.jar"
        fullCommand = baseCommand + " quickstart." + demoName + " " + args
 
        os.chdir("quickstart")
        # Some demos require a cache server. Start one up.
        cacheServerNeeded = 0
        if "cacheServerArgs" in demos[demoName]:
                cacheServerNeeded = 1
                print "Starting a cache server for this demo."
                cacheServerCommand = baseCommand + \
                    " com.gemstone.gemfire.internal.cache.CacheServerLauncher " + \
                    demos[demoName]["cacheServerArgs"]
                if verbose:
                        print "Starting cache server using: ", cacheServerCommand
                os.system(cacheServerCommand)
 
        # Start the demo.
        if verbose:
                print "Starting demo using: ", fullCommand
        os.system(fullCommand)
 
        # Stop the cache server if we started one.
        if cacheServerNeeded:
                print "Stopping cache server."
                cacheServerCommand = baseCommand + \
                    " com.gemstone.gemfire.internal.cache.CacheServerLauncher stop"
                os.system(cacheServerCommand)
                if verbose:
                        print "Stopping cache server using: ", cacheServerCommand
 
        os.chdir("..")
        demoName = raw_input("Press enter to return to the menu.")
 
def doMenu():
        while 1:
                print "Ready to run a demo."
 
                # Present choices.
                after = ""
                demoNames = demos.keys()
                demoNames.sort()
                for demo in demoNames:
                        formattedDemo = "%-33s" % demo
                        print formattedDemo, after,
                        if after == "": after = "\n"
                        else: after = ""
                print
                input = raw_input("Enter your choice: ")
                tokens = input.split(" ")
                demoName = tokens[0]
                arguments = string.join(tokens[1:], " ")
                try:
                        demos[demoName]
                        runDemo(demoName, arguments)
                except KeyError, ValueError:
                        print "Unknown demo, try again."
                        time.sleep(1)
 
def main():
        # Ensure we have all the files we need.
        for file in files.keys():
                downloadFile(files[file])
 
        # Extract the samples if they don't already exist.
        if not os.path.isdir("quickstart"):
                os.system("unzip -n -q GemFire_v6.5_SampleCode.zip")
 
        # Disable verbose output.
        fd = open("quickstart/classes/gemfire.properties", "w")
        fd.write("log-level=warning")
        fd.close()
        doMenu()
 
main()

You need a system with Python, Java and unzip all in your path (no environment variables, please!). Other than that it should work anywhere. Leave me a comment if it you try it and it doesn’t work. Otherwise have fun checking out what GemFire has to offer, don’t forget to check out the continuous query and function execution examples!

Written by Carter Shanklin

December 19th, 2010 at 8:09 am

Posted in Uncategorized

Tagged with , ,