$blogname

vFabric GemFire — 20 demos in less than 5 minutes.

with 4 comments

vFabric GemFire is a Java-oriented in-memory distributed caching platform. To translate that into English, it’s a way of sharing and processing that’s extremely fast because the data is kept live in memory, and is also extremely scalable because GemFire applications can automatically discover and share data between peers over the network.

GemFire can run peer-to-peer or standalone. When you’re using it peer-to-peer you embed GemFire into your application. When the application launches, it discovers all its peers automatically via the network.

You run GemFire standalone through something called cacheserver. In the standalone case you launch one or a few cacheservers and then a large number of clients can connect to it. This mode resembles what you’d see if you used something like memcache or redis. This mode is good if you need to centralize data, write data to disk or if you want to perform continuous queries.

GemFire includes an extensive example set that gives you a good sense of what’s possible. Unfortunately I found the getting started guide a bit hard to follow. Part of it was just the usual Java BS: any time you’re asked to set environment variables to get software working you know you’re in for a rough ride. But beyond that there are  various different piece parts you have to collect and configure before you can really get started, and the way to run the examples themselves is not very clear, the directory structure of the sample files seems inconsistent with the way the instructions tell you to run the samples. Maybe that was my error but I bet a lot of people would make the same error.

To try to help things a bit I wrote a script that tries to completely automate the demo experience. The script downloads all the necessary code and then presents a menu where you can select any demo you want to run. Some of the examples require a cacheserver, and in those cases the demo script will automatically start and stop one for you. There’s nothing to think about, just run the script and it should work (leave me a note if it doesn’t!)

Download the script or just copy and paste it from the box below. After you download, just run it and you’re on your way (see below for a note on system requirements).

#!/usr/bin/python
 
import string
import time
import urllib
import os.path
 
# If you want to see the commands as they execute, set this to 1.
verbose = 0
 
files = {
  "gemfire" : {
    "name" : "GemFire JAR",
    "file" : "gemfire-6.5.1.jar",
    "url"  : "http://dist.gemstone.com/maven/release/com/gemstone/gemfire/gemfire/6.5.1/",
    "sum"  : "f9b633da145a2afb27d2015ab76010818c0c2ee2"
  },
 
  "antlr" : {
    "name" : "ANTLR JAR",
    "file" : "antlr-3.1.3.jar",
    "url"  : "http://www.antlr.org/download/",
    "sum"  : "acd5253cf1eba7bdb133f14cd77b0ba2fd219f98"
  },
 
  "examples" : {
    "name" : "GemFire Examples",
    "file" : "GemFire_v6.5_SampleCode.zip",
    "url"  : "http://community.gemstone.com/download/attachments/6032137/",
    "sum"  : "6a1d6f5e4a9a4e1236caecac2935b5cb841bcb10"
  }
}
 
demos = {
 'BenchmarkAckConsumer' : {},
 'BenchmarkAckProducer' : {},
 'BenchmarkClient' : { "cacheServerArgs" : "start cache-xml-file=xml/BenchmarkServer.xml" },
 'ClientConsumer'  : { "cacheServerArgs" : "start cache-xml-file=xml/Server.xml" },
 'ClientWorker' : {},
 'CqClient'        : { "cacheServerArgs" : "start cache-xml-file=xml/CqServer.xml" },
 'DataEviction' : {},
 'DataExpiration' : {},
 'DataOverflow' : {},
 'DataPersistence' : {},
 'DeltaPropagationClientFeeder' :
        { "cacheServerArgs" : "start cache-xml-file=xml/DeltaServer.xml" },
 'DeltaPropagationClientReceiver' : {},
 'DeltaPropagationServer' : {},
 'DistributedLocking' : {},
 'DurableClient' : {},
 'DurableServer' : {},
 'FunctionExecutionPeer1' : {},
 'FunctionExecutionPeer2' : {},
 'I18nClient' : {},
 'MultiuserSecurityClient' : {},
 'MultiuserSecurityServer' : {},
 'MyArrayListResultCollector' : {},
 'PartitionedRegionVM1' : {},
 'PartitionedRegionVM2' : {},
 'PushConsumer' : {},
 'PushProducer' : {},
 'Querying' : {},
 'SecurityClient' : {},
 'SecurityServer' : {},
 'Transactions' : {},
}
 
def download(url, file):
        urllib.urlretrieve(url, file)
 
def downloadFile(fileInfo):
        # Check to see if the file exists.
        if os.path.isfile(fileInfo["file"]):
                return
 
        # Otherwise download it.
        url = fileInfo["url"] + fileInfo["file"]
        print "Downloading " + fileInfo["name"]
        download(url, fileInfo["file"])
 
def runDemo(demoName, args=""):
        baseCommand = "java -classpath classes:../gemfire-6.5.1.jar:../antlr-3.1.3.jar"
        fullCommand = baseCommand + " quickstart." + demoName + " " + args
 
        os.chdir("quickstart")
        # Some demos require a cache server. Start one up.
        cacheServerNeeded = 0
        if "cacheServerArgs" in demos[demoName]:
                cacheServerNeeded = 1
                print "Starting a cache server for this demo."
                cacheServerCommand = baseCommand + \
                    " com.gemstone.gemfire.internal.cache.CacheServerLauncher " + \
                    demos[demoName]["cacheServerArgs"]
                if verbose:
                        print "Starting cache server using: ", cacheServerCommand
                os.system(cacheServerCommand)
 
        # Start the demo.
        if verbose:
                print "Starting demo using: ", fullCommand
        os.system(fullCommand)
 
        # Stop the cache server if we started one.
        if cacheServerNeeded:
                print "Stopping cache server."
                cacheServerCommand = baseCommand + \
                    " com.gemstone.gemfire.internal.cache.CacheServerLauncher stop"
                os.system(cacheServerCommand)
                if verbose:
                        print "Stopping cache server using: ", cacheServerCommand
 
        os.chdir("..")
        demoName = raw_input("Press enter to return to the menu.")
 
def doMenu():
        while 1:
                print "Ready to run a demo."
 
                # Present choices.
                after = ""
                demoNames = demos.keys()
                demoNames.sort()
                for demo in demoNames:
                        formattedDemo = "%-33s" % demo
                        print formattedDemo, after,
                        if after == "": after = "\n"
                        else: after = ""
                print
                input = raw_input("Enter your choice: ")
                tokens = input.split(" ")
                demoName = tokens[0]
                arguments = string.join(tokens[1:], " ")
                try:
                        demos[demoName]
                        runDemo(demoName, arguments)
                except KeyError, ValueError:
                        print "Unknown demo, try again."
                        time.sleep(1)
 
def main():
        # Ensure we have all the files we need.
        for file in files.keys():
                downloadFile(files[file])
 
        # Extract the samples if they don't already exist.
        if not os.path.isdir("quickstart"):
                os.system("unzip -n -q GemFire_v6.5_SampleCode.zip")
 
        # Disable verbose output.
        fd = open("quickstart/classes/gemfire.properties", "w")
        fd.write("log-level=warning")
        fd.close()
        doMenu()
 
main()

You need a system with Python, Java and unzip all in your path (no environment variables, please!). Other than that it should work anywhere. Leave me a comment if it you try it and it doesn’t work. Otherwise have fun checking out what GemFire has to offer, don’t forget to check out the continuous query and function execution examples!

Written by Carter Shanklin

December 19th, 2010 at 8:09 am

Posted in Uncategorized

Tagged with , ,

4 Responses to 'vFabric GemFire — 20 demos in less than 5 minutes.'

Subscribe to comments with RSS or TrackBack to 'vFabric GemFire — 20 demos in less than 5 minutes.'.

  1. “4 billion IP address should be enough for anybody” — Some Java person.

    Note that if you’re running on a modern OS which has IPv6 on by default, Ubuntu 10.04 for instance, lots of Java stuff still can’t cope well with IPv6. This stuff won’t work as written if you’ve got IPv6 enabled.

    The easiest thing to do is just disable IPv6 if you don’t need it.

    Carter Shanklin

    16 Jan 11 at 4:13 am

  2. I savor, lead to I discovered exactly what I was having a look for. You’ve ended my four day long hunt! God Bless you man. Have a great day. Bye

  3. Everything is very open with a clear clarification of the issues. It was definitely informative. Your site is very useful. Thank you for sharing!

    Talisha Keib

    7 Nov 13 at 11:20 pm

  4. Thanks foor the ρerform. Post assisted me a good deal

    god hygien

    18 Aug 14 at 5:53 am

Leave a Reply