Category: Coherence

I’m calling this Part 1 of what may be an ongoing theme in this blog: considerations for storing bitemporal in a distributed data store. In my particular case, that data store is Oracle’s Coherence.

Standard three-timestamp or four-timestamp bitemporal data makes use of a Transaction or Transaction Start time to give distinguish entries with partially or fully overlapping Valid Time ranges. In fact, the Transaction Start time should be unique for all entries which refer to a given atomic unit of data in the data store (a particular object, if your datastore records objects, or a particular attribute if that is what it records). In some RDBMS, the uniqueness of Transaction Start is accomplished automatically. In others, it may be easily accomplished by virtue of having a single centralized transaction point and system clock. However, in distributed environment, one must make a little extra effort due to the distributed nature of transactions and the probable differences in system clocks among the members of the system.

The solution I have used is to write a so-called “sequential now” timestamp generator. This is a service provider which behaves very similarly to a distributed ID generator. It transactionally ensures that, when demanded, the client receives a timestamp that represents “now” except in the case that “now” is equal to the previously provided “now”, in which case it increments the previous “now” by the smallest possible increment (defined by the granularity of the system or chronon) and then provides it. Provided that the granularity of your system or chronon is small, ie. on the order of nanoseconds, this is likely to be an acceptable approach. If, on the other hand, your granularity is on the order of seconds, minutes, or hours, increasing transaction rates will cause an undesirable situation where the “sequential now” creeps further and further away from system now or realtime now. In such a situation, another approach is likely to be required.

In Coherence, such a generator can be easily implemented as an EntryProcessor. That is left as an excercise for the reader.

The situation I am going to describe might be obvious to some people, but apparently it wasn’t to me. I was diagnosing a problem where the tangosol.coherence.clusterport system property specified on the command line (with the -D flag) was being ignored by my coherence-enabled code. The code was not overwriting the system property, however it was specifying values for other system properties that override Coherence settings, among them, it was specifying an override file by setting the tangosol.coherence.override property. Here’s an analysis of the problem:

If you write your own Coherence “operational override” file (specified with the tangosol.coherence.override property), you must re-iterate the system-property attribute on each element whose default you wish to maintain.

For example, in tangosol-coherence.xml, the clusterport command line override is specified in the port tag with: . If you were to override in your custom override file, you must include the system-property if you want to keep using it. In other words, the “preconfigured overrides” they mention throughout the documentation (especially here) are only guaranteed to be present if you use the default configuration files. Otherwise it is up to you.

This makes sense if you look at the provided override files, for example tangosol-coherence-override-dev.xml. In the time-to-live element, for example, it re-iterates the system-property setting that is also present in tangosol-coherence.xml. Unfortunately, I did not notice this, and instead assumed that I did not have to re-iterate the “preconfigured override”. I like this ability to remove the command line override functionality on specific elements. It allows us to make some of our configurations a bit more rock solid so that command line parameters cannot mess it up.

Are you getting the following error when trying to use a Coherence Extend client?

java.lang.IllegalStateException : SafeCluster has been explicitly stopped or has not been started

You may want to disable the packet publisher on the client (override is tangosol.coherence.tcmp.enabled). To be extra sure, you could even set client’s TTL is set to 0 (which will prevent it from finding clusters on other computers), and if you run the cache server locally you may even want to give it a clusterport that no cluster members use. Why? Because even though Extend clients are supposed to communicate over TCP, in my experience they will try to discover clusters via multicast and, if found, attempt to join them via the usual UDP connection. Particularly when they are run on the same machine as the server.

Oracle claims that the Coherence API is the same whether or not during runtime it is a full member of the grid or it is an Extend client:

Coherence*ExtendTM clients provide the same rich API support as the standard Coherence API without being full data members of the cluster.^ref

While strictly speaking that is true, I am learning the API usage may differ. The latest example I have is the following code, which works fine when run inside (as a full member of) the grid:

InvocationService invocationService = getInvocationService();
Set members = invocationService.getCluster().getMemberSet();
Map<Member, CoherenceMember> memberResults = invocationService.query(  
    new MyCacheDistributionQueryInvocable(), members);

However, when run as an Extend client, this code gives an error that Google as yet can shed no light on:

java.lang.IllegalArgumentException : directed query not supported; the specified Member set must be null

So, the second parameter to InvocationService#query() has to be null, if run as an Extend client. So, the question is now: how do I get information about the individual members of the grid from the Extend client? Or is it even possible? This post may end up being helpful.