By Jack Vaughan
Interoperability is going along for the ride in some of the more advanced applications these days. That is to say that Java technologists are coming up with new distributed architectures for fast transaction systems, improving reliability, and then throwing in some degree of interoperability between Java and .NET applications.

Take as example: Tangosol Inc. Founded by early Java advocates Cameron Purdy and Gene Gleyzer, the company gained ground ‘smoke jumping’ into engagements where J2EE applications were not performing the way they were supposed to. Tangosol was recently purchased by RDB giant Oracle.

In the early days, Tangosol’s principles discovered a common pattern of ‘data starvation,’ or data latency in tiered architectures, according to Peter Utzschneider, vice president of marketing, Tangosol. The firm came out with the Coherence Data Grid, which takes the memory space of individual servers, clusters it, and allows you to use that as a shared memory space for objects.

The approach effectively allows an in-memory data store to be used by multiple applications. Parallel processing and querying is also a use.

An early goal of the product was to improve dependability of Java apps. “We looked at the problem and we put reliability foremost,” said Utzschneider.

With Coherence, he said, “you don’t need to know where the stuff is.”

“We came to focus on the goal of a reliable distributed in-memory architecture,” he indicated. “We automatically backup so it’s there if it fails.”
 
“As people put more data into it,” said Utzschneider, “people realized it did more than solve latency issues. They decided they could [do] parallel queries and parallel processing on top of our technology.”

.NET and J2EE both exist in the types of organizations - often Wall Street powerhouses - that companies like Tangosol pursue.  Some of those customers would like to see their developers able to work with objects in the Coherence Data Grid while still using their native development language. That impetus is behind a recent move Tangosol made to introduce Coherence for the Microsoft .NET Framework.

With this software, the company enables .NET developers to use C# to access data and services. [Tangosol also recently announced Coherence Data Grid for Spring, brining its clustering data grid to Spring Framework users.]

In order to provide interop, Tangosol fashioned PIF-POF, which stands for the Portable Object Format/Portable Invocation Format. It serves as the foundation for Coherence integration with non-Java languages and platforms, such as C++ and .NET.

Company literature describes PIF-POF thusly: “The Portable Object Format (POF) allows object values to be encoded into a binary stream in such a way that the platform/language origin of the object value becomes irrelevant. The Portable Invocation Format (PIF) allows method invocations to be similarly encoded into a binary stream.”

“We are seeing a lot of customers interested in it, “ said Jason Howes, staff engineer, Tangosol. “The have heterogeneous environments and they need something to play nicely.”

The early adopters of this type of software are often denizens of the so-called Real-Time Enterprise. Financial services firms are big adopters, as they gain immediate benefit for performing complex transactions more quickly. Even for these deep-pocketed sorts, the advent of things like blade servers and InfiniBand connections has had a hand in making the Real-Time Enterprise a bit more economical.

With Oracle’s announcement last month that it entered into a definitive agreement to acquire Tangosol, data grid technology will gain broader attention. For its part, Oracle said it purchased Tangosol to further  efforts in what it described as “extreme transaction processing,” or XTP.

Tangosol Coherence technical information

Microsoft’s Dino Chiesa gently takes IBM to task for some aspects of its WebSphere Service Registry and Repository. His take is that connecting to it requires tightly coupled approaches. He wonders if IBM were to unilaterally create WSDL/SOAP interface for IBM WSRR, if that would not qualify as “an IBM-defined protocol, an IBM-defined schema, an IBM contract.” But, of course, he is representing, Microsoft, and that could color his analysis.

… MS Office defines a WSDL and Schema for any Research Service. It is interoperable, and I have shown Java apps acting as Research Services for MS Word 2003. But the interface itself is non-standard. That is to say, Microsoft defines it unilaterally.

Somehow that situation seems ok, while the situation with IBM defining the interface for its SOA Repository seems inappropriate. HAhahahaha! I find this very challenging terrain to travel over. (I understand my view here may be due to a biased perspective, but I am trying hard to guard against it.) The Research Service interface for MS-Word is a feature of the product. MS Word was not designed to be the center point of a SOA infrastructure, as would a SOA repository. MS Word can best be thought of as an endpoint or end-node in a SOA mesh, and the research service interface is just one small artifact that affects that end-node, not the entire mesh. Furthermore, the Research Service interface is small and simple, in comparison to the richer interface you’d need for a SOAP repository. Because of all this, the MS-Office Research Service interface need not be an internationally sanctioned standard, while it seems to me that because a SOA Service Repository does play that central role, and is a much more complex service, it ought to have network interfaces that are based on multi-vendor standards. Is this hypocrisy? I don’t think so.

Read the rest of the story .. Tell us what you think.
WSRR and Interop - on All About Interop Blog

By Scott Balmos
Posted by Jack Vaughan
In my previous blog entry, we looked at a simple way to start combining SOAP and object relational persistence. In this entry, I expand on the example to show pitfalls with Maps and lazy fetching, and the way I work around them.

Here’s the entities again, with the addition of a Map to the User for a collection of user preferences, along with some additional code in the Group entity. I’ll explain below…
package test;

@Entity
public class User implements Serializable {
    private Long id;
    private String userName;
    private String password;
    private String firstName;
    private String lastName;
    private Map preferences = new HashMap();

    @Id
    public Long getId() { return id; }
    public void setId(Long id) { this.id = id; }

    public String getUserName() { return userName; }
    public void setUserName(String userName) { this.userName = userName; }

    public String getPassword() { return password; }
    public void setPassword(String password) { this.password = password; }

    public String getFirstName() { return firstName; }
    public void setFirstName(String firstName) { this.firstName = firstName; }

    public String getLastName() { return lastName; }
    public void setLastName(String lastName) { this.lastName = lastName; }

    @CollectionOfElements @XmlJavaTypeAdapter(StringStringMapAdapter.class)
    public Map getPreferences() { return preferences; }
    public void setPreferences(Map preferences) { this.preferences = preferences; } }

package test;

    @Transient @XmlElement(name = “members”)
    private List getMembers_()
    {
        List vals = new ArrayList(members.size());
        for (User member : members)
        {
            User entry = new User();
            entry.setId(member.getId());
            vals.add(entry);
        }
        return vals;
    }
    private void setMembers_(List members) { this.members = members; } }

So we have a few new things here… In the User class, we have a Map of preferences, with an XmlJavaTypeAdapter pointing to a StringStringMapAdapter class. More on that in a few moments.

Let’s look in the Group class now. We have a few new annotations for the members collection. On the existing getMembers(), we have added XmlTransient. This prevents the XML marshaller in the SOAP engine from returning the full members collection. This is because of lazy fetching, on by default in all mapped collections in JPA.

Lazy fetching keeps the load down on our database servers by not loading all the details of each member entity in a collection. Such as in our Group entity, lazy fetching keeps the database server from having to load more details about the User entities in a group other than just the entity IDs. Other details for each member entity in a collection are retrieved in realtime if they are accessed. Normally this is all well and good. But think about the SOAP XML marshaller. It walks all fields in an object graph. So it will always touch each member in a collection, causing the ORM layer to constantly ask the database to load all details of that member. This completely negates the usefulness of lazy fetching, and can severely degrade your application’s performance.

So in collections that are lazily fetched, I use XML attribute pairing, where the only entity get/set methods available to my SOAP backend application are the original ones, which is within the ORM context of lazy fetching like I want. But now I have a pair of private get/set methods, which is what I want the XML marshaller to use. I mark it with the Transient JPA annotation so the ORM layer doesn’t scream about an unmatched entity field. And I use XmlElement to tell the XML marshaller that this is the get/set method pair I want to use when dealing with that collection. I write the get method so that we build an identical collection to return with member entities that only contain the IDs set.
It doesn’t matter that they’re private, since the XML marshaller uses reflection and ignores method access modifiers.

Now, I will get back a Group entity that I expect, with only a collection of User IDs, just like I would get in a lazily-fetched Group object in my normal Java application. If I wanted more data for the Users, I might have a separate endpoint method that takes a List of IDs, say List getUsersByIDs(List userIDs).

Back to Maps and the User class now… The SOAP spec, oddly, has no base class for any type of key-value Map (whether you call it a Map, Hash, Dictionary, whatever). A few SOAP stacks automatically, in the background, create their own custom Map type. But I’ve lost track of which ones do and which don’t. Most times, they’re incompatible across other stacks.

Here, I use an XmlJavaTypeAdapter to specify a custom XML marshaller / unmarshaller, when dealing with Maps. The code for StringStringMapAdapter is below.
public class StringStringMapAdapter extends XmlAdapter> {
    public Map unmarshal(MapEntry[] mapEntries) throws Exception
    {
        Map map = new HashMap();
        for (MapEntry entry : mapEntries)
        { map.put(entry.getKey(), entry.getValue()); }
        return map;
    }

    public MapEntry[] marshal(Map map) throws Exception
    {
        MapEntry[] values = (MapEntry String>[])Array.newInstance(MapEntry.class, map.size());
        int i = 0;
        for (Map.Entry entry : map.entrySet())
        {
            values[i] = new MapEntry();
            values[i].setKey(entry.getKey());
            values[i].setValue(entry.getValue());
            i++;
        }
        return values;
    }
} and the code for MapEntry, which is a simple generic custom type for a key-value pair, is below.

XmlAdapter is a base class that deals with XML marshalling and unmarshalling. The first generic type is the XML value type you want to marshal to. The second is the native Java type in your application you want to unmarshal from. Most of it is self-explanatory, after reading my explanation of the attribute pairs above for lazy fetching. I do a little bit of reflective nastiness in marshal(), in order to get around the fact that you can’t directly declare a generic array in Java (e.g. MapEntry[] values = new MapEntry[5] for example).

One annoying part about XmlAdapter when dealing with Maps is the fact that you need to write separate classes for each different key-value type pair. Personally, I have one for Map, one for Map, and a few others.

I seem to remember a bit of rumbling in the JAXB forums on possibly integrating automatic Map marshalling / unmarshalling, to remove the need for custom XmlAdapters. And I’m not sure whether .Net has auto-marshalling for Maps. I write much more Java than C#. I’d be interested to hear what other stacks handle Maps automagically, and how they end up looking in SOAP messages on the wire.

By Scott Balmos There are days where I wish to return to the world of ASCII only encoding. Life was simpler back then - in a naive way. Now, we have ASCII, UTF-8, UTF-16, and so on. Even within them, there are different names for different locales (Sun sometimes using different names than the ISO ones come mind). And as far as I have seen so far (please correct me!), SOAP and friends do not really specify encodings to use when passing data that smells like a string. This could seriously blow a major hole in the side of a ship if you want your SOAP service to be truly international in its language support.

Think about it - you have Java on the one hand using UTF-16. Thankfully, so does the CLR (.Net), which covers a decent majority of language interop cases. But what about other languages that use UTF-8, or even pure C/C++ with Favorite Multibyte String Library Of The Week bolted on?

The default choice for multilingual language handling with Linux apps is UTF-8, which covers a lot of the C/C++ programs written there. PHP uses UTF-8. Ruby’s support for Unicode is also minimal, likewise barely handling UTF-8. Python is in the UTF-8 camp also. So, unfortunately, it looks as if we’re heading to another language interop battle. Back to one of the other favorite political battles that Ted Neward was writing about - The Java/.Net “enterprise/large app” languages, vs. the “small app” scripting languages.

Take SOAP out of the equation completely at the moment. What about in the simple cases of a Unicode text file? Is it UTF-16? Is it UTF-8? You can’t really tell by the text file itself. You have to know the encoding beforehand.

Some cursory Googling around reminded me of RFC 1641, which is a spec for specifying the Unicode encoding in the MIME type of a datastream. It seems that it might generally be a good idea to possibly look into extending the SOAP spec to specify encoding type in a string datatype.

Of course that leaves it up to the implementing language to deal with the encoding and decoding magic. But at least it’d be a start. As it is now, we have to explicitly encode our string data into UTF-8 or elsewhere if we want to be “completely” interoperable with languages.

It’d definitely be nice to not have to worry about what encoding my various SOAP client programs use. Just bring in the data, let the string encoding bloodhound class sniff at it a few times, and if it’s an encoding different than my language’s native encoding, it chews up the string byte array and spits out one to me that is in my native encoding, without any extra effort on my part. Ideally, this would simply be an extension to the string data marshaller/unmarshaller in the SOAP stack.

Data types *are* supposed to be vaguely universal, aren’t they?

“Remember, I’m pullin’ for ya… We’re all in this together…”
Scott’s personal blog can be found at http://members.simunex.com/sbalmos/serendipity/.

We had a conversation with a long-time software hand the other day, one whose company had created a means to convert and run Java within .NET. This individual told us of his surprise at the developer community’s reaction to this software when first released. One might summarize the reaction as a good dose of vitriol and some asides to the effect that, if the Creator meant for Java to run within .NET.. well, that actually would have been a mistake on His part. In other words, let’s not mix things.Ted Neward’s post on this Interoperability Blog have pointed to the need to look afresh and ‘leave politics’ at the door when considering apps that integrate Java and .NET functions. Looking back at what has gone before, some of Neward’s words, from a 2004 whitepaper on the topic, which follow, still ring true. And they provide a good entry into what TheServerSide Interoperability Blog seeks to be about. – J. Vaughan, Site Editor, TheServerSide.NET

Making two platforms interact is at once a simple and difficult problem. Simple, in that it’s a fairly closed-requirements solution: if I can work out a few technical details, interaction is achieved. It’s also fairly easy to achieve success–if they can talk, you did it, if not, there’s still work to go. In fact, once you’ve worked out low-level issues like byte order, file/data format, and hardware compatibility, basic interaction between any two platforms is pretty straightforward. (As a matter of fact on this basis was the Internet built.)

* * *Historically, we’ve preferred “n”-tier systems to client/server 2-tier ones, because of the increased scalability intrinsic to “n”-tier systems. An “n”-tier system can scale to much higher user counts, due (among other things) to the shared database connections from a central middle tier to which clients connect. We also prefer the “n”-tier approach because it tends to allow for better separation of responsibilities in code: presentation-layer code goes on the client tier, business logic goes on the middle tier, and data-access code (largely represented by SQL) goes on the resource tiers either on or behind the middle tier machines. 3-tiers, 3 layers, but not always mapped one-to-one.

Drawing a distinction between the tiers and the layers is necessary for good interoperability between the platforms, because interoperability across layers is going to necessitate very different decisions than interoperability within layers. For example, if you have a Windows Forms .NET application that wants to display a Swing applications as a child window, very different decisions are in order than if you want that same Windows Forms applications to talk to a J2EE back end. Web services might suffice for the second requirement; it’ll be an unmitigated disaster if you use Web services for the first.

* * *So often, demos done on the expo show floor clearly prove that the product knows how to talk to the same vendor’s product on the other side of the wire, but rarely if ever demonstrates working with another vendor’s product. So, for example, an ASMX Web Methods Web service can easily declare itself as returning a Hashtable, for example, but once marshaled and sent across the wire, what format should it resemble in the J2EE space? While Java certainly has its own implementation of Hashtable, there’s no love lost between them in implementation details. As a result, it’s a fair bet (barring special code to the contrary), the .NET Hashtable will get rendered into a custom data format that has little to no bearing on the .NET Hashtable in widespread use.

For these reasons, just as with data exchange using XSD, when writing WSDL-based services, always start with the “parts in the middle”: in this case, the WSDL.

What do you think? Have things changed over the years? Are there places where both of these major platforms should be used within a single application?

Interoerability White Paper - 2004

JAXB provides databinding support when developing interoperable Java clients and service providers that interoperate with Microsoft .NET clients and services providers. It is a key part of Project Tango (aka WSIT - Web Service Interoperability Technology), which recently reached a Milestone 2 binary.
http://weblogs.java.net/blog/haroldcarr/archive/2006/09/wsit_milestone.html
http://weblogs.java.net/blog/sekhar/archive/2006/09/jaxb_and_wcf_da.html