Next Generation of SOA is Here: Are You Taking Advantage?

[Cross posted from my JackBe blog]

Since blogging my 2013 BI predictions, I’ve come across ZapThink’s predictions and one of them caught my eye. No, it is not about Big Data. It’s about something that has gone out of fashion, almost. It was about SOA. Here is the excerpt from ZapThink:

Next generation SOA begins to coalesce – For years, ZapThink has touted the difference between the practice of SOA and purported implementations of SOA. Our mantra has always been that SOA is protocol and technology independent: it doesn’t require Web Services, or ESBs, or any of the heavyweight IT infrastructure that has given SOA its reputation of complexity and failure. With the rise of Cloud Computing, architects are increasingly realizing that the bits and pieces of SOA best practice – loose coupling, intermediary-based abstraction, and automated governance, to name a few – can and should be applied as appropriate, independent of the existence of any specific, funded SOA effort.               

Back when we started building our Presto platform at JackBe—seems long ago now—our goal was to actually build the bridge between all the services and data sources in an enterprise with the business users who really need to access it. Over time, we built our flagship Enterprise Mashup Server to fulfill this need and introduce this slice into the enterprise architecture as a meaningful way of extracting ROI from your SOA investments and delivering real value to end users and business users. Now, we do that and more.

For example, we’re delivering comprehensive insight into the full spectrum of operations for critical decision-making and for measuring the business impact of every occurrence. And we’re doing it all in real-time. We don’t advocate complex architectures or re-architecting your existing systems in the name of SOA. Why would you if there is a better way? We like to work with what already exists, leverage and complement them to bring the value of these otherwise inaccessible systems and data sources.

I like to think that we liberate these silos of information, bring them together in a quick, nimble and rapid way that we are known for, and generate new business value out of thin air. By correlating existing data sources and combining new and old information sources, we are able to mash them up to generate new insights in real-time for our customers. This now proven approach is already here and is what we have always believed as the continuation of the SOA marches toward the users. Such an approach already embraces the concepts of loose coupling, intermediary-based abstraction, and so forth, and yes also works with stringent security mechanisms already in place in your enterprise architecture.

So, the next generation of SOA is already here, you don’t need to wait for 2014 to take advantage of it. Are you ready to exploit it?

Using enterprise mashups to save billions

I just came across this from Joe McKendrick on ZDNet Blogs that caught my eye – Study: Increase data usability, save billions.
Here is an excerpt:
Researchers say data usability can be improved by focusing on the following factors:
  • Intelligence of data “can be improved through the accuracy of the prediction, trends analysis, recommendations and profile matching/associations made by the associated applications. For example, what percentage of recommendations made by a business intelligence application results in cross-selling?”
  • Remote access to data and applications is essential in an increasingly mobile workforce.
  • Sales mobility “involves the ability of salespersons to use portable devices and applications to exchange information related to all aspects of a deal or transaction with a customer.”
  • Improvements in data quality will result in improvements that “may come through better and timely decisions (which may increase customer satisfaction, loyalty and hence revenues), as well as fewer errors and rework, lower working capital requirements, faster receivables, etc. (which will lower costs).”

A 10 percent improvement can add up to big dollars. Researchers determined that if a median Fortune 1000 business (36,000 employees and $388,000 in sales per employee) increased the usability of its data by just 10 percent, it would translate to an increase in $2.01 billion in total revenue every year, or $55,900 in additional sales per employee annually. End of excerpt

I find this is very interesting. But, the question is how do you go about achieving this.
  1. You don’t want to be spending millions to save millions.
  2. You don’t want to take years to achieve this goals.

If you can afford to do either, then I suggest, you read no further.

To me, enterprise mashups have been at this for a few years. Take remote access to data and applications for instance. It is dead easy for us to create a new enterprise mashup that wraps the existing data and applications, creates a specific usable view of that data, and then expose this mashup as a Web Service (SOAP or REST), using Apps to your end users and customers. This does not take years, it can be done in hours and days today.

Consider mobility. You want to not only create a more usable view of data, but in turn ensure that this data is available for your mobile users to interact with wherever they are via any portable device. This too is fairly easy to achieve using enterprise mashups.

Basically, enterprise mashups create that agility layer in your enterprise architecture to deliver concise, specific, usable data and applications to your users, without disrupting your current enterprise architecture. This new agility layer can respond rapidly to new business needs by changing the enterprise mashups and creating new enterprise mashups when required.

Enterprise mashups don’t solve the traditional problem of data cleansing in the traditional way…Extract/Transform/Load (ETL). That’s the whole point. Most customers can’t afford (time or resources) cleansing data that way. I think that enterprise mashups thrive when conventional solutions become expensive and time consuming.

Mashups: New and Agile way to Integrate

I came across this interesting post: How Mashups Could Eliminate Integration Projects by Loraine Lawson. In a related post, she refers to John Crupi’s article Enterprise Mashups Part I: Bringing SOA to the People which I would recommend to readers who want to understand JackBe’s take on defining mashups. Anyway, Loraine’s post led me to Ron Schmelzer’s ZapFlash.

Here are some excerpts of Ron’s article that caught my eye, with my take on it:

A year or two ago, assuming that a mashup was a web browser-based, static, user interface composition of web-based functionality would be a reasonable presumption. But in the enterprise context, none of those assumptions necessarily hold – we might want non-Web access to mashed applications, we might want to change them regularly, and we might want to mash up information that exists below the user interface abstraction. For sure, Web mashups might embody the ideals of the original mashup concept, but we now have the desire to mash up a wide variety of IT resources from application to infrastructure to data that might be exposed with a wide range of interfaces – or without. And, it’s the desire to mash up information freed from the application that diversifies the mashup term to include the concept of the data mashup.

Introducing the Mashup Tier
Introducing the Mashup Tier

My take: This hits the point right on what we at JackBe have been saying all along about mashups. While some mashups are done purely in the UI/Browser, in the enterprise, such mashups need to be supported by a new tier, the mashup tier, which sits between the presentation and business tier. So enterprise mashups will have some mashing done in the client, but most of the mashing happens in server side where security, governance, policies can be applied before any mashing can happen in the client.

Another excerpt:

There are many scenarios for composing data, but some are better suited for static, tightly-coupled, IT-driven, non-Service Oriented form. In fact, 80% of the value that businesses derive from data come from the 20% of fixed, highly optimized data integration approaches implemented over decades. In this realm, traditional data integration approaches retain high value. However, it’s the other 80% of data integration requirements, most of which come from the need to meet short-term, often ad hoc, integration requests that cause 80% of the problems. Anyone who has lived long enough in the enterprise IT space knows that business-driven requests for reporting, forecasting, analysis, or other interpretations of data can present significant complications and cost to the IT organization. The reason for this is that the IT organization is set up to meet the recurring needs of the business and not “situational” needs for information.

Long Tail of Enterprise Software Demand
Long Tail of Enterprise Software Demand

My take: This highlights another issue which we have been talking about at JackBe about the long tail & enterprise applications need which was so nicely discussed here by Dion Hinchcliffe.

Bottom line: Something new and interesting is happening in the enterprise architecture space. A new flexible and agile tier is being introduced in the architecture to meet the increasing demand on IT and  add value to existing architecture, applications, services and data. Question is, are you embracing this inevitable change? If not, it’s still not too late. 🙂

SV Code Camp Session Report: Enterprise Mashups

Session: Enterprise Mashups / Presenter: Kishore Subramanian, JackBe

[ Disclaimer: I work for JackBe ] There are about 30 attendees. Kishore is going over the concepts behind our mashup approach like mashables, mashups, and shareables.

Kishore demonstrating SalesForce mashups and walking through the internals including EMML code. Talking about how you can merge data from SalesForce with data from internal proprietary data source and merge/combine to produce a mashup (virtual) service. Showing how to invoke it as a REST interface, which makes it easier to use a complex/mashup service. Now showing how to make a Mashlet using MashletMaker and sharing the mashlet. Now he is demo’ing Wires to show how to visually mashup by drag, drop and connect. Continue reading SV Code Camp Session Report: Enterprise Mashups

Mashing the Enterprise Service Cloud

[Cross-posted from http://blogs.jackbe.com]

In my last blog entry, I defined what I mean by a mashup. In this entry, I want to expand upon how I think mashups differ in the enterprise compared to some of the consumer mashups out there.

I mentioned in my earlier post that mashups based on maps are ubiquitous and something that the users can immediately grasp. But, once you step inside an enterprise, things can be a lot different for mashups. Enterprises are not greatly interested in map-based mashups, neither are they interested in RSS or ATOM services which I think are still not fully leveraged inside an enterprise. Over the past few years, enterprises have invested heavily in SOA and because of that, they are just about getting done with service orienting their enterprise with Web Services (primarily using WSDL). And there are many applications in the enterprise that are still very database oriented. It has become relatively easy for an IT developer to expose services as WSDL based web service. And almost every middleware product out there in the enterprise is capable of exposing services via WSDL or makes it really easy to do so. Hence if you start looking around in an enterprise, what you will find is what I call the Service Cloud.

So what you have in an enterprise is a cloud of services where some are WSDL based, some are REST based, some or XML/HTTP, some are database objects, and so forth. Now, the real challenge is how to mashup these disparate kinds of services, with different service contracts and data formats. Any respectable enterprise platform that offers to mashup services in the enterprise must negotiate these disparate contracts these services expose (e.g. while a WSDL based web service defines a service contract in a WSDL document, a REST based web service has no such contact definition). And not only that, it must be done so that that we can make it easy to access for developers and users alike, and to be able to do that securely and with high-performance and scalability. Some enterprises might regulate the service consumption of even public web services that most of us can freely access on the net (think of the ubiquitous RSS feeds for example). Enterprises might allow consumption of these services, but would like to do so with governance underneath to manage, monitor and secure such activity within the enterprise. This is why we built our enterprise mashup platform at JackBe from the ground up, to address the unique challenges of mashing in the enterprise so that we allow our users to do the following:

  • Create and customize mashups that consume all kinds of services: RSS, ATOM, WSDL, REST, DB, SDO, Java Objects, etc.
  • Securely govern the mashups: policies for who can mashup, what can they mashup
  • Share the mashups to collaborate in the enterprise

My colleague John Crupi wrote about enterprise mashups becoming the new front-tier in the enterprise. I would like comment a bit on what he said in his post. While SOA efforts have been IT driven and IT oriented, even with involvement by the business units, the outcome has been primarily IT and developer focussed. It has neglected an important stakeholder in the enterprise, and that is the User. Therefore, I view the new enterprise mashup layer as a new service layer in the enterprise application architecture that finally begins to leverage SOA from the end user perspective and to make the services layer more user-friendly (and developer-friendly) to build, deploy, share the next-generation enterprise web 2.0 mashup applications.

What do you think?

Defining Mashups

[Cross-posted from http://blogs.jackbe.com]

Over the last year, the term mashup has become extremely popular not only in the consumer software space, but also in enterprise software. It has been a little more than a year since I joined JackBe as VP of Engineering responsible for executing our vision. Over this time, I and our team at JackBe has been busy creating our enterprise mashup platform, which is a unique innovative offering in enterprise software today, by combining the benefits of RIA technologies such as Ajax/RIA, Web Services, Service Oriented Architecture and Mashups.

At the internal technology briefing last week, I and my team presented and demonstrated the features of Presto, JackBe’s enterprise mashup family of products, to 25 developers and consultants. One of the important areas of discussion was around what we mean by a mashup and how the term ‘mashup’ has evolved over the last year to mean almost anything. For example, the following is the definition of the term mashup on Wikipedia:

A mashup is a website or application that combines content from more than one source into an integrated experience.

The problem with this definition from my perspective is that it allows you to practically label anything as a mashup, including a portal application, composite application, and what have you that brings disparate content onto a single web page.

I don’t think I want to disagree with this or any other definition of a mashup floating around in the industry. However, I feel that all the existing definitions and explanations are lacking something so important about the real intent of these mashups. This was also apparent at the discussion a few weeks ago at the IBM Mashup Ecosystem Summit. So, now I feel compelled to offer you what we at JackBe mean by a mashup:

A mashup is a shareable software block created by a user, encapsulating ad-hoc user driven processing of disparate data sources, delivered with a user focused view.

Allow me to elaborate further using the following characteristics:

  1. Mashups are made up of user driven micro-orchestrations. In other words, when I access some data or service, I am performing small operations on the way data is computed and rendered in a very informal and iterative manner. For example, consider how one uses a spreadsheet which is a great example of a canvas where mashups happen in the enterprise when a user pulls in data from different sources, combines, sorts, computes, filters and visualizes by iteratively processing it until he gets the desired results. If you really think about it, what the user is doing is really orchestrating getting data from different sources and processing using one or more operations to yield an integrated view of the data.
  2. Mashups are user focussed, i.e. they are primarily created by the user for the user. They are not the product of some developer or IT guy writing a lot of code. However, current examples of mashups that we see are primarily developer oriented, it is just the beginning. The real value of mashups comes when we enable and empower our end users to perform these mashups the way they want to do it without having to call and rely on IT needs.
  3. Mashups are ad-hoc creations and are very situational based on users immediate short term needs. They are designed by the user to meet a specific need. These mashups have a very short life span varying from hours to days. This is the reason that the users cannot wait and demand IT to create and deliver these mashups.
  4. Mashups can be visual and non-visual. The most common kind of mashup that is out there is probably something that overlays some information on some kind of a map. However, there is more to mashups than just displaying pins on a map. I can mashup disparate data to create an integrated data view. I can mashup processes to create a new process. I can mashup domain models of different application domains to create a new application domain model.
  5. And finally, mashups can be performed on the client side and on the server side. However, I think the best value of a mashup is when we combine both client side and server side mashup functions since we can defer heavy lifting to the server and build intelligent clients that can enhance, complement and focus on user experience.


Before I conclude, I would like to emphasize a few points.

  1. Whether we do visual or non-visual, client-side or server-side, mashups must be user driven and user focussed.
  2. Just because mashups are user driven and user focussed does not mean that we no longer need IT. IT is still a critical part of the whole mashup infrastructure in an enterprise. However, instead of the users going to IT for every little thing they need which can be transient, situational and ad-hoc, they should be able to accomplish what they need without having to rely on traditional IT and experienced programmers to build what they need. For example, imagine what would happen if Microsoft Excel users needed IT help to write a macro or to create a chart from a table of data they put together. Mashups in the enterprise must address these kind of needs of enterprise users accessing disparate data and services to create and share new information in the Enterprise Web 2.0 world.

In my next blog entry, I will discuss more about our work in creating our comprehensive enterprise mashup platform that encompasses the above characteristics. It finally feels great to be able to talk about what we have been obsessively building over the past year!

What do you think? I am really interested to hear other opinions and thoughts around this.

JackBe and the IBM Mashup Ecosystem Summit

[Cross-posted from http://blogs.jackbe.com]

[As a preface to this blog, I want to give my apologies for being away from blogging for the last few months. I’ve been busy creating Enterprise Mashup software here at JackBe. Now that our software is production-ready, I’m looking forward to getting back to posting regularly and getting your feedback on my development efforts. Deepak.]

I was at the Mashup Ecosystem Summit organized by IBM at their offices in San Francisco last week. Our CTO, John Crupi, and our Chief Architect, Raj Krishnamurthy, also attended with me. It was an interesting mix of people from different backgrounds and companies all converging on the concept of Mashups. Jeff Nolan (ex-Teqlo, ex-SAP) gave an interesting talk about his experiences in a starting up a mashup company. Some notable points were: (lack of) availability of APIs; Do-it-yourself Data Formats; Performance can be a challenge; Need for strong visual composition tools; Lack of Standards. I think these are questions that this group will be able to tackle over time. (At least, I hope!)

At one point during the end of the meeting, someone asked: “What really is a Mashup?”. This led to a brief but inconclusive discussion. Which goes to show how nascent this field is that even among the experts in the industry, there is still some uncertainty about how to define and qualify a mashup. (Coincidentally, JackBe has a webcast coming up on May 23 on this very topic: ‘Enterprise Mashup Bootcamp: What, Why and How’. You can register for that webcast here.)

I managed to slip in a question at the Mashup Summit about how the group feels about Composite Applications which were a hot topic a year or more ago and how they relate (or not) to mashups. Predictably, there wasn’t a big desire for this discussion. However, my own personal take is as follows. We in the SOA software industry have been busy implementing SOA in the enterprise over the last few years in our architecture and IT infrastructure. This effort in SOA has largely ignored the end users, and mostly focused on the IT and business stakeholders. Composite Applications are, in a way, IT efforts to provide integration of data, services and processes. While there has been a proliferation of services around the internet/intranet, users have no good tools to use them to do their job more effectively. Therefore, end users do not see much direct benefit of SOA or services or even composite applications.

These SOA/services/composite applications efforts are now undergoing an (r)evolutionary transformation. Enter mashups. And enter the users! The integration is now happening at a higher level in the application stack, much closer to the user. (This is also what Dion Hinchcliffe points out in his blog, where he talks about the 5 styles of mashups.)

There was also some question/opinions about security and how to govern mashups. I think there is no need to be overly paranoid about this. However, service access control is still important in an enterprise world of mashups. Consider that today users are already doing mashups, whether you like it or not. They are doing this mostly in their heads or in a spreadsheet, mostly manually and mostly in a tedious and time consuming way, and in a not-easy-to-replicate-or-teach manner. If the users have access to the data, you don’t really have control on what they do with it in their heads or spreadsheets. So, why not make it easy for the users to do this, faster, more efficiently, collaboratively (so they can share their analysis/knowledge), and to do all this with the least amount of programming skills required (leaning towards no programming required). We are aiming for this at JackBe, as I am sure other vendors in the mashup space are.

Other notable topics of discussion at the Mashup Summit were:

  1. Microformats. My favorite topic. I think Microformats (www.microformats.org) are currently under rated/under used in an enterprise. Expect to see a lot of interesting things in this space in the coming months. We have a thing or two in the works about this as well at JackBe. (Also see Jeff Nolan’s comments on this topic.)
  2. Mashup of Data vs. Processes. It is easy to confuse the two and it’s important to remember they are not the same.
  3. Offline & Synchronization: Ross Mayfield of SocialText asked how important this feature is and whether there really are good use cases that demand this feature. He blogs on this (and related) topics quite a bit and has a interesting perspective.

Bottomline, we at JackBe believe that a mashup has to be user-focused and user driven (not IT driven). Let’s stop searching for the killer app. The killer app is here and it is the User. Let’s enable and empower them to do what they really can do with mashups.