open-source business

Web services are no silver bullet

John Stahl recently gave me some heat for my assertion that Elgg should be built on top of Drupal.

I think that the next few years are going to bring tremendous challenges for applications that do not easily communicate with other applications that are “outside their platform” i.e are written using a different language/framework, run on a different server, etc....The days of monolithic application stacks that try to do everything are fading fast. A new “network-centric” software ecosystem is starting to bloom.

This is wishful thinking. I've spent much of the past few years puzzling over this exact question. While I am personally very much a proponent of web standards and web services I am pessimistic as to how much immediate impact they will have in the evolving marketplace of non-profit/ngo & advocay web technology services.


I didn't always think this way. If you told me a year and a half ago that I would be hawking CivicSpace/Drupal as the über-platform that could meet virtually every need of any size organization I would have told you you were nuts....

After the Dean campaign ended and we started work on CivicSpace our assumptions were:

  • Drupal wouldn't be able to "scale" to meet the needs of large organizations
  • We would not be able to develop the platform quick enough to meet all the core needs of organizations any time soon
  • We would integrate third party service providers to fill the functionality gap organizations required

Then some interesting and unexpected things happened:

  • It turned out Drupal could scale
  • CiviCRM showed up and filled the functionality gap
  • Lots of vendors established quickly growing businesses servicing the technology. The top half dozen firms now employ ~ 50 people between them and each are looking to hire.

This reshaped my thinking on the future of the web-technology marketplace quite a bit. We have an immediate opportunity to commodify the core web-technology organizations need to a single integrated and scalable open-source application stack and this is a very good thing for the marketplace. Over the next few years I believe the advantages afforded by this stack of technology (CS/Drupal/CiviCRM) will far outweigh the benifits realized by the integration of applications accross web-services in terms of costs saved and passed on to technology owners and innovations in technology and services.

Integration accross web-services and web-standards is relatively costly

  • Web applications that are not built from the ground up to be integrated will never play nicely with one another. Let me repeat: if an open-source application is not built to be integrated with your web-application it probably isn't worth the effort to try to integrate it. This renders 95%+ of open-source web applications useless for those looking to leverage the work of other communities.
  • If an application has a strong API it is still tricky to integrate. Even integrating with CiviCRM, an application that was built from the ground up to interface with CMS's and shares the same web-app environment as Drupal, it costs us 2-10X more to integrate with it than it does to integrate with a standard Drupal module.
  • The most cutting edge private sector web-service and web-standards are not that advanced. We still don't have workable single sign-on solution, the cutting edge of semantic information interchange across the web is to embed it in XHTML, the hottest API's from SalesForce and Flickr haven't made much of a dent in the marketplace, and please don't get me started on the semantic web.

The market currently prefers a single integrated stack

  • The majority of the market for web-technology services is owned by companies like Kintera, GetActive, and Convio that serve as one-stop shops for virtualy all an organizations web-technology needs. They have trained technology owners to make purchasing decisions under the assumption that they can cut a check to a single entity that will provide and support a complete "solution". I.E. there is not much point to shopping around for "best of breed" services to integrate, "just give us your business and all your technology problems will go away". This creates an uphill battle in the marketplace for vendors selling what will be seen as "piecemail" solutions created by amassing various web-technology providers products across web-services.
  • There are huge unanswered usability concerns created by integrating together two different applications. This is espcially felt in functionality that is presented to an organization's contituents. Technology owners demand a seamless user-experience across their user facing application space (CMS, event tools, community tools, etc). In the future as more and more functionality moves out of the back-office and onto the web these concerns will only increase.
  • Open-Source vendors eat costs when integrating third party services, mantaining integration, and licensing software, so they have economic incentives to service a complete pre-integrated stack of technology instead of servicing a suite of other providers products integrated over web-services.

The future of the application stack and the role I think webservices will play

I hope and expect that in the next year the CS/CiviCRM/Drupal stack will evolve to a point where it can compete head on with the likes of Kintera, GetActive, and Convio. When this happens open-source vendors will grow into full blown ASP's that will be able to sell services that undercut the current market of proprietary service providers and will be able to grow downmarket to smaller organizations and horizontally to for-profits with overlapping technology needs. With so many organizations and vendors based on the same codebase it will create a very efficient marketplace that supports application development and services. It will also open up the marketplace to anyone wishing to specialize their services towards a vertical, sell data services, or offer 'best of breed' applications. Since the majority of the vendors business will be based entirely around customization and hosting / support and not licensing fees to support product development and sales, they will be much more likely to partner with 3rd party providers or create specialized services themselves. Over time as web-standards evolve the third party services and specialization will grow increasingly important in the marketplace. And then we can all live happily ever after....

Elgg and Drupal: Benefits to competition?

Bill Fitzgerald from the Elgg project recently responded to my previous post, "Moodle, Elgg, CivicSpace, CiviCRM and Drupal should join forces".

"All in all, it's an interesting idea, but not one I'd get behind. The different apps -- particularly Drupal and Elgg -- have some overlapping functionality, that indicate against joining the projects..... Variety is a good thing. To create a loose metaphor, I see it as akin to genetic variability. Differences within the open source community make us stronger."

Ben Werdmuller, the technical lead on Elgg followed up by saying:

"We're big on integration through open standards and would like to encourage that wherever possible; we're not so big on merging and creating bloated, monolithic systems."

In my experience with CivicSpace, Drupal, and now CiviCRM I have found the opposite of what Bill and Ben have said to be true.

Tight interlocking of open-source projects make us stronger, not differences

  • I can garuntee you that if we had chosen to build CivicSpace as a stand-alone app like Elgg instead of on top of Drupal the project would have failed.
  • At the recent Drupal Con in Vancouver there were well over 100 attendees, many of whom make a living around the platform. By my estimates about nearly a third of those people came to Drupal via the CivicSpace project. While our core code contributions to the Drupal project so far have been pretty modest our evangalizing and community support efforts (like DrupalDocs) have had a notable impact to the Drupal ecology.

Building on the same web-app stack makes for a cleaner implementation

  • If two web applications were not built from the ground up to work together they will not be easily integrated. We've seen numerous efforts in the Drupal community to try to integrate external user / content management applications such as PHPList and Gallery but even though the applications provide unique functionality, the integration difficulties are so intense it is yet to be seen if the effort is worth it in the end.
  • Drupal's succesfull Ecommerce module package is a full blown commerce system that handles all product point of sales, billing, and shipping, and tracking inside Drupal. It is a great example of the avantages afforded by building on the Drupal application framework rather than integrating an external specialized application like OSCommerce.
  • Even with an application like CiviCRM that was built from the ground up to be integrated into CMS's like Drupal it is still very costly to integrate.

Building Elgg independently from Drupal will lead to unnecessary competition instead of collaboration between the two projects

  • Not a single functionality point of Elgg is unique. Drupal + contributed modules can be configured to power a site that does _basically_ what Elgg is designed to do. This means virtually all engineering effort going into Elgg is in duplicate to work done in the Drupal community.
  • Decision makers in Universities IT departments will be forced to choose between adopting Elgg or Drupal instead of being able to invest in a solution that captures the best of both projects.
  • The KDE vs. GNOME battle is a great example of the negative effects of open-source project competition vs. the dramatic recent success of the Ubuntu project built on top of Debian.

Digging into OpenBRR of Moodle and Sakai

"Business Readiness Ratingâ„¢ (BRR) is being proposed as a new standard model for rating open source software. It is intended to enable the entire community (enterprise adopters and developers) to rate software in an open and standardized way....sponsored by Carnegie Mellon West Center for Open Source Investigation, O'Reilly CodeZoo, SpikeSource and Intel...BRR will give companies a trusted, unbiased source for determining whether the open source software they are considering is mature enough to adopt.... The ultimate goal of BRR is to give companies a trusted, unbiased source for determining whether the open source software they are considering is mature enough to adopt. It will help adopters assess which open source software is best suited to their needs and enable them to share findings with the community."

Functionality (25%)
Moodle: 5
Sakai: 3

Usability (20%)
Moodle: 4
Sakai: 4

Documentation (15%)
Moodle: 3.1
Sakai: 3.1

Community (12%)
Moodle: 4.2
Sakai: 3.2

Security (10%)
Moodle: 4.2
Sakai: 3.4

Support (10%)
Moodle: 4
Sakai: 1.5

Adoption (8%)
Moodle: 4.4
Sakai: 4.2

(Note: the Y axis scale is 4.1 - 4.4, not 0-5. Sorry, couldn't figure out MS Word)

Total Weighted Score
Moodle: 4.19
Sakai: 3.23


Oracle is trying to buy open-source

Read the story here

  • Jboss is asking for $400M
  • Zend is asking for $200M

Why? This is pure defense, not an open-source embrace. Anyone who thinks Larry Elison loves the penguin is smoking crack. Oracle makes it's billions on licensing, Jboss and Zend make their millions on extensions and support. The open-source stack commodifies Oracle cash cow and sucks it's margin's bone dry. How many people who work inside Oracles big black tower read slashdot and play with LAMP or Ruby-on-Rails in their free time? Enough to know closed source ERP is fucked. Enough to freak out Elison. Enough to spend a billion or so squashing some peons. Zend is the ringleader of PHP and Jboss is the darling of open-source Java and Oracle is trying to take them out.

Higher-ed LMS market penetration: Moodle vs. Blackboard+WebCT vs. Sakai

  1. Moodle: 2,981 deployments / 54% market share
    8,772 X (348 / 1024)
  2. Blackboard + WebCT: 2,500 deployments / 45% market share
    3,700 - 1,200 K-12
  3. Sakai: 35 deployments / .63% market share
  4. Taken from the Sakai partners page.

Related Posts:
Sakai vs. Moodle
Digging into OpenBRR Rating of Sakai and Moodle

Costs of forking

A lot of full time Drupal/CS consultants I've been talking to here in Vancouver have been sharing concerns about how to meet the needs of their clients while still being able to contribute code they development back to the Drupal community. The general assumption is that most clients are oblivious to the mechanics of open-source communities and assume that they are paying simply for an engineer to develop new functionality and ignore all the time required to work with the Drupal community to make sure contributions and patches are accepted and merged back.

I think the simplest solution is to figure out a way to asses costs to forking code that can be communicated to customers upfront when contracts are negotiated.

How about this estimate?

  1. Costs to mantain your own forked module: 25 developer hours X $100 an hour X 2 times a year = $5,000 a year per module.
  2. Open-Source development resources lost: 5 hours per patch X $100 an hour = $500 per patch.

Using these estimates I would think that consultants should be able to justify adding in a line item to their contracts up front to cover their costs to contribute their code back to the community. Do you think this will work?

Supporting Drupal platform development

I am posting from the basement of 800 Robson Center Vancouver BC at the Open Source CMS Summit / DrupalCon 2006. This is without a doubt the most fun work-travel event I've taken part in (I've done more than twenty in the past year). One year ago at the DrupalCon in Vancouver there were about a dozen of us. In Portland six months ago there were thirty or fourty. Today there are well over one hundred Drupaler's in attendance hanging out, learning, hacking, discussing, drinking beer, and scheming for Drupal world domination.

The Drupal software, the user base, and espescially the economy around the platform have seen an explosive growth in the last year. When we started CivicSpace a year and a half ago there were only a few of us who made a living in and around Drupal. Today there are well over one hundred people employed to hack on Drupal between the contract firms and the user-developer base. With a little napkin math I estimate that in a year the Drupal economy has grown from ~$500K a year (10 X $50K) to well over $5M a year!

Despite this phenominal growth I have some concerns about what the next year holds in store for the Drupal platform and the community around it. As I see it, most of the value created by the Drupal community that vendors and consultants rely on to sell as services to customers come from unpaid core contributions. Examples of this are the majority of 4.7 improvements: multiple block regions, new forms API, installer work - these were all unpaid contributions to the community by volunteers or pro-bono work from Drupal firms. At the same time, many of the seasoned Drupal core hackers are getting job offers left and right to come and do Drupal consulting full time. This is creating a scenario in which while many more people are paying their bills through Drupal hacking and more and more business are built around business value created by the Drupal community, it is not getting any easier to make sure the heavy lifting on core work is getting done. For example: I know of no funding availiable to support key Drupal core projects such as continued forms API work, actions & workflows development, CCK, and most importantly; usability improvements.

In the next year I am expecting the economy around Drupal to continue on it's fantastic pace. I would not be surprised if by 2007 the economy was well over $20M annually. My big question here in Vancouver is: What can we do now to make sure that some of the money coming into the community over the next year is put back in to supporting the core work that we as a community are relying on?

Syndicate content