IBM Turns to Open Source Development

By David Worthington | Published June 13, 2005, 2:47 PM

(continued from previous page)

BetaNews: So IBM feels it is taking the best of the open source culture. How will IBM keep its source code secure and guarantee the code's safety and integrity inside of this system?

Heintzman: Interesting question. You know that question cuts two ways. There is considerable merit in open source when you've got many eyeballs you can drive out bugs and security holes and flaws and fix them more efficiently. I think that that's true. I think that it is also true that commercial software companies like IBM have very rigid quality control best practices, reviews and scans - disciplines that they have developed over decades.

We are sharing with the open source community to try to help them mature. So we find ourselves borrowing, as you say, the best of this new culture and merging it with a lot of the lessons, tools, techniques and practices that we have developed over many, many years to come up with this productive hybrid. That's what Community Source is really all about.

BN: Could Community Source assist in breaking down language barriers between IBM's locations throughout the world since programming code is a universal language? IBM is a company with many laboratories outside of the United States so there must be linguistic challenges.

Heintzman: Absolutely. We've got a whole bunch of laboratories on every corner of the planet. I've never really thought about it. Certainly in all of the various different programming reviews that we have looked at, linguistic diversity has never been one of our prohibitors that I am aware of.

Will this help? Sure. Is it "the" reason to do it? I would say it's probably a "nice to have."

BN: So is linguistic diversity now somewhat easier to have than in the past?

Heintzman: I wouldn't quite phrase it that way. The frank reality is that in the world of coding there really is kind of one language. Programmers work with certain programming tools and we build our reference materials and our documentations in reference languages and then we massively translate things. Our laboratories have been working from common linguistic code bases for ages. It's too complex to try to integrate all of that componentry from different laboratories unless you have some kind of normalization.

BN:In a press release IBM said, "Community Source has made it easier for IBM developers to create and trade Lego-like components of code that can be plucked from a library, quickly assembled and re-used later."

Considering how many ongoing projects IBM has, how will these 'legos' be organized? We touched on this briefly when we discussed the full time IBM employees that are dedicated to making sure that these modules of code are used appropriately.

Heintzman:We've got a division whose role is to work with the core architects for all the software groups to get the big, big, big picture. They drill down what the big platform picture looks like, they identify targets that would lend themselves to componentization and high degrees of reuse.

This is a journey for us. We are starting the modules that will give us the best payback and we will move to other secondary modules. They sit down and kind of look from an architectural standpoint and they define lists. Then they go to the laboratories that have responsibilities in those areas and they sit down and say, "This is what it means to be defined as a component. This is how we are going to normalize things to allow for high degrees of reuse."

That's an interesting dialog because when you take on the responsibility of owning a component, you are actually taking on a higher degree of financial burden. It is more expensive to develop to the high degree of specification which components require and to subsequently support them and provide support to all of the groups that consume them.

Be that as it may, we have made a decision that says, "This is our strategic direction," and each of the laboratories understand that they have responsibilities to live up to, and yes they may be spending more money on one particular component, but they will also get benefits because they are now consuming components that they don't have to fund themselves.

BN: IBM states that development is 30 percent faster using an open source model: I am just curious do you have any empirical data to back that up and how did you come to that conclusion?

Heintzman: That's a good question, and unfortunately I am not the most skilled person to answer the specifics of the methodology behind those kinds of numbers.

At the high level there are all different kinds of code and there are all kinds of ways of measuring the productivity of programming. Say you are building a product and you need an administration console. The fact that there is one in the library that you can plug in is almost infinitely productive relative to hiring a bunch of coders and going out and building the thing. So how much extra productivity do you get from that now? A lot.

If you use these Community Source approaches, the more quickly and more cost effectively you get to the right answer. You could get that console built by taking those programmers and locking them in a room and then off they go, but will they produce a piece of code at the end of the day that really is highly productive for reuse across the company?

There are all sorts of ways of measuring it and 30 percent is kind of our rough averaging of how quickly the projects that we have brought into production have succeeded versus the percentage of them that have been deemed failures, and how many developers its was required to code a particular function versus how we have done it before. We actually do have specific metric we measure for everything and if you take a step back and put your thumb out we are getting about a 30 percent improvement.

BN: It struck me that IBM said, "at companies like Microsoft (just think of Longhorn, taking years to deliver at the risk of obsolescence upon delivery)." That's a pretty bold statement. How can IBM say that considering IBM itself has for a very long time used the same hierarchical approach?

Heintzman: I think there are actually two issues that we are commingling here, so let's make them a little distinct. So one of the issues is code that is highly integrated code that was built originally for a specific purpose. These code bases are very expensive to modernize, to add functionality to develop and support.

Certainly the Microsoft Windows platform, and we've witnessed the continual delays of the platform, is an incredibly complex engineering problem they have. They've got millions and millions of lines of code and they are all spaghetti'ed together and as soon as they go and change or fix something it can ripple all the way through the code.

We had exactly the same challenges in our Lotus Notes platform. It is a very mature, very complex highly functional platform, but it is very expensive to develop it and to maintain it. And so what we've decided to do is work to rip the Notes code out and replace it with components to kind of systematically move towards componentization.

Our hope is -- and our early experience with it is actually very encouraging -- that we will be able to bring products to the marketplace very, very quickly. If you look at the rate of innovation, feature addition and quality enhancement in something like our portal platform, they are versioning at unbelievable rates. It went from almost nothing to an industry leading incredibly comprehensive platform very quickly.

You couldn't do that with the same rate of innovation using the highly structured models of the code bases. So componentization gives you some extra flexibility there.

The second question: Now, if you buy the first argument that says that integrated platforms become huge and go into multi-million lines of code, the engineering problem becomes incredibly difficult to scale. The question is what kind of programming culture do you want to put in place using this componentization approach.

BN:So has Microsoft put itself at a disadvantage?

Heintzman: Microsoft has a unique challenge. In their position in marketplace and dealing with the compatibility issues, they have an enormously complex engineering problem that with every release just gets worse and worse and worse. At some point that system really breaks down and you get diminishing returns on your investment.

We kind of took a review and projected out where our various different platforms were going years ago and we projected that they were going to hit a knee in the curve that made them incredibly unproductive because of the engineering complexity. So we decided on this new development strategy and this new philosophy, and we turned to Community Source as one of the enabling technologies to allow us to obtain engineering efficiencies.

BN:One last question Doug: What is the future of the program?

Heintzman: You don't do these things over night. There are a lot of good things in the open source development process, the community development and the culture. There are also some deficiencies with that process and having some architectural discipline to make sure that you have the most efficient deployment of resource makes very good sense.

We are really trying to harvest the programming culture and practices that we have honed, developed and optimized over many, many years, along with some of these new techniques that we have learned through our exposure to the open source communities and have tailored to our purposes internally.

We will merge these things, and certain projects that have been done in highly traditional ways will adopt some of the practices. New projects that grew up on these new processes will also adapt to many of the traditional disciplines that we have matured over the years. We hope we end up with the appropriate balance of the best of both of these worlds.

← Previous Page | 1 | 2

Comments

View comments by with a score of at least

Typical IBM hyperbole. Is that Rational {Rose]? The same software that was a tremendous failure at Liberty Mutual years ago--generalized junk which could not be tailored to specific needs.

Score: 0

|

I'm surprised, and tentatively hopeful, to hear that Rational is taking a lead in using this model. We're a ClearCase shop and it's been very tough to adopt open source techniques with ClearCase. I'd be happy to discuss this with the right folks. -- Ken.MacLeod@RadiSys.com

Score: 0

|

My experience with generalized code development is that the first version of the code is not very generalized. Even with the best of intentions the initial developers will produce code that is overly specific to its first usage. Generally, the second time that the code is used it will be rewritten to be more truly general purpose code.

------------------
Steve Stites

Score: 0

|

A few points of clarification. I want to draw a distinction between open source code and open source development procedures. IBM is a very active participant and contributor to a great many open source communities. We believe that open source and the standards that it promotes and the collaborative innovation it represents is an important and significant phenomenon and we are executing a strategy to leverage open source code to deliver value to our customers. The point I was trying to make in this article is somewhat different. Many of the code lines that IBM and other commercial software companies have been working on have become very large and difficult to manage and represent significant engineering challenges. We made a strategic decision a number of years ago to move to a much more componentized/modularized group of code bases. It is our belief that this will allow us to deliver more flexible, consistent, and cost effective solutions to the market in a more timely manner. To accomplish this goal, and coordinate the many programming resources that we have around the globe we have been adopting many of the practices, procedures and tools from the open source community. In this case I am not referring to leveraging open source code but instead open source culture which we have adopted internally and refer to as community source.

Score: 0

|

Thanks for the clarification Doug. The approach IBM is taking is a very smart one (in my opinion), and perhaps Microsoft can learn a thing or two about learning from the open source approach - even if the code is only open internally.

Score: 0

|

"IBM, will liberate the creativity of programmers, drive efficiency, and bring products to the marketplace at a faster rate then was previously possible - avoiding a top down approach that IBM says could make Microsoft's Longhorn obsolete upon arrival."

IBM making another OS or something? Longhorn obsolete, very funny.

Score: 0

|

As if Appple was ever a a threat to IBM? PLease. They have goverment contracts, they are going to be here for a minute.

Score: 0

|

IBM is not dying. Apple may have been a blow to them, but they have several other deals for their proccessor (maily gaming consoles). IBM also has several other businesses such as consumer computers & servers, software and storage. They also have massive R&D which although not directly profitiable, it may lead to something good in the end.

just a note: Betanews is starting to have a lot of articles with misleading titles. The title makes it seems like IBM is using Open Source or going Open Source. They are just using techniques used in the Open Source arena.

Score: 0

|

I wouldn't say IBM was dying, but it certainly has started to deconstruct itself. A reliance on open source may be technically wise, but from a profitability standpoint it will probably be a hard sell to investors. Then again, I suspect we will find IBM's version of "open source" is somewhat different than the normal definition of same. Imposing layers of architecture to stifle creativity is a more likely outcome than actually leveraging the value of an open source community.

Score: 0

|

Read the interview all the way and you'll see the plan is to leverage open source internally - it doesn't necessarily mean IBM will start to rely on the masses to program what it sells, which is what you're thinking. IBM is taking the areas of open source that have worked well and applying them to its current methodologies. It's very possible many of these components only remain open source within IBM as a corporation.

Score: 0

|

Actually, Nate, my post was not as clear as I intended, but I was referring only to IBM's internal process there.

What I was getting at is that given IBM's bulky methodology (not always a bad thing), a lot of the open source qualities that are beneficial will probably be lost in translation. It is hard to be creative in an architectural framework that is by nature heavy, and within the corporation I am doubtful that they can both maintain their architectures effectively and at the same time benefit from open source to any real degree. One of the main reasons open source approaches work is that they tend to place creativity in high regard.

I'm doubtful that IBM can make this transition without deconstructing its business from the point of view of savvy shareholders.

Score: 0

|

In response to nate: cause school1012 is an idiot who likes making dumb comments that he thinks will make him look smart.

Score: 0

|

IBM is a company that is dieing. Just let them die, open source is the best way to do so. Open Source has not helped IBM nor Novel.

Score: 0

|

IBM has a market cap of $121 billion, just signed deals to get its chips into both of the major next generation gaming consoles, and recently purchased a number of leading enterprise software vendors.

Can you please explain why you think IBM is dying?

Score: 0

|

Because he knows everything.

My guess, he's somewhere between 11 and 16.

Score: 0

|

He's actually 13 going on 30 !!! GO BACK... NOW !!!

Score: 0

|

ahhhhhhhh.... "dieing". One of the few words that can prove illiteracy. school1012, you are an illiterate.

Score: 0

|

I got into linux at the age of 11 and I am currently 15. I compile my own kernel and can expertly modify any configuration. I understand the opensource model perfectly. Age doesn't matter in stupidity. Please do not tarnish the future. 11-16 year olds are linux's future. Some of us "kids" will be the next Alan Cox or Linus Torvolds. Some will help take KDE, X11, and Gnome to a level hopefully beyond Microsft. Some linux kids will take the kernel farther than many could ever dream of. Some us will grow up and be the next Novell and IBM. Some will have businesses and run computers at schools. These areas need linux badly and these kids will take it there. Yes, some are kids who hack all the time and damage other teens' reputations.

( I am not trying to start a war, just inform the general public)

The more documentation and How-Tos out there will help me and other teens of "the next generation" surpass others before us. We need you as linux users and coders to help pass on your knowledge. Please use screen shots and other visual aids to help make your howtos user friendly.

Score: 0

|

mmcm4,

I have met many young people like yourself. You (and others like you) are one of the reasons that I spend so much time and energy supporting the Free and Open Source Community.

Thanks for speaking up.

maddog

Score: 0

|

Bing gets geekier with new Wolfram Alpha integration

Microsoft's Bing is now teamed up with Wolfram Alpha for computational search results.

HP to acquire 3Com for $2.7 B in cash, focus on China

A long and uncertain comeback trail comes to an end for the one-time network equipment giant.

Universities reject Kindle DX as a textbook replacement

Two universities running Kindle DX pilot programs have rejected the device.

Snow Leopard and Windows 7 still can't crack the netbook problem

Apple has killed Atom support in OS X 10.6.2 and Windows 7 Starter Edition is stripped of "basic" functionality.

Facebook for iPhone developer goes from Apple supporter to 'I quit!' in 3 months

Fed up with Apple's App Store policies, the developer of Facebook for iPhone has bailed on the iPhone.

Bing vs. Google rematch on video search

After Microsoft folds some old MSN Video features back into Bing, do they add to the search engine's functionality or take away?

New EU telecoms framework mandates user consent before getting cookies

Do you want a cookie? No. Do you want a cookie? No. Do you want a cookie? No. Do you want...Are you annoyed yet? That's a preview of 2011.

The Samsung Intrepid: A nice phone, if you can accept Windows Mobile

Samsung appears to have built solid enough hardware, but it's the software that seems uncomfortable and unintuitive.

A real beta process at work: Mozilla fires up Firefox 3.6 Beta 2

In the clearest sign yet that public input really does help the development process, a flurry of bug detections provoked Mozilla to release Beta 2 of the next Firefox.

Kindle for PC opens in beta, underwhelms

Amazon has opened the beta of Kindle for PC, a companion to the Kindle, but little else.

European ministers approve watered-down 'neutral net' language

The latest provision in the EU's telecoms regulatory framework would let businesses cancel individuals' Internet access, if they go to court first.