Planet SCM

The international Certified Configuration Manager association (iNTCCM) is founded in March 2006, with the objective of setting the standard for professionals in configuration management. Recently, I joined the iNTCCM association as a member.

The last few weeks I have been more involved in hardware CM, or more specifically hardware CM for IC (chip) development. The design process for a chip if very similar to the design process of software.
To design an analogue chip, you first make a model the electronic circuits using an modeling tool where you connect signal and power lines between standard components. This model goes into a simulator where you "test" your design. For a digital chip, modeling is even more similar to software development: you write code using a programming language. A code checker is then able to calculate the output bits based on the input bits. So in both cases, you create source files to describe the system behavior and you run a build to check the code or model, and generate files needed to feed the (chip) fab. The different between hardware (chips) and software is what happens with the build result.

So I would expect that CM for the design process of chips would be similar or even identical to the design process of software. Well, in my present experience it is not! CM for chip design is as we did it for software in the 80-ies of the previous century, which is 25 years ago. There is hardly any parallel development and the only controlled versions are the current versions (/main/latest) and the baselines (tagged versions).

From a software perspective, absence of parallel development is hard to imagine. How come that chip design can do without, and still develop multiple product variants at the same time? The answer is: reusing block designs. A chip consists of a combination of (more or less) standard blocks. These can be high level blocks (e.g. an ADC, or a Bluetooth controller) or low level blocks (e.g. a transistor, an amplifier or a FIFO). These blocks are used to compose a chip design. The challenge is to combine as many blocks as possible, on a chip as small as possible, with the best performance as possible (e.g. low noice levels, minimum power dissipation, largest temperature range, etc.). The cost of a chip is determined by the size, but the market value is determined by the functionality (maximum number of blocks) and the quality (best performance).
One of the biggest challenges of chip design is how to place and connect the blocks and get still get the chip within spec. This is a 3-D problem - because a chip consists of multiple layers - and almost all changes in the design has impact on the complete chip. Therefore, you cannot change one part of the chip in one team, which another team works on another part, and therefore there is not much parallel development.

Development of those reusable blocks (e.g. making a smaller ADC using less power and with better performance) is done separately from the chip development. These blocks are called IP blocks. The designers of an IP block don't really care about placement and connection of the block on the chip; they care about the internals of the block, making the design possible. The only reason they care about other blocks on the (test) chips is interference: for example, what is the effect if we place the ADC next to a power management block?
Since almost all changes within an IP block have impact on the whole block, there is little use to designing one part of the block in parallel with another part of the same block. And if that is feasible at all, it is more useful to make them separate IP blocks which can be used (and designed) independently.

Now let's go back to configuration management. We have seen that chip design and IP block design have little use for parallel development. So CM does not need branching or streaming. If there is a need for (for instance) product variants, it is easy enough to just design them as separate product types, reusing the ideas of one product type for the other one. It may even be sensible to design them sequentially where the first product type takes several months to design and the next one only a few weeks after it.
If we look at software development, we may notice that the use of standard blocks is not "common practice". Designing a software system involves (1) designing the software blocks and (2) integrating them into a system. So what chip design does in two separated processes (usually also separate projects), is combined in a software project. This is possible because in software you have infinitely more freedom in interconnecting "blocks" and infinitely higher speed of changing the characteristics of a blocks. If a software designer has a block that "almost" fits in a system, it is often cheaper to change to block itself than to change the system around it to make it fit. In a modular design changing the block internally has less impact on the whole system than changing the environment of the block. And therefore, software blocks are often developed in separate teams that work simultaneously, using a (more a less) fixed interconnection called the interface architecture. Variants of blocks can - and do - exist simultaneously within the same system, where at compile-time, link-time, install-time or run-time is it decided which variant of a block is actually used.

The simultaneous and concurrent existence and design of software blocks, and the dynamics of integrating those blocks into a system is so much more complex than for chip design, that it requires a lot more complex configuration management approach. The SCM approach requires parallel development in branches and streams, both on block level and system level. It also requires a lot more flexibility of dealing with integration scenarios, variants and circumstantial differences. And last but not least, chip designs require an (expensive) manufacturing process that is very expensive to change once it is established, while software is the final product after the build. Consequently, the dynamics of change in software systems is much higher then for chip designs.

In summary I would say that although the design processes for chip design and software design look very similar, the configuration management requirements are of completely different magnitudes.

In the Netherlands, there is a new configuration management initiative called CM Platform [no link to a website yet]. The initiators are from Thales and KPN. Although it is still in its early stages of exploration, there is a clear need for more and better collaboration and sharing of the configuration management knowledge.

Other initiatives in the Netherlands are:

• Dutch Rational User Group (HRUG), initiated by Bosch Security Systems and TOPIC Embedded Systems
  
• CM workshop organized by TOPIC Embedded Systems and Philips Applied Technologies

In a podcast on IBM developerWorks, Dave Michell explains about Software as a Service (Saas):

the customer does not take ownership of the software, but they're going to rent this solution in a subscription model, and it's delivered remotely. And that's very different from the ASP model. In the ASP model, or the application hosting model, what the customer is doing is they're buying the software.
[...]

In the Software as a Service model, again, the customer does not own the software. They subscribe to an offering and they buy that offering in a subscription model per user per month. So there's no software purchase to be made.

And the other key point here is in the Software as a Service model, the customer really has no say over what the infrastructure is. They don't get to decide what the hardware is or the middleware or the database — and more to the point, they don't really care.

So in the SaaS model, the customer says "I want to buy that application functionality. I'm willing to pay this much per user per month," usually is their typical model. "And the service-level agreement meets my needs. So I'm going to acquire that application that way."

Considering a configuration management system, such as ClearCase, many customers struggle with configuring the system for optimal performance, setting up and configuring servers, network and client parameters. In many cases, the users don't really care whether it is ClearCase, Synergy or Subversion behind the scenes, as long as that can do version control, baselining (or labeling or tagging), parallel development (or branching or streaming), delivery, rebasing, releasing, status promotion, structuring, setting attributes, and all the other CM stuff. Users don't care about sufficient licenses, disk space, replication across sites and other IT issues; they just want the system to be available and to work properly to their needs.

So, the CM system seems a perfect candidate to be deployed as Software as a Service. Even more, if the CM service is offered as a Web 2.0 application, the internet will be the platform allowing local and global accessibility.

If in addition there would be a standard set of CM service features, the CM back-end could be supported by just any CM vendor that complies with this standard. The front-end could even be of a different vendor or proprietary to the customer, e.g. as a dedicated integration with other systems. Customers could then focus more on the tool-vendor that suites best with their organizational and IT needs, and users could better focus on dealing with true configuration management issues, rather than being hampered by infrastructure and IT issues.

Why wouldn't a product like ClearCase be offered an SaaS model, if IBM is such a promotor of SaaS?

There are many definitions of configuration management, most of them containing four basic elements: identification, control, status accounting and auditing. Although formally correct, the translation into practice is far from easy for many organizations. I wonder why this is...

Let's have a look at a car for a minute. From a user perspective, a car is a way of transportation from point A to point B. From a technical perspective, a car is an system where you can put people and luggage in and that allows a controlled way of acceleration, deceleration and steering. Given the technical view, there is no way to define that a car brings you from any particular point to another point.
Now let's go back to configuration management. In fact, identification, control, status accounting and auditing are the technical view on CM. But what is the user view on CM? Why do we need CM?

I think that the main and possibly only reason why we need CM is that we need to communicate and share. The effectiveness of communication depends on the integrity of the information we communicate and the effectiveness of sharing depends on the integrity of the objects we share (work products). If the information and work products would be static, a (simple) library or stocking system would be enough. But in practice the information and work products are not static; they are evolving in a planned way (if the planning is followed). The integrity (or validity) of the information and work products also constantly changes.

Now because the information and work products are constantly changing, we need some guidance in determining what is correct and what is not. That's why we need concepts like streaming or branching to facilitate parallel development, we need the concept of tagging, labeling or baselining to determine the status of a particular configuration, we need the concept of promotion to visualize the state of evolution.

Communication of information, sharing of work products, and assuring the integrity of the whole while it is constantly changing, is the main challenge of configuration management. Hopefully, it is easier to understand configuration management from this angle than from its formal definition.

"Not every change is an improvement, but an improvement is always a change"

Configuration management is about managing change, assuring the integrity of the business assets during their lifecycle. This implies that the work products of a project are protected against loss or damage. Every change is a risk of introducing bugs, damaging the existing functionality or degrading the system's performance. A good way to assure the integrity is to minimize the amount of change: if you don't change it, you can't break it.
This is precisely what a CCB (Change Control Board) is doing. Every change request is discussed and only when the business value of the change is high enough, the change is accepted for implementation. Other changes are rejected. Thus, the system will undergo minimum change and minimum risk of degradation. A positive side-effect is that this approach leads to minimizing costs and to minimizing time-to-market: if you don't change much, it does not cost must money or time.

On the other hand, especially in a consumer market of retail or professional products such as mobile phones, audio and video equipment, medical diagnostics and treatment systems, transportation or manufacturing, the competition is tough. The game here is to maximize the amount of functionality, to maximize the level of quality, but at the same time to minimize the costs and to minimize the time-to-market.

Now what should configuration management aim for: minimizing change or maximizing change?

Well, in fact the answer is quite simple: that's not up to configuration management. Regardless of the amount of change being high or low, configuration management should manage the work products and their changes, assuring their integrity. The decision to minimize or maximize change is a business decision. This may be different for different businesses, but also for different projects within the same business. It may even be different depending on the time of the year (e.g. Christmas season). But of course, the CM approach must be different in both cases, so it must be aware of the business situation.

This implies that there is no single best CM approach; there is no one-fits-all solution. To find a common solution, we need to focus on the one thing that both have in common: change will happen! Any change may be an improvement or a degradation (or both). We must avoid that changes that we don't want are slipping through to the customer, and we must assure that changes we do want are actually brought to the customer, with quality and in time.
This requires at least a proper level of visibility, which bring us to the four pillars of CM:

• What changes do we want, and what changes don't we want? (Configuration identification)
• What solutions do we have, and which ones don't we have? (Configuration control)
  
• At what stage is every change? (Configuration Status Accounting)
• Are we sure about that? (Configuration auditing)

The most important job of configuration management is to achieve a maximum speed for propagating desired changes, for instance storage when the change is available, promotion when the promotion criteria (e.g. quality criteria) are met, distribution to the teams for the next processing step (e.g. verification) and communication for visibility (e.g. to enable project monitoring and control). But at the same time configuration management is to detect undesired changes as soon as possible (preferably before someone is working on them), and stop the propagation of solutions of undesired changes when they slip through or when we change our minds about desired and undesired changes.

So it is always a balance between speed and control, between volume and quality, between maximizing and minimizing change. Moreover, it is a constantly change balance too, which makes the profession of Configuration Management a challenging and interesting one!

Last week I was one of the speakers at the 4th CM workshop organized by TOPIC Embedded Systems and Philips Applied Technologies. I spoke about how Web 2.0 technology will have its (change) effect on configuration management. As a speaker, we received a nice presents, a book by John Kotter and Enter Rathgeber:

Our Iceberg is Melting

Most of you know how I hate reading books. So at first I forgot about the book, leaving it in my suitcase for over a week. Today I started reading it, and finishing it! And I love it!
It reminds me of the book Peopleware by DeMarco and Lister which I also read in a single breath, and loved. Peopleware and Our Iceberg is Melting are the only books I read in the last decade that I absolutely love - and finished.

Now you probably wonder what is so special about these books. Well, first of all you should read them yourself. Both books are about people, in case of the Iceberg in the form of a fable with penguins. I had no difficulty whatsoever to project the story on a real life situation at work and real people from work.
Quite often I encounter people like Fred, Alice, the Professor, Buddy and of course NoNo. Unfortunately, I hardly ever encounter talents like Louis, a respected senior manager who knows how to manage how to manage people.

Anyway, take a look at The Iceberg is Melting, it is worth it.

Jens Norin has written een article about Lean configuration management. There is a lot that I disagree with, but it also gives a lot of things to think about.

During a Configuration Management workshop in Mierlo, a little town near Eindhoven in the Netherlands, I had an interesting discussion about CM certification. The concept of certification is simple: if you comply with an amount of selection criteria, you get your certificate.

The discussion was what the added value is of CM certification. In my view, the may be - at least - two reasons why you need certification:

• The distinction between professional and amateurs is difficult to make
• The industry is using certification as a selection method

Currently, there is no common, generally accepted, understanding of what a configuration manager is. So if a certification institute defines certain rules and criteria for certification, it is arbitrary. There will be lots of good CM-ers that will fail and bad CM-ers that will pass, depending on whether your knowledge and work experience complies with those criteria. The result will be that within the crowd of certified CM-ers there will be good and bad ones, and within the crowd of non-certified CM-ers too.
In addition, the community of CM-ers is a small world. Bad CM-ers are easy to pick, and if someone is completely unknown within the CM community, it might be a reason to some extra screening when he or she applies for an assignment as a configuration manager.

So my argument was that the added value of certification of individuals within the CM world is too small. This does not justify the investments to set up a certification institution, not does it justify the individual investments to get certified.

I think CM certification is nonsense. It is useless to strive for a certificate unless it clearly boosts my career if I do, or damage my career if I don't.

Some links:
Institute of Configuration Management
International Certified Configuration Manager

Recently, I have created a Configuration Management space with an underlying Configuration Management wiki on IBM developerWorks. Initial focus will be on IBM products (ClearCase, ClearQuest and UCM) and on CM concepts and practices. Later, it is my intention to extend it to other CM products too. The space and wiki are an extension to the existing discussion forums on dW.

The content is still in its initial stages, so bear with me to let it grow. For that purpose I am still looking for co-editors who are willing to put more content to it. Although wiki's are intended for open collaboration, I want to be careful with making it completely open for modification to the world to assure proper structure and quality.

For new events or other news and changes on the CM space portal, feel free to contact me.

Recently, I attended the IBM Rational Software Development Conference (RSDC2007) in Orlando, Florida. Apart from the typical presentations about the products, there was a lot of attention for Organizationally Distributed Development, or Globally Distributed Development (GDD). ODD/GDD is about crossing the organizational boundaries, which many people find scary.

At the RSDC they showed a prototype of a product called Team Concert, a tool to support collaboration between people to build software. It is based on the Jazz technology, which is

Jazz is a joint project between IBM Rational and IBM Research to build a scalable, extensible team collaboration platform for seamlessly integrating tasks across the software lifecycle.

New and exciting is that the Team Concert product is developed as a commercial product in an open source manner - see the press annoucement. People from the global community can join in to the development team. Thus, the project is deploying the potential of the global community.

Another exiting thing is that the Team Concert is a collaboration product, creating full transparancy and integration of technical and control information. And since this is a collaboration project, they use (pre-released) versions of Team Concert to support the development (globally distributed). So they are using the product that they are making.

But this is not the only thing that illustrates that IBM is taking collaboration technology seriously. On IBM developerWorks they have introduced spaces and wikis, next to the discussion forums that have existed for many years. For company communication this is not new, but what is new is that they also allow non-IBM people from the user community to create a space. The Configuration Management space is the first one.

Coincidently, at NXP Semiconductors (formerly Philips Semiconductors) there is an initiative to create a Common Collaborative Core (C3), a set of applications (e.g. wikis, forums, website, IM) to support the - internal - community with collaboration, sharing and building knowledge and competence. NXP happens to be the customer that I am assigned to currently.

Apparently, both NXP and IBM recognize the same thing - unleash the power of the mass instead of trying to build competences with the (small) recruited crew.

Transparent collaboration across organizational and geographic boundaries is a necessity to survive the ever growing technological and organizational complexity.

Today I came across a little video about Don't be an asshole by Bob Sutton explaining the importance of people learning to be indifferent at times to things that cannot be changed, and ignore those things. It struck me because I tend to get irritated or frustrated about things that I don't like and I don't seem to be able to change.

More on the Don't be an asshole rule on his weblog.

In his weblog article Strong Opinions, Weakly Held, Bob Sutton writes about a psychological paradox that to move forward in knowledge and wisdom, we need

• A strong opinion to assure that we are inspired to find the best arguments to support the opinion, but also
• Not be attached to the opinion to avoid getting blind for evidence against it

At first this felt weird. At work, I like to either take a firm standpoint for my opinion or take no standpoint at all and listen to others. It often confuses people when I play Devil's Advocate and stick to my opinion for the sake of the discussion just to challenge it. In other situations, I seem unable to make up my mind and form a opinion.

In the end, it's all a matter of communication...

To me it is highly arguable whether software creation is an art. In the early days, the challenge of software was to find the most efficient or the most effective algorithms to crack a mathematical problem. Nowadays, most of the creative art has made place for crafted engineering: integrating existing concepts and technologies into new systems.

Nerds and wizkids, able to work in isolation and inventing the most unexpected solutions have been replaced by architects and engineers working together in efficient social structures to build and integrate predictable results in a predictable way. Beauty and cleverness have been replaced by control and productivity.

Did all the fun disappear then? No, it has changed. Instead of being a soloistic challenge (like athletics), software development has become a team challenge (like football). And instead of breaking world records time after time, the objective is to play in the champions league beating the competition. It requires completely different strategies and tactics. To me, being in a winning team is more fun than being better than everybody else.

Last week I had a meeting with a customer that is in the process of implementing a new configuration management system. We had a tough discussion about building in security measure to protect the system from malpractice of software developers.

Surprisingly, they were very much concerned about deliberate, intentional misbehaviour of their software developers. For them it was even more an issue than intentional, accidental faults. The argument was that intentional misbehaviour may be very difficult to detect because the engineers try to be extremely smart on that. The risk they are concerned about is that after delivery of those changes, the integrator may not discover problems in the system through the build and smoke test. So the problem is propagated to system level and may be even to the customer.

What they try to do is maximise the security mechanisms to make it is impossible for developers to bypass it. In my opinion, it is absolutely foolish to even try it. First, you will never outsmart those saboteurs and try it takes extreme costs and efforts. A better approach is to build in logging and detection mechanisms. Then, if someone runs in an unintentional malpractice you can build in protection mechanisms, but if someone deliverately hacks the system he should be warned first and fired next regardless of his other competences and knowledge level. It is better to spend the money on finding the right people and on try to secure the system against those terrorists.

What do you think we should do about it?

It has been a hard week last week. Being involved in a large process change process for a software organization, I have been foreseeing a great deal of problems that are now materializing in reality. We are on the go for several months now, preparing new process definitions, changing software architecture, changing software archive structure, changing build processes, changing supportive tools and scripts, training people, etcetera.

One of the issues I have expected is that the people are just continuing their old habits, their old ways of working, in the new environment. But the new environment has far more flexibility on one side, and more control mechanisms on the other side. So what happens is that the new flexibility is not used, or it is used in an inefficient way and the new control mechanisms are more and more getting in the way. The result is that process are becoming less efficient and less effective, even though the objective was (and is) to gain development speed, to gain management ability to steer in the right direction and to gain visibility of the process.

Now I am wondering: Should I have taken action earlier? In fact I did, but nobody listened. Why did nobody listen? Because they did not have a problem yet. One of the hardest things for a (change) consultant is to change something that is not a problem for those who have to change. They simply are not open for change, even though they may want to. Resistence is not a matter of attitude (although it often is), but it is a natural byproduct of being inexperienced in changing. Even worse, many of the quality programs aim for reproducibility, predictability and repeatability, not for change in spite of the term Process Improvement".

But now they know! Now they are confronted with many different things that do actually go wrong, and that have to be corrected immediately because the project is suffering delays. Of course they blame the new process! And they know it should be address right now, a solution must be ready by yesterday. In other words, time is right for change now, although we do not really have enough time and opportunity to make the change because the project is delaying by every day. And in the past, when we had the opportunity to change things, it was simply impossible to do so.

For many, many years I have fought against these windmills, trying to change people, processes and the organization to prevent them from running into problems, before the time was right. It was a hard and endless struggle and in many occasions my evaluation and my carreer suffered from it. Now I have learned that knowing the best solution is a handicap sometimes, since it is not always the best solution at the time. I have learned not to try to convince people too hard to do something about it, but just to prepare them that it will go wrong. And then, I have learned to be patient until time is right. Act too early, and you will spoil the opportunity to make the difference.

Next Thursday, November 18th, there will be an event organised by Bits&Chips; about managing complex projects. Sure, it will be mostly in Dutch but it will be interesting anyway.

The challenge for software projects is to increase the productivity significantly; not by 10% or 50% but with 10 to 100 times or more. Will current process improvement initiatives achieve that? I doubt it. In my opinion, we need to reduce the size of projects instead of increasing it, we need to replace manual work and communication by machine work.

Making software, making software development processes and making music have a lot in common. To me it gives excitement and joy to do it, and it helps me relax.

On december 12, 2004 I played at a small concert in the public library of Eindhoven. They also made some photographs.

Let me tell you about a very nasty disease I have:

• I hate reading books
• I hate spend money on books I don't read

The latest book I read completely was Peopleware by DeMarco, about 6 years ago. It changed my way of thinking quite considerably. I got In Search of Excellence by Peters as a present, but was unable to finish it. Same for Gödel, Escher, Bach which is one of the most boring books I have ever looked into. I wish I had read the book A Mythical Manmonth by Brooks. I read Alice in Wonderland on the little screen of my Palm V, but forgot about most of it.

My wife and kids love to read. Many of the people I admire are addictive readers and I wonder where they find the time to do it. My problem is that I can't read while driving a car, I can't read while working as a consultant for a customer, I can't read while I am playing the piano, I can't read while eating and I can't read while reading or writing on the internet. And I can't read in airplanes on business travels because I do not travel enough and when I do I am too interested in listening to the cockpit radio (I think only United Airlines has an audio channel with the cockpit radio) or preparing for the meetings to come.

If any of you has any idea how to cure my disease, then please tell me... thanks in advance.

Marc Siegel, IBM Rational's community manager, has introduced the IBM Rational Discussion Facilitators in an article on IBM DeveloperWorks.

Many customers may have noticed that since Rational has been taken over by IBM some years ago, the communication between the customer (including the responsible IT and Development managers, and the actual users working with the tools) and IBM Rational (including sales and support) has changed considerably, not for the better in most cases. User to User communication is an emerging (support) strategy for customers, and it has proven its effectiveness for open-source software for many years.
However, User-to-ToolVendor communication has been difficult in the past and will become more difficult in the future due to cost-saving on service organizations. For that reason, I think that IBM has made a smart move to institutionalise the use of Discussion Facilitators for direct communication with Product Management, as they are not on the payroll of IBM (i.e. they are ordinary customers themselves) while they provide a bilateral service to the user community.

The IBM Rational discussion forums are the discussion forums for users of Rational products. The IBM website, including the DeveloperWorks website, are rather difficult to navigate through so you'd best go to it through the direct link.