A Django site.
August 6, 2008
» Four Rules for Simple Codelines

Some of you may be aware of Kent Beck's Four Rules of Simple Code that state simple code:

  1. Correctly runs (and passes) all the tests
  2. Contains no duplication (OnceAndOnlyOnce and The DRY Principle)
  3. Clearly expresses all the ideas/intentions we needed to express (reveals all intent and intends all it reveals)
  4. Minimizes the number of classes and methods (has no superfluous parts)
(I've seen some boil this down into some of the same rules for writing clear prose: correct, consistent, clear, and concise.)

Lately I've been noticing some parallels to the above and rules for what I would call "simple codelines" and I think there may be a similar way of expressing them...

Simple codelines:
  1. Correctly build, run (and pass) all the tests
  2. Contain no duplicate work/work-products
  3. Transparently contain all the changes we needed to make (and none of the ones we didn't)
  4. Minimize the number and length of sub-branches and unsynchronized work/changes

To elaborate further...

Correctly build, run (and pass) all the tests

This is of course the most obvious and basic of necessities for any codeline. If the codeline (or the "build") is broken, then integration is basically blocked, and starting new work/changes for the codeline is hindered.

Contains no duplicate work/products

The same work and work-products should be done OnceAndOnlyOnce! Sometimes effort is spent more than once to introduce the same change/functionality. This is sometimes because of miscoordination, or simply lack of realization that what two different developers were working on required each of them doing some of the same things (and perhaps should have been accomplished in smaller chunks).

Other times, rather than modify or refactor a common file, some will simply copy-and-paste the contents of one or more files (or directories/folders) because they don't want to have to worry about reconciling what would otherwise be merges of concurrent changes to the common files.

This is akin to neglecting to refactor at the "physical" level (of files and folders) as opposed to the "logical" level of classes and methods. It adds more complexity and (over time) inconsistency to the set of artifacts and versions that make up the codeline, and also eventually adds to the time it takes to merge, build, and test any integrated changes.

If content is being added to the codeline, we want that content to have to be added only once, without any duplicate or redundant human effort.

Transparently contains all the changes we needed to make (and none of the ones we didn't)

The above is sometimes the cause of much undesirable additional effort that is imposed for the sake of attaining traceability and ensuring process compliance/enforcement. Here, I mean to focus on the ends rather than the means, and I say transparency rather than traceability for that very reason.

If people are working in a task-based and test-driven manner, it should be simple to report what changes have been made since a previous commit and that only intended tasks were worked-on and integrated.

If a codeline is truly simple, then it should be very simple and easy to reveal all the changes that went into it without adding a lot of overhead and constraints to development. It should be easy to tell which changes/tasks have been integrated and what functionality and tests they correspond to. One very simple and basic means of tying checkins (or "commits") to backlog-tasks and their tests can be found here; others are mentioned in this article.

Minimizes the number and length of sub-branches and unsynchronized work/changes

Branching can be a boon when used properly and miserly. It can also add a heck of a lot of complexity and redundancy for maintaining two or more evolving variants of the project. The additional effort to track and merge and build many of the same fixes and enhancements in multiple configurations can be staggering.

Sometimes such branches are useful or even necessary (and can help with what Lean calls nested synchronization and harmonic cadences). But they should be as few and as short-lived as possible, preferably living no longer than the time it takes to complete a fine-grained task or to integrate several fine-grained tasks.

Even when there are no sub-codelines of a branch, there can still be un-integrated (unsynchronized) work-in-progress in the form of long-lived or large-grained tasks with changes that have not yet been checked-in or synced-up with the codeline. Keeping tasks short-lived and fine-grained (e.g., on the order of minutes & hours instead of hours & days) helps ensure the codeline is continuously integrated and synchronized with all the work that is taking place.

Another (possibly less obvious form) of unsynchronized work is when there is a discrepancy between the latest version of code checked-in to the codeline, and the latest version of code that constitutes the "last good build." Developer's lives are "simpler" when the latest version of the codeline (the "tip") is the version they need to use to base new work off of, and to update their existing workspace (a.k.a. "sandbox").

When the latest "good" version of the codeline is not the same (less recent) than the latest version, it can be less obvious to developers which version to use and become less likely that they use/select it correctly. Some use "floating tags" or "floating labels" for this purpose where they "move" the LAST_GOOD_BUILD tag from its previous set of versions to the current set of versions for a newly passed/promoted build. Sometimes the developers always use this "tag" and never use the "tip" (except when they have to merge their changes to the codeline of course).

Even with floating tags however, it is still simpler and more desirable when the last good version IS the latest version. Even if the latest version is known to be "broken", the lag between "latest" and "last good" version of a codeline can be a source of waste and complexity in the effort required to build, verify and promote a version to be "good" (and can introduce more complexity when having to merge to "latest" if your work has only been synchronized with "last good").

Plus, this lag-time often leads many a development shop to separate merging (and integration & test) responsibilities between development and so called integrators/build-meisters, where the best developers can attempt is to sync-up their work with the "last good build" and then "submit" that work to a manually initiated build rather than being directly responsible for ensuring the task is "done done" by being fully integrated and passing all its tests.

Such separation often leads to territorial disputes between roles and build/merge responsibilities. This in turn often leads to adversarial (rather than cooperative and collaborative) relationships and isolated, compartmentalized (rather than shared) knowledge for the execution and success of those responsibilities.

So there we have it! Four rules of simple codelines.

Simple Codelines should:
  1. Correctly build, run (and pass) all the tests
  2. Contain no duplicate work/work-products
  3. Transparently contain all the changes we needed to make (and none of the ones we didn't)
  4. Minimize the number and length of sub-branches and unsynchronized work/changes

Sometimes there are legitimate reasons why some of the rules need to be bent, and there are important SCM patterns to know about in order to do it successfully. But any time you do that, it makes your codeline less simple. So you want those scenarios to be few and far between, and to keep striving for the goal of simplicity. (Other SCM patterns, such as Mainline, can help you refactor your codelines/branches to be more simple.)

July 6, 2008
» Iterative and Incremental redefined redux

The agile community has written much about this in the past year or so:

Apologies in advance for being a "stick in the mud" on this one - I'm not particularly happy with the definitions so far. I searched around some more on the WWW and came across one I like a lot that I think better meets our needs.

It is from the paper What is Iterative Development? (part 1), by Ian Spence and Kurt Bittner,
Iterative and Incremental Development:
A style of development that involves the iterative application of a set of activities to evaluate a set of assertions, resolve a set of risks, accomplish a set of development objectives, and incrementally produce and refine an effective solution:
  • It is iterative in that it involves the successive refinement of the understanding of the problem, the solution's definition, and the solution's implementation by the repetitive application of the core development activities.2

  • It is incremental in that each pass through the iterative cycle grows the understanding of the problem and the capability offered by the solution.

  • Several or more applications of the iterative cycle are sequentially arranged to compose a project.
Sadly, development can be iterative without being incremental. For example, the activities can be applied over and over again in an iterative fashion without growing the understanding of the problem or the extent of the solution, in effect leaving the project where it was before the iteration started.

It can also be incremental without being truly iterative. For example, the development of a large solution can be broken up into a number of increments without the repetitive application of the core development activities.

To be truly effective the development must be both iterative and incremental. The need for iterative and incremental development arises out of the need to predictably deliver results in an uncertain world. Since we cannot wish the uncertainty away, we need a technique to master it. Iterative and incremental development provides us with a technique that enables us to master this uncertainty, or at least to systematically bring it sufficiently under control to achieve our desired results.

I like that this definition separated iterative from incremental and then defines them together. I would summarize it as follows (but I like the above better, even if it is longer):
Iterative development is the cyclical process of repeating a set of development activities to progressively elaborate and refine a complete solution. The “unit” of iterative development is an “iteration”, which represents one complete cycle through the set of activities.

Incremental development
is the process of developing and integrating the parts of a system in multiple stages, where each stage implements a working, executable subset of the final system. The “unit” of incremental development is an “increment”, which represents the executable subset of the system resulting form a particular stage

Iterative and Incremental development
is
therefore ...
the application of an iterative development lifecycle to successively develop and refine working, executable subsets (increments) of a solution that evolves incrementally (from iteration to iteration) into the final product.
  • Each iteration successively elaborates and refines the understanding of the problem, and of the solution's definition & implementation by learning and adapting to feedback from the previous iterations of the core development lifecycle (analysis, design, implementation & test).
  • Each increment successively elaborates and refines the capability offered by the solution in the form of tangible working results that can be demonstrated to stakeholders for evaluation.
An Agile Iteration is a planned, time-boxed interval (typically measured in weeks) whose output is a working result that can be demonstrated to stakeholders:
  • Agile Iterations focus the whole team on collaborating and communicating effectively for the purpose of rapidly delivering incremental value to stakeholders in a predictable fashion.
  • After each iteration, the resulting feedback and data can be examined, and project scope & priorities can be re-evaluated to adapt the project's overall performance and optimize its return-on-investment
So in addition to the non-agile-specific definitions above, we see that Agile iterations are adaptive, in that they use the previous results and feedback to learn, adjust and recalibrate for the next iteration. And Agile increments are tangible, in that they can be executed and made accessible to stakeholders for demonstration and evaluation.

That's my story and I'm sticking to it!

» Traceability Matrix in an Agile Project

InfoQ.com summarized an email-list discussion thread on the subject of using a Traceability Matrix in an Agile Project.

I contributed quite a lot to the thread, and InfoQ apparently included many of the key things I said along with the related URLs to articles I've written. (Thanks guys!)

» The Laws of Codeline (Thermo)Dynamics

Some of the discussion with my co-authors on our May 2008 CM Journal article on Agile Release Management spurred some additional thoughts by me that I hope to refine and work into a subsequent article later this year.

Release Management is about so much more than just the code/codeline (and it being "shippable") it's not even funny. Some other articles to reference and mention some key points from are:

Kevin Lee has written some GREAT stuff on Release Management that relates to Agile. The best is from the first and last chapters of his book on "The Java™ Developer's Guide to Accelerating and Automating the Build Process" but bits of pieces of it can also be found at:

ANY discussion about Release Management also needs to acknowledge that there is no single "the codeline", not just because I may have different codelines (Development-Line plus Release-Line) working toward the same product-release, but ESPECIALLY because no matter how Agile you are, the reality is that you will typically need to support MULTIPLE releases at the same time (at the very least the latest released version and the current under development version, but often even Agile projects need to support more than one release in the field)

So, when dealing with multiple release-line, and any "active development lines" for each of those, and the overall mainline, we really should say something overall about how to manage this "big picture" of all these codelines across multiple releases and iterations:
  • What is the relationship between development line, release-line and release-prep codeline?
  • How do the above three "lines" relate to "mainline"
  • What is the relationship between the different release-lines for the different supported releases
  • What is the overall relationship between the mainline and the release-lines (and if the mainline is also a release-line, which release is it?)
The above questions and the ability to give some big picture "advice" on relating it all together (the stuff of pattern languages) is precisely where Laura Wingerd's writing on "channeling the flow of change" and her chapter on "How Software Evolves" fits in! It tells us
  • The overall Mainline model
  • The different types of codelines ("line" patterns), and what kinds of builds take place on each of them
  • The relationships of those to the mainline
  • When+Why to branch (and from which "types" of codelines)
  • When+Why to merge across codelines (as a general rule)

These are where Laura's rules for "the flow of change" apply. And her concept of "change flow" is very much applicable to the Lean/Agile concept of "flow of value". The Tofu scale and "change flow" rules/protocol have to do with order+flow of codeline policies across the entire branching structure when it comes to making decisions about stability -vs- speed. One codeline's policy might make a certain tradeoff, but it is the presence of multiple codelines and how they work together, and how their policies define the overall flow of change across codelines, that forms the "putting it all together" advice that is key to release management across multiple releases+codelines.

In some way's you could make an overall analogy to the Laws or Thermodynamics and the realities of codeline management. Software and codelines tend, over time, to grow more complex and, if unchecked,
"Entropy" (instability) quickly becomes the most dominating force to contend with in their maintenance. See

The "entropy" (instability) doesnt just happen within a codeline. It can actually get far more hideous when it happens across codelines via indiscriminate branching from, or merging to, other codelines. This is what happens when you don't respect the principles and rules of "change flow" (from Wingerd) which ultimately stem from the rules of smooth and steady (value-stream) flow from Lean.

The Laws of Thermodynamics are about energy, entropy, and enthalpy. In the case of release management and codelines ...
  • energy relates to effort & productivity
  • entropy relates to stability/quality versus complexity
  • enthalpy relates to "order" (i.e., in the sense of structure and architecture as Christopher Alexander uses the term "order"). It is the "inverse" of entropy.

We could call them them "Laws of Codeline Dynamics" :-)

Energy misspent degrades flow, creates waste, and hurts productivity/velocity. In traditional development, we often see "fixed scope" with resources and schedule having to vary in order to meet the "scope" constraint. IN Agile development we deliberately "flip" that triangle upside down (see the picture in the article at here under the title "The Biggest Change: Scope versus Schedule - Schedule Wins"). So we are fixing "resources" and "schedule" and allowing scope to vary.

This might be one way of viewing the law of conservation of energy. If we fix resources and time (and insist on "sustainable pace" or "40hr work week") then we're basically putting in the same amount of effort over that time-box, but the key difference is how much of that effort results in "giving off energy" in the form of waste ("heat" or "friction") versus how much of that energy directly adds value. Both "Value" and "Enthalpy" degrade or depreciate over time, and adding more energy (effort) doesnt necessarily mean value is increased.

To make sure that energy goes toward adding value (and minimizing waste) we need to focus on the flow of value, and hence the flow of change/efforts to create value (the latter is one reasonable definition of a "codeline" or a "workstream"). to ensure a smooth, steady, and regular/frequent flow, there are certain rules we need to impose and regulate stability within and across codelines to better manage all those releases.

Zeroth Law of Thermodynamics (from Wikipedia)
- If two thermodynamic systems are each in thermal equilibrium with a third, then they are in thermal equilibrium with each other.
Translation to codelines ... this law of "thermal equilibrium" is a law of "codeline equilibrium" of sorts. (Does this mean If two codelines are are "in equlibrium" with a third codeline, then they are "in sync"? and with each other? here "in sync" doesnt mean they have the same frequency, it means their is some synchronization pattern regarding their relative stability and velocity. In Lean Terms, this would refer to "nested synchronization" and "harmonic cadence"). This might imply the "mainline" rule/pattern or one of Wingerd's rules of change-flow.

First Law of Thermodynamics
- In any process, the total energy of the universe remains the same.
This is the statement of conservation of energy for a thermodynamic system. It refers to the two ways that a closed system transfers energy to and from its surroundings - by the process of heating (or cooling) and the process of mechanical work.

This relates to effort & changes expended resulting in the creation of value and/or the creation of waste. We have activities that add value (which we hope is development), activities that preserve value (which is what much of SCM attempts do, given that it doesnt directly create the changes, but tries to ensure that changes happen and are built/integrated with minimal loss of energy/productivity/quality), and then we have activities (or portions of activities) that create waste (and increase entropy rather than preserving or increasing enthalpy/order)

Second Law of Thermodynamics
- In any isolated system that is not in equilibrium, entropy will increase over time
So this is the law of increasing instability/complexity/disorder. The "key" to preventing this from happening is achieving and then maintaining/preserving "equilibrium". How do we achieve such equlibrium? we do it with the "release enabler" patterns for codeline management (which help ensure "nested synchronization" and "harmonic cadence" in addition to achieving a balance or equilibrium between stability and velocity (to smooth out flow).

Third Law of Thermodynamics
- As temperature approaches absolute zero, the entropy of a system
approaches a constant minimum.
In our case, "Temperature" could be regarded as a measure of "energy" or "activity". As the energy/activity of a codeline approaches zero (such as a release in the field that youve been supporting and would LOVE to be able to retire that codeline sometime real soon), it's instability approaches a constant minimum.

This is perhaps another more polite way of saying something we already said in our article on "The Unchangeable Rules of Software Change", namely that "absolute stability" means dead (as in, "no activity"), and should serve as a reminder that our goals is not the prevention of change in order to achieve some ideal "absolute stability", for such an absolute would mean the project not just "done" but "dead".

On the other hand, it also speaks to us as a guideline for when it is safe to retire old codelines, and when to change their policy in accordance with their "energy level"

» An Agile Approach to Release Management

My Agile SCM co-authors Rob Cowham, Steve Berczuk, and myself have written an article for the May CM Journal on An Agile Approach to Release Management

We're relatively pleased with the article, and all collaborated together quite well.

» BOOK: Software Teamwork - Taking Ownership for Success

My review of Jim Brosseau's Software Teamwork: Taking Ownership for Success is available in the May issue of the Agile Journal. It is nothing less than outstanding!

I found Software Teamwork to be an immensely helpful, intensely practical, profusely insightful field guide to improving team outcomes and changing team behaviors by focusing on interpersonal action and personal leadership. This book belongs on any software team-leader's bookshelf, along with Jean Tabaka's Collaboration Explained and Murray Cantor's Software Leadership.

Other articles in this issue on the theme of "Challenges with Distributed Agile" are:

» From PMBoK to Agility

I recently learned that Michelle Sliger, author of the wonderful 4 part series of articles on Relating PMBoK to Agile Practices, is co-authoring a book with Stacia Broderick entitled the Software Project Manager's Bridge to Agility. You can even download an excerpt from her website.

I'm looking forward to this book a great deal, judging by the excellent articles and presentations of hers that I've read.

May 27, 2008
» Three Pivotal Practices to Eliminate Waste

I received my program for the Better Software Conference & Expo this coming June 9-12 in Las Vegas (alas, I will be unable to attend). The description for the keynote that will be given by Jean Tabaka caught my eye. Jean Tabaka is an Agile Coach from Rally Software Development and the author of Collaboration Explained: Facilitation Skills for Software Project Leaders).

Her keynote is entitled "Attacking Waste in Software: Three Practices We Must Embrace Now" and the abstract is as follows:

One of the seven principles of Lean Thinking is “eliminate waste.” Eliminating waste means minimizing the cost of the resources we use to deliver software to our stakeholders. Jean Tabaka proposes three pivotal practices that we must embrace to aggressively attack waste in software delivery —- Software as a Service (SaaS), Community, and Fast Feature Throughput:
  1. Software as a Service (SaaS) eliminates waste by deploying software-based services without the cost inherent in traditional software delivery—materials, shipping, time delay, and more.

  2. Community involves stakeholders working together to create products rather than competing among themselves for limited resources. Community eliminates waste by democratizing software development to obviate the need for multiple systems with the same functionality.

  3. Fast Feature Throughput refers to development methods that embrace change and quickly deliver value to customers. It eliminates waste by responding to market pull with short, incremental delivery cycles.
When IT and all software organizations embrace these practices, they will eliminate waste within their organizations, reduce the waste that consumes our entire industry, and ultimately support the broad 21st century global mandate to manage our scarce resources.

I can't help but think how these same "pivotal practices" apply equally well to Agile CM, resulting (presumably?) in Software CM as a Service (SCMaaS), Community, and Rapid Change-Flow (where the latter refers to both quickness and responsiveness of change assessment and approval, as well as to development velocity as the changes flow through codelines and become integrated, built, promoted and released).

May 12, 2008
» Software Process-Line Architecture and Common Processes

Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing how software product lines relate to software process architecture and "common processes" across an enterprise.

Many organizations strive for standard common processes, often as part of a CMM/CMMI-based process improvement. All too often I have seen the mantra of "common process" misused and abused to make the practitioners serve the process instead of the other way around.

Processes don't create great software. People and Teams do!
And while the process needs to meet the needs of the business and the needs of the customer, it has to first and foremost serve the needs of the practitioners so that they in turn may better serve the needs of the business to deliver operational business value.

Many in management seem to have the mis-impression that "common process" means "no tailoring" and everyone does everything the same way across products and projects throughout the organization. Process variation across products and projects is regarded as something to eschewed and stamped out, beating the offenders into compliance with top-down dictates and mandates and sanctions. If everyone does everything the same way then the people are more or less "plug-and-play replaceable" and can quickly and easily be reallocated to another project or product with zero learning-curve and associated start-up costs.

This is a dangerous myth that causes irreparable harm to process improvement and common/standard process efforts. Anything that focuses on individuals and interactions as subservient to common processes and standard tools is doomed to fail, and those organizations often end-up with the processes they deserve (along with many disgruntled, frustrated workers).

The purpose of such common processes and tools is not to be a rigid restrictive straightjacket for replaceable people. The intended purpose is to recognize that such people are irreplaceable and to provide a flexible knowledge framework to guide and enable them as the help each other collaborate to learn, grow, and lead in the discovery, practical application and effective execution of practices and improvements that are the best fit for a particular product, product, community and business-environment.

The intended purpose common software processes is quite simply that of process and knowledge reuse! And as such it shares many of the same fundamental problems and solutions as that of software reuse. Indeed it could even be argued that software process reuse is but a special case of software reuse. And current prevailing industry wisdom on the subject suggests that software product-lines show the greatest promise of leveraging software reuse for greatest business value.

In software reuse, we seem to recognize that "one size does not fit all." We acknowledge that even though different products, components, and platforms may share common features, that each one may have different project parameters and environments with different quality attributes and engineering-tradeoffs that need to be "preferred" and optimized that particular application: dynamic versus static, performance versus memory, storage versus latency, throughput versus bandwidth, single versus multi processing, optimistic versus pessimistic concurrency, security versus availability, and on and on.

Software process reuse is no different. Different products and projects have their own uniquely differentiating value proposition (if not there would be no business-need to attempt them in the first place). And those differentiating aspects warrant many kinds of process variation across projects, products, technologies and teams.

Those coming from a SixSigma background may point out how SixSigma strives for reducing process variation. But it's all too easy to forget the context for that is when repeatably reproducing the same output from the same inputs for the same desired set of quality attributes and tradeoffs (not to mention the "kinds" of variation SigSigma is appropriate for trying to eliminate).

So I would advocate the translation and application of software product-line practices to software processes (software "Process-Lines" or "Process Families" if you will) and the treatment of such common processes as first class architectures that need to accommodate the views and perspectives of ALL their critical stakeholders, and which should identify their essential quality attributes and tradeoffs, approaches to managing commonality and variability, and apply appropriate patterns and tactics (such as modifiability tactics for software processes and projects) to meet those objectives.

In light of the above, Lean & Agile software development define an architectural style for such process families and their architecture. Lean and Agile principles identify some of the critical process-quality attributes for such efforts. And the corresponding enterprise and its product offerings and their market segments may identify additional quality attributes that needs to met (such as security, regulatory auditability/compliance, large-scale and/or distributed projects and teams, etc.)

April 23, 2008
» Software Process Architecture Views and Quality Attributes

After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.

Well, about a year ago I wrote that Software CM is NOT a Process! ...

Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.

The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.

So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:

  • What are the views and perspectives of an SCM solution architecture?

  • How do software architecture quality attributes relate to software CM (or even process) quality attributes?

  • What are the quality attributes that, if attained, will make the resulting software CM and/or process environment Agile?

I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove most challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.

The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
  • Process Availability might correspond to the availability of the underlying tools and technology that support the process. But it might also need to include both the physical and cognitive "availability" of the process itself. It probably also needs to include the availability of key information (i.e., metrics and reports) to create information radiators, and big visible charts & reports.

  • Process Modifiability is the ease with which the process itself can be adapted, extended/contracted, perhaps even "refactored", and ultimately improved. Rigid processes can't be changed very rapidly in response to a change in business need or direction.

  • Process Performance is probably what most closely translates to flow or throughput of software development. (Although for tools and the supporting computing environment, it clearly has the usual meaning there.)

  • Process Security is ... hmmn, that's a tough one! Would it mean "safety" as in keeping the practitioner safe/secure? Would it mean process quality? Or might it mean the security of the process itself in terms of making sure that only authorized/authenticated persons have access to the knowledge of the system (its requirements, designs, etc.) which may be proprietary/confidential and a significant competitive advantage, and that only those authorized individuals are allowed to execute the roles & workflows that create & modify that system knowledge? Perhaps "security" in this context is all about trust and trustworthiness: How well does the process ensure the trust and integrity of the system and of itself? How well does it foster trust among its practitioners and consumers?

  • Process Testability might correspond to the ease with which the process and its results can be tracked/reported (transparency) and audited (auditability). Perhaps it is also related to the ease with which parts of the process can be automated.

  • Process Usability probably has to do with the amount of "friction" the process imposes on the flow/throughput of development. Is it too big? too complex? a poor fit? easy to understand and execute? easy to tell if you did it correctly?


What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?

How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?

March 19, 2008
» Software Process Architecture Views and Quality Attributes

After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.

Well, about a year ago I wrote that Software CM is NOT a Process! ...

Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.

The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.

So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:

  • What are the views and perspectives of an SCM solution architecture?

  • How do software architecture quality attributes relate to software CM (or even process) quality attributes?

  • What are the quality attributes that, if attained, will make the resulting software CM and/or process environment Agile?

I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove most challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.

The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
  • Process Availability might correspond to the availability of the underlying tools and technology that support the process. But it might also need to include both the physical and cognitive "availability" of the process itself. It probably also needs to include the availability of key information (i.e., metrics and reports) to create information radiators, and big visible charts & reports.

  • Process Modifiability is the ease with which the process itself can be adapted, extended/contracted, perhaps even "refactored", and ultimately improved. Rigid processes can't be changed very rapidly in response to a change in business need or direction.

  • Process Performance is probably what most closely translates to flow or throughput of software development. (Although for tools and the supporting computing environment, it clearly has the usual meaning there.)

  • Process Security is ... hmmn, that's a tough one! Would it mean "safety" as in keeping the practitioner safe/secure? Would it mean process quality? Or might it mean the security of the process itself in terms of making sure that only authorized/authenticated persons have access to the knowledge of the system (its requirements, designs, etc.) which may be proprietary/confidential and a significant competitive advantage, and that only those authorized individuals are allowed to execute the roles & workflows that create & modify that system knowledge? Perhaps "security" in this context is all about trust and trustworthiness: How well does the process ensure the trust and integrity of the system and of itself? How well does it foster trust among its practitioners and consumers?

  • Process Testability might correspond to the ease with which the process and its results can be tracked/reported (transparency) and audited (auditability). Perhaps it is also related to the ease with which parts of the process can be automated.

  • Process Usability probably has to do with the amount of "friction" the process imposes on the flow/throughput of development. Is it too big? too complex? a poor fit? easy to understand and execute? easy to tell if you did it correctly?


What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?

How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?

March 18, 2008
» Brad Appleton

After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.

Well, about a year ago I wrote that Software CM is NOT a Process! ...

Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.

The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.

So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:
  • What are the views and perspectives of an SCM solution architecture?
  • How do software architecture quality attributes relate to software CM (or even process) quality attributes?
  • What are the quality attributes that, if attained, will make the resulting software CM and/or process environment Agile?
I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove must challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.

The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
  • Process Availability might correspond to the availability of the underlying tools and technology that support the process. But it might also need to include both the physical and cognitive "availability" of the process itself. It probably also needs to include the availability of key information (i.e., metrics and reports) to create information radiators, and big visible charts & reports.

  • Process Modifiability is the ease with which the process itself can be adapted, extended/contracted, perhaps even "refactored", and ultimately improved. Rigid processes can't be changed very rapidly in response to a change in business need or direction.

  • Process Performance is probably what most closely translates to flow or throughput of software development. (Although for tools and the supporting computing environment, it clearly has the usual meaning there.)

  • Process Security is ... hmmn, that's a tough one! Would it mean "safety" as in keeping the practitioner safe/secure? Would it mean process quality? Or might it mean the security of the process itself in terms of making sure that only authorized/authenticated persons have access to the knowledge of the system (its requirements, designs, etc.) which may be proprietary/confidential and a significant competitive advantage, and that only those authorized individuals are allowed to execute the roles & workflows that create & modify that system knowledge?

  • Process Testability might correspond to the ease with which the process and its results can be tracked/reported (transparency) and audited (auditability). Perhaps it is also related to the ease with which parts of the process can be automated.

  • Process Usability probably has to do with the amount of "friction" the process imposes on the flow/throughput of development. Is it too big? too complex? a poor fit? easy to understand and execute? easy to tell if you did it correctly?

What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?

How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?

January 28, 2008
» Mary Poppendieck on How to Fail with Agile

Marry Poppendieck gave a keynote at SQE's Agile Development Practices Conference during the last months of 2007. The slides for that presentation are now available online. The presentation is entitled Welcome to the Mainstream (and How to Fail with Agile)

An interesting read!!!

January 15, 2008
» BOOK: The Art of Agile Development

My review of The Art of Agile Development is in the January 2008 issue of the Agile Journal

This is an amazing book! The Art of Agile Development is nothing less than 10+ years' worth of agile development experience distilled into a single compendium of practical insight and mindful application of Agile practices and principles. James Shore and Shane Warden have succeeded marvelously in doing exactly what they set out to do: "packed everything we knew about the day-to-day practice of agile development into 400 pages ... to provide a complete how-to guide and starter kit for beginning and intermediate agile practitioners" (quoted from the book's website).

... I can't think of a better XP practitioner-guide to date that conveys both the practices and principles of XP for novices and intermediate-level readers, and also goes beyond explaining them to provide quintessential insights, tips, contraindications, alternatives, and organizational strategies for how to overcome the many technical and organizational barriers that can stall an otherwise successful attempt adopting XP.

Read the full review

January 11, 2008
» The Truth About Agile Processes

After my previous long-winded entry about "What do you wish your CEO knew about Agile?", I came across this Forrester report on "The Truth about Agile Processes"

It has a summary of what CEOs/CIOs and managers need to know are the changes they need to make in order to adopt agile:

  • Larger-teams should be broken into smaller ones
  • Functional silos have to be broken down, or at least weakened
  • Specialists may have to pick up new skills
  • Teams must learn to self-manage, and managers must learn to let them
  • Routine activities have to be automated (integration, builds, testing, etc.)

These underlie the message that "going Agile" is more than just a change in one's technical processes. It also requires changing (mind-set as well as process) how one does not only project-management, but also overall management - including status-accounting and reporting of results. And it requires a great deal of rigor/discipline (even tho it may be less formal) in these practices as well as the technical/engineering practices.

You can't expect to make a successful go of Agile by modifying only your technical process & practices. Management has to make some changes too, and the most significant change may be the mind-set for how they are accustomed to measuring and reporting progress (especially for those currently using stage/gate management).

» What do you wish your CEO knew about Agile?

Esther Schindler has been posting on a number of different Agile-related groups & forums asking participants the question:

If you could get the boss to understand one thing, just ONE THING, related to Agile development... what would it be? Why that?

When she posted this to the Agile CM forum at cmcrossroads.com, my reply was as follows ...

My "one thing" would be a bit more basic than either CM or "Agile" (IMHO), and culled from Steve McConnell's talks on "Critical Insights for C-level Executives" and "The 10 Most Powerful ideas in Software Engineering" both of which are available online from Construx (unfortunately you have to register to download them, although registration is free and they promise not to spam you)

McConnell starts off with what he says are 5 insights that C-level Executives already know:
  • Software projects are late
  • Software projects are out of control
  • Answers from software staff are often incomprehensible
  • Answers from software staff often seem evasive
  • Software is the most unpredictable part of the business

Then he goes on to describe what he calls the 7 Critical insights about software development that C-level executives need to know:
  • Typical Budgeting Processes Undermine Effective Development (and don't acknowledge the "cone of uncertainty")
  • Stage-Gate Processes Are Prevalent In Leading Companies
  • Low Quality Is The Single Largest Cost Driver
  • People (Staff) Exert The Largest Impact On Project Outcomes
  • Software Improvement Works Best When Supported at the Organizational Level
  • The Tradeoffs Between Cost, Schedule, And Quality Might Not Be What You Think They Are
  • Improved Software Practices Offer Exceptionally High ROIs

McConnell describes the 10 most important/powerful ideas in software development as follows:
  • Software Development Work is Performed by Human Beings (people are the single biggest influence factor for success)
  • Incrementalism is Essential
  • Iteration is Essential
  • The Cost To Fix A Defect Increases Over Time, Throughout the Development Lifecycle (even when using iterative/incremental/agile development)
  • There’s an Important Kernel of Truth in the Waterfall Model
  • Ability to Create Useful Software Estimates Can be Improved Over Time (The Cone Of Uncertainty)
  • The Most Powerful Form of Reuse is Full Reuse
  • Risk Management Provides Critical Insights into Many Core Software Development Issues
  • Different Kinds of Software Call For Different Kinds of Software Development - The “Toolbox” (there is no single best best practice or method across all software projects/products)
  • Software Engineering Body of Knowledge - SWEBOK (existing body of knowledge provides a wide spectrum of support for software development practices, especially CM, PM, RM, QM, Testing, Design, Construction, Management, Maintenance, Methods and Tools)

So how would I distill all of that into a single statement to communicate to C-level executives? Tom Gilb get's an honorable mention for his "The Ten Most Powerful Principles for Quality in [Software and] Software Organizations", which he summarizes as a single principle saying:
    "Motivate people
    towards real results
    by giving them numeric feedback frequently
    and the ability to change anything for success."
For me, the gist of all the above is that most corporate planning, estimation and management "systems" for software are badly broken because they utterly fail to acknowledge the inherent uncertainty and variation that cannot be removed by "up front" attempts at perfectly stable project plans and/or product requirements (see my article on "The Unchangeable Rules of Software Change"):
  • The solution is not going to come about by striving ever harder for perfectly stable plans and requirements earlier in the project!
  • The only successful way to manage that is through effective and continuous management of risk and change!
  • And the most effective means of doing that is through the use of iterative and incremental techniques that put people first in order to give frequent feedback and reflection on real-results throughout the course of the project (while still leveraging the existing body of knowledge to serve those people so they can best serve the business).

Many companies seem to behave as if merely identifying risks and having mitigation plans is sufficiently effective, and they just need to do that "up front" and then monitor the risk-list without having to spend much time adjusting, adapting, and refining it throughout the project.

Similarly, many companies seem to behave as if they can effective manage change is they basically strive to prevent changes once the project/requirements is (are) initially baselined.

All that is poppycock! Effective risk and change management comes about only by taking a continuous approach to manage them that use regular increments and iterations to achieve frequent feedback on tangible, valued results. Even that by itself is not enough, because if you do not enable and empower your people to act on those results, collaboratively, throughout the entire lifecycle, then you won't be very effective or efficient at reaching your goals. Your process has to leverage the power of your people rather than being an obstacle that overly constrains/controls them.

That all goes for CM and managing change just as much as it goes for development and managing projects! Stop thinking it is feasible to drive out 100% of the variation and uncertainty early in the project, and rechannel that energy instead on managing risk and change continuously, utilizing iterative/incremental means to attain frequent & regular feedback. Then make sure you empower your people to act on that information by effecting necessary changes as quickly as possible, and using processes that regularly embrace such changes instead of stalling or preventing them.

January 8, 2008
» Flexible Development, Manage It, and Continuous Integration

During my hiatus from this blog since the summer, I still managed to keep up with a few of my book reviews for The Agile Journal. Here they are (with hyperlinks) ...

Flexible Development: Building Agility for Changing Markets
by Preston G. Smith
published September 2007 by Jossey-Bass

Manage It! Your guide to Modern Project Management

by Johanna Rothman
published June 2007 by the PragmaticProgrammer's Pragmatic Bookshelf
    My rating: Outstanding! If a budding project manager came to me, with knowledge of text-book project management concepts and project management tools, and asked me "what one book would you recommend to help me swiftly bridge the gap from theory into modern-practice that would most increase my chances of succeeding" I'd be hard pressed to find another book that I'd recommend more highly than Manage It!
    Read the full review in the November 2007 Agile Journal.

Continuous Integration: Improving Software Quality and Reducing Risk

by Paul Duvall, Steve Matyas, and Andrew Glover
published June 2007 in the Addison-Wesley Signature Series
    My rating: Outstanding! For those of you seeking a book on the subject of Continuous Integration (a.k.a. "CI") ... look no further. The book you have got to run out and get is here! No other book to date even comes close to being such a comprehensive and authoritative source of principles, practices, and current technologies and resources about continuous integration.
    Read the full review in the October 2007 Agile Journal (also see the book's website at www.integratebutton.com)

Visual Studio Team System Better Software Development for Agile Teams

by Will Stott & James Newkirk
published May 2007 by Addison-Wesley Professional

Oh, and don't forget to look for my upcoming review of The Art of Agile Development in the January 2008 issue of the Agile Journal.

July 16, 2007
» BOOK: xUnit Test Patterns

xUnit Test Patterns: Refactoring Test Code, by Gerard Meszaros (Addison-Wesley Signature Series, May 2007) is nothing short of FANTASTIC!

As I'm reading through the book, I keep saying to myself "Yes!" and "Aha!" over and over again. This book is very appropros for me right now because I'm dealing with a team that has a legacy codebase and is just trying to develop automated tests and they don't yet have a lot of training or experience refactoring or in automating their own tests, or the xUnit-style tools, much less using all of them together.

This book is EXACTLY what the doctor ordered for my current situation. It covers the basics of test automation styles and frameworks using harnesses and drivers, data-driven as well as other styles and the challenges to be addressed, especially with test setup. It has comprehensive coverage of 68 patterns and an awesome website at http://xunitpatterns.com/ which does a far better job of depicting them than I could on this blog.

If you're trying to get a small group of folks to learn how to do agile-style test-automation and refactoring, this is the book with all the patterns and insight you will want to convey to your team.

'Nuff said!

July 2, 2007