Peter Eeles and I must be subconsciously on the same page. Because at the same time I was blogging about Software Architecture Views and Perspectives and Software Architecture Quality Attributes and their direct applicability to SCM/ALM solution architecture (and software process in general), Peter was working on an article for IBM developerworks entitled The Rise of the Development Environment Architect:[Development] environments present challenges; and, interestingly, these challenges are similar to those of the systems they support. For example, development environments have to deliver against the required functionality and properties (such as performance and usability), often have to coexist with legacy systems (such as, in the case of a development environment, existing methods and tools), and have to acknowledge other constraints (such as the distributed nature of development teams, and existing skills and infrastructure).
All in all, creating a well-oiled development environment that accelerates, rather than hinders, project performance is a science unto itself. This is why IBM® Rational® has spent many years specifically developing a services capability that understands the challenges faced by organizations that 1) want to improve developer productivity, and 2) regard their development organization as a strategic differentiator, rather than simply a cost center.
Our experience has led the Rational team to define a role within the software development lifecycle called the "development environment architect." In October 2007, one hundred of Rational's most experienced development environment architects from across the globe gathered together in the first conference dedicated to this role to share their experiences. This article is a result of that conference and the discussions that took place.
As you read the concepts presented here, you may well question whether the development environment architect should be a role itself, or whether the individual or team who normally functions in the software or systems architect role should simply add consideration of the development environment to their list of architectural concerns. I believe that both propositions are valid. Furthermore, whenever the role of the "architect" is discussed, it is always qualified with the domain under consideration; thus we speak of a "building architect," "software architect," "systems architect," "enterprise architect," etc. The development environment is simply one of these domains, and one that is not traditionally a concern for the "software architect" role. I therefore believe that the "development environment architect" role is one that hasn't been emphasized before -- hence this article.
This article has several audiences and objectives. It is relevant to organizations undertaking an improvement to their development environment and who need to understand the value of a development environment architect to help guide their initiative. It is also relevant to those who are responsible for the technical content of the development environment -- i.e., development environment architects -- because this article introduces this responsibility as a role not previously defined. Finally, this article may supplement material contained within a development environment, in helping communicate its content, the role of its architect, and the benefits of having such a role in place.
Read the full article here. I may blog later about the similarities and differences between the sort of architecture that Eeles describes versus my 4+2 Views Model of SCM/ALM Solution Architecture
Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing how software product lines relate to software process architecture and "common processes" across an enterprise.
Many organizations strive for standard common processes, often as part of a CMM/CMMI-based process improvement. All too often I have seen the mantra of "common process" misused and abused to make the practitioners serve the process instead of the other way around.Processes don't create great software. People and Teams do!
And while the process needs to meet the needs of the business and the needs of the customer, it has to first and foremost serve the needs of the practitioners so that they in turn may better serve the needs of the business to deliver operational business value.
Many in management seem to have the mis-impression that "common process" means "no tailoring" and everyone does everything the same way across products and projects throughout the organization. Process variation across products and projects is regarded as something to eschewed and stamped out, beating the offenders into compliance with top-down dictates and mandates and sanctions. If everyone does everything the same way then the people are more or less "plug-and-play replaceable" and can quickly and easily be reallocated to another project or product with zero learning-curve and associated start-up costs.
This is a dangerous myth that causes irreparable harm to process improvement and common/standard process efforts. Anything that focuses on individuals and interactions as subservient to common processes and standard tools is doomed to fail, and those organizations often end-up with the processes they deserve (along with many disgruntled, frustrated workers).
The purpose of such common processes and tools is not to be a rigid restrictive straightjacket for replaceable people. The intended purpose is to recognize that such people are irreplaceable and to provide a flexible knowledge framework to guide and enable them as the help each other collaborate to learn, grow, and lead in the discovery, practical application and effective execution of practices and improvements that are the best fit for a particular product, product, community and business-environment.
The intended purpose common software processes is quite simply that of process and knowledge reuse! And as such it shares many of the same fundamental problems and solutions as that of software reuse. Indeed it could even be argued that software process reuse is but a special case of software reuse. And current prevailing industry wisdom on the subject suggests that software product-lines show the greatest promise of leveraging software reuse for greatest business value.
In software reuse, we seem to recognize that "one size does not fit all." We acknowledge that even though different products, components, and platforms may share common features, that each one may have different project parameters and environments with different quality attributes and engineering-tradeoffs that need to be "preferred" and optimized that particular application: dynamic versus static, performance versus memory, storage versus latency, throughput versus bandwidth, single versus multi processing, optimistic versus pessimistic concurrency, security versus availability, and on and on.
Software process reuse is no different. Different products and projects have their own uniquely differentiating value proposition (if not there would be no business-need to attempt them in the first place). And those differentiating aspects warrant many kinds of process variation across projects, products, technologies and teams.
Those coming from a SixSigma background may point out how SixSigma strives for reducing process variation. But it's all too easy to forget the context for that is when repeatably reproducing the same output from the same inputs for the same desired set of quality attributes and tradeoffs (not to mention the "kinds" of variation SigSigma is appropriate for trying to eliminate).
So I would advocate the translation and application of software product-line practices to software processes (software "Process-Lines" or "Process Families" if you will) and the treatment of such common processes as first class architectures that need to accommodate the views and perspectives of ALL their critical stakeholders, and which should identify their essential quality attributes and tradeoffs, approaches to managing commonality and variability, and apply appropriate patterns and tactics (such as modifiability tactics for software processes and projects) to meet those objectives.
In light of the above, Lean & Agile software development define an architectural style for such process families and their architecture. Lean and Agile principles identify some of the critical process-quality attributes for such efforts. And the corresponding enterprise and its product offerings and their market segments may identify additional quality attributes that needs to met (such as security, regulatory auditability/compliance, large-scale and/or distributed projects and teams, etc.)
Continuing the previous discussion on software product-lines ...
Central to the notion of product-lines and product-families are tracking and managing three different kinds of software assets:
Architecture for such product-lines is all about managing commonality and variability, and easing their evolution to achieve a diverse family of products to achieve economies of scale from reusing common assets. Change/Configuration Management for SPLs is a very challenging problem. And variability management techniques often come down to a matter of binding-times. There are also more advanced strategies (some involving mathematical models).
A few resources on the subject of Commonality and Variability are as follows:
In July 2006 I presented at the Dr Dobbs' Architecture & Design World conference about SCM Patterns for Agile Architectures, which included a section on managing variations. I summarized that portion of the presentation as follows:Use Late-Binding instead of Branching:
Think about which of the following needs to "vary" and what needs to stay the same:
Commonality & Variability analysis helps identify the core dimensions of variation for your project
Use a combination of strategies based on the different types of needed variation and the "dimension" in which each one operates
Extending the analogy of software architecture views and quality attributes for software process architecture, I'd like to spend some time discussing software product lines. According to the SEI website on software product-lines, A Software Product-Line is defines as follows:A software product line (SPL) is a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.
At SoftwareProductsLines.com, Charles Krueger defines them as follows:Software product lines refers to engineering techniques for creating a portfolio of similar software systems from a shared set of software assets using a common means of production.
The key objectives of software product lines are: to capitalize on commonality and manage variation in order to reduce the time, effort, cost and complexity of creating and maintaining a product line of similar software systems.
Closely related to software product lines is the notion of software product families and Product Family Engineering. In many cases the terms product-line and product-family are used interchangeably. Sometimes a product-family is slightly more general in that a product-family may comprise one or more product-lines. The SEI has established a Framework for Software Product-Line Practices that encompasses topics such as architecture, organization, patterns, business-case, and even a section on configuration management for software product-lines.
After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.
Well, about a year ago I wrote that Software CM is NOT a Process! ...Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.
The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.
So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:
I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove most challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.
The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?
How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?
After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.
Well, about a year ago I wrote that Software CM is NOT a Process! ...Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.
The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.
So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:
I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove most challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.
The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?
How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?
After my previous postings on Software Architecture Views and Perspectives, Software Architecture Quality Attributes and Software Modifiability Tactics, the question remains as to what all this has to do with Agile processes or with CM.
Well, about a year ago I wrote that Software CM is NOT a Process! ...Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.
The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.
So now I want to take the perspective of Software CM as an architecture, and I want to consider questions like:
I think I answered the first of these questions in my Dimensions and Views of SCM Architecture. I take the perspective of a 4+2 views model comprising Product, Project, Evolution, Environment, Process (+1), and Enterprise (+2). These views straddle the conceptual and physical aspect of both the content and context of the different kinds of "containers" that are to be managed and interrelated. And the dimensions of SCM complexity that prove must challenging are those of scale and diversity, and of differences between artifact change/creation time and decision binding-time.
The next question involves translating what Availability, Modifiability, Performance, Security, Testability, and Usability mean for a "process" architecture. I'll make an initial stab at that (feedback is encouraged):
What are the "quality" attributes of an "agile" process? Do they include ALL of the above? what about: adaptive? lean? result-driven (i.e. "working software")? self-organization? iterative? collaborative?
How about some of the traditional "quality" attributes of a CM system: traceability (vs. transparency?), reproduceability? repeatability?
I'm fairly interested in the literature on Software Architecture Views and Perspectives. Folks here may remember by work on Dimensions and Views of SCM Architecture as one of the reasons why ...
The text of the entire 2nd edition of the "Software Architecture in Practice" textbook is available online as one source fo information on the subject (among others). I found another good link (& book reference) at http://www.viewpoints-and-perspectives.info/
It's the website for the book "Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives" by Nick Rozanski and Eoin Woods. They classify/use "Viewpoints" and "Perspectives" as follows:
Viewpoints:
Perspectives:
Found a few other links too:
Getting back to the subject of my previous blog-entries on Software Architecture Views and Perspectives and Software Architecture Quality Attributes, I wanted to talk more specifically about the quality attribute of Modifiability.
The Modifiability of a software system is related to how minimal is the cost/effort to develop and deploy changes to the software. This relates to well known concepts and principles of coupling, cohesion, maintainability, etc. and is the basis for many of the elements of object-oriented, component-based, and aspect-oriented design ("reuse" is a close cousin for all these as well).
Software Modifiability Tactics are presented in section 5.3 of Software Architecture in Practice. A taxonomy is given which relates architectural tactics to architectural patterns ("styles") and the design patterns which are largely concerned with achieving the attribute (in this case "modifiability") for various types of products and contexts. The article Understanding Architectural Patterns in Terms of Tactics and Models even has a nice matrix that maps architectural patterns or styles to the various kinds of modifiability tactics.
The taxonomy for software modifiability tactics is broken down as follows:
In September 2007, an SEI Technical Report on Software Modifiability Tactics was published that provides a comprehensive discussion of these modifiability tactics and the architecture/design patterns that can be used to implement them (and some of the tradeoffs involved).
Links and Resources on Software Modifiability Tactics:
In a subsequent blog-entry I will muse about how Modifiability relates to Agility in both software architecture/design and in software process architecture/design.
Following on to my previous blog-entry about Software Architecture Views and Perspectives, the book "Software Architecture in Practice" also describes a method called Attribute-Driven Design or ADD. This is not yet-another-design-method like BDD or TDD. ADD is concerned software architectural design (so it's at the "architecture-level" rather than what we might normally think of as the "design-level").
ADD is concerned with explicitly identifying the desired quality attributes of an architecture. Many of us know that simply implementing the (functional) requirements correctly is just the beginning of any good design, and possibly not even the most important attribute of the design. In addition to other attributes like security or availability, there are also attributes like modifiability of an architecture. And it is often these attributes that, if attained, are the true indication of whether or not we've done a good job.
Some of the more commonly desired quality attributes are:
- Availability
- Modifiability
- Performance
- Security
- Testability
- Usability
ADD also explicitly mentions the use of patterns and tactics as part of its methodology for achieving quality attributes and making design tradeoffs (quality attributes are often the "forces" for a pattern). For each of the common quality attributes above, it identifies a taxonomy of common tactics and patterns used to make good tradeoff decisions for particular aspects of the design.
Although it look s a bit heavyweight, ADD looks promising in its use of patterns and tactics and recursive/iterative nature. But what I look most about it is that fact that it makes explicit the kinds of quality attributes and their non-functional use-cases or "stories" which can then have business value/priority associated with it, and thereby justify the existence of activities that help realize those attributes of the system.
For some more information on ADD, see the following:
- Software Architecture in Practice
- The SEI webpage on Attribute-Driven Design
- The paper Quality Attribute Design Primitives and ADD
- Introduction to the ADD Method
- A Practical Example of Applying ADD (SEI Technical Report)
- Links from Real-world Software Architecture in .NET Developer's Journal
- Presentation from Paul Clements on Designing Software Architectures
- The paper Generalizing a Model of Software Architecture Design from Five Industrial Approaches (by a star-studded list of authors in the field)
April 2007 marked the release of theObject-Oriented Analysis & Design with Applications, 3rd edition by Grady Booch et.al.
For many, this book was the one that first turned them on to object-orientation as more than just a particular programming language or two, but as an overall way of thinking about how to analyze and design software programs. The book has many classic epiphanies that were precursors to things UML, RUP, patterns and agile development. Booch himself is practically a demi-god of modern software development, architecture and practice.
I guess I've moved on a bit since reading the first edition in the early nineties, because the third edition didn't have as much "zing" for me. And there are so many other good texts around now, that this one reads a bit dry and academic.
I'm actually more interested lately in the work that Booch is doing toward a Handbook of Software Architecture and in his blog.
The (hopefully) provocative title of today's blog-entry is inspired by Phil Armour's essay "Software is Not a Product!", which is one of many profoundly insightful essay's from his book The Laws of Software Process.
Armour maintains that:The core issue with the business of software is that we misunderstand what software really is. Software is thought of as being a product, it is looked at as being a product, it is mostly managed as if it were a product. But it's not a product... The hard part about creating software systems is not creating them, it is in acquiring the knowledge necessary to create them (correctly).
Therefore:Software is not a product, it is a medium for storing executable knowledge.
Our lack of understanding of this basic fact is one of the key issues facing the software industry. We do not "build systems"—we acquire knowledge. The systems we ship to our customers are actually the byproducts of the real activity, which is to learn.
The business imperative is that the real product is the knowledge that is in the systems we ship to the customer, and we don't manage that at all.
So today, I'll attempt a similarly-themed "riff" about Software CM, beginning with the assertion that Software CM is not a process!
This assertion might seem shocking to some. Other CMers might say they have known this all along, saying CM is a discipline (rather than a process), and that numerous formal definitions of SCM have said this for quite some time: SCM is a "discipline" that "governs" or applies "technical and administrative direction and surveillance" to the functional components of CM (item identification & planning; item storage, versioning & change control; status accounting, audit & review).
If software is a medium for storing executable knowledge, and software development is a knowledge acquisition & creation (learning) activity, then the result of software CM is the medium through which software development changes and collaborative learning must ultimately flow.
If the result of software CM is a medium for the conveyance of changes by executing the knowledge gained from learning, then software CM itself must impose some form of order (structure & rhythm) to achieve this resulting collaborative flow of change-flow.
For me, terms like "order" and "structure" evoke memories from my years participating in the software patterns movement and study+discussion of the works of Christopher Alexander. For Alexander, patterns were all about discovering the innate order in structures that emerged naturally from the recurring attempt to resolve (trade-off) the same set of competings concerns (forces) for the same problem in the same/similar contexts. The resulting structures and their sequencing and interaction is what all comes together as an overall (emergent) architecture.
Hence, I would say that Software CM is an emergent yet intentional architecture that ensures the orderly, efficient flow of software development change & collaboration.
The elements of this Software CM architecture include practices, tools & technology, teams & organizations, valued deliverables & intermediate work-products, changes to and assemblies of these deliverables & work-products, and the set of needed status/tracking reports & measures.
Hmmn, this fits in rather nicely with my 4+2 Views of SCM/ALM Solution Architecture and perhaps even does a good job of tying together SCM with Agile/Lean (collaboration & flow), and software patterns (particularly SCM Patterns :).
So there you have it: Software CM creates the medium through which software development changes & activities must flow. Therefore, Software CM is the intentional architecture of software development change-flow.
What do you think?
I just received a copy of "Release It! Design and Deploy Production-Ready Software" by Michael Nygard. This looks to be yet another winner from the PragmaticProgrammers' Pragmatic Bookshelf. In fact it made #1 on Amazon.com's "Hot New Releases" list for Design Tools and Techniques for the past two weeks.
Actually, this book isn't about what I had originally thought from reading the early descriptions. I thought it was going to be related to build+release+deployment engineering (CM guy that I am :). It's not about that at all. It is about architecture and design for production deployment concerns, such as: hardware/software "fit", availability/reliability, operability, stability, capacity, and maintainability (not just of the code, but of the deployed product at the customer's site).
"Release It!" covers all those things many of us software-only, high-level abtractionists forgot (or worse yet, never learned) about design & architecture for where the rubber meets the road. I need to add it to my reading list!
Nice article on Defining Design Quality at InfoQ.com. The article heavily references the work of James Shore, who is currently working on a book with chromatic entitled "The Art of Agility". It has a pretty nice definition of design quality:"A good software design minimizes the time required to create, modify, and maintain the software while achieving run-time performance."
It also references a few other nice articles:
This also ties in quite nicely with Agile ideas about "Simple Design" and some earlier blog-postings of mine about Simplicity in Design (and ChangeThis for Simplicity).
Interesting paper from Linda Levine and the SEI about Reflections on Software Agility and Agile Methods: Challenges, Dillemmas & the Way Ahead (see the corresponding presentation slides)
Some other related documents from the SEI are:
See! Not everything from the SEI is about CMM or CMMI ;-) They also a have a wealth of resources on Software Architecture and Software Product-Lines and other areas of Software Engineering!
There are a lot of REALLY GOOD manifestos over at ChangeThis.com! Back in May 2006 I blogged about Simplicity in Design and included several links/resources on the subject. Turns out ChangeThis.com has a few good manifestos on the subject as well (and gets more every month). Here are the ones I liked most:
Other manifestos not necessarily related to the above that I found interesting are:
Charles Betz' book Architecture and Patterns for IT Service Management, Resource Planning, and Governance: Making Shoes for the Cobbler's Children really ought to be required reading for anyone that fancies themselves a "CM professional" (especially Software CM) or an "Enterprise Architect."
I've been following his erp4it blog for over a year now (and his corresponding erp4it YahooGroup). Those looking for a taste of what the book is like can look there, and also at the paper ERP for IT which I understand was an early precursor for the book that has since been vastly updated and expanded in the latter.
Even though the subject of the book and the blog doesnt explicitly scream "CM", the content in the book and the blog are filled with often fundamental and profound insights about CM from the enterprise view (not just IT/ITIL) and where software CM, infrastructure CM, and network/element CM all fit in to the bigger enterprise picture. Several of his post that are among my favorites made their way into the book in some form or another:
And if that weren't enough, the book also puts together three of my favorite topics: CM, patterns, and architecture.
Nostarch Press recently published the book Code Craft: The Practice of Writing Excellent Code by Pete Goodelife.
I have to say the book looks pretty "excellent" itself. It may even become a new classic, replacing the old classic "Writing Solid Code" by Steve MacGuire. And if Steve McConnell hadn't recently updated his even more classic "Code Complete" with a 2nd edition, I might even put Code Craft on a par with that. It's that good!
It's not just about programming & design either. Like Code Complete it covers pretty much all of "software construction." It has some good stuff about build, integration, test, source-control, and more.
About my only pet peeve is that in the "Answers and Discussion" section it talks about recursive make versus inclusive make schemes. It has a pretty good discussion, but I couldn't find any reference to the classic paper by Peter Miller entitled "Recursive Make Considered Harmful". That kind of bothered me because I don't think there's any good excuse for it. It also made me wonder if there were some other places I didn't know about where appropriate references/citations were missed or neglected. (It's not severe enough to make me recommend the book any less highly though!)
I would probably put Code Craft on the "REQUIRED READING" list for any relatively new professional programmer, and have them read it just before before reading Code Complete.