A Django site.
December 4, 2007

Robert Cowham
Robert Cowham's Weblog
» Custom Perforce GUi Installers (for large sites)

For quite a while there has been an option to create your own custom installer for Perforce deployments (including automation). This can be particularly useful for larger sites with lots of new users coming along regularly.

The customizable element of the scripted installer is a configuration file named perforce.cfg. The Perforce Administrator edits this file, and places it on a shared network drive along with the expanded contents of the perforce.zip file. Perforce client programs can then be installed by running setup.exe from each desktop.

One of the annoyances with this approach is that you have to expand the .zip file and put things on a network drive. It is not easy to do this sort of thing within a single executable for example (which makes it hard to put on your intranet and get users to double click).

Well, having played around with various installers and options, I have discovered a fairly simple way of doing this via a self extracting executable (SFX), and the open source 7-Zip program.

Instructions

Create a directory structure:

  • some root dir
    • 7-zip - contains a batch file, a couple of config files and perforce.cfg - see below for contents.
    • p4winst - contains expanded p4winst.zip (without the perforce.cfg)
    • p4vinst - contains expanded p4vinst.zip (without the perforce.cfg)

Currently (as of 2007.2) the perforce.cfg file is the same for both P4Win and P4V.

You need to get a couple of files from 7-zip.org/download.html

Edit perforce.cfg to customize for your installation.

Create a p4winst.conf and p4v.conf (which must both be UTF-8 - Notepad can save this format, as can Notepad++ or other editors) in that directory.

Run make_installers.bat from the command line in the appropriate directory and check that the following are created (which you can put on an intranet or whatever):

  • p4winst.exe
  • p4vinst.exe

You can then send your users a single exectuable which unzips itself and automatically runs the perforce setup.exe with the included perforce.cfg - QED!

make_installers.bat

This contains something like the following:

:: Batch file to package up P4Win using .zip file and the 7-Zip freely available 
:: Zip program (which can create SFX - Self Extracting Archives). 

@echo off 

:: May need to customize the following 
set ZIP_DIR="d:\apps\7-zip" 

call :make_install p4winst 
call :make_install p4vinst 
goto :exit 

:make_install 

set INSTALLER=%1 

:: First create new clean versions of the zip files (including current directory version of perforce.cfg in preference) 
if exist %INSTALLER%.7z del %INSTALLER%.7z 
if exist %INSTALLER%.exe del %INSTALLER%.exe 

%ZIP_DIR%\7z a %INSTALLER%.7z ..\%INSTALLER%\* perforce.cfg 
:: Copy SFX file plus config file plus zipped installer into a single SFX .exe
copy /b %ZIP_DIR%\7zsd.sfx + %INSTALLER%.conf + %INSTALLER%.7z %INSTALLER%.exe 
if not errorlevel 1 goto :EOF 
if exist %INSTALLER%.exe goto :EOF 

echo *** ERROR: failed to create %INSTALLER%.exe 
goto :exit 

:exit 

p4winst.conf (and p4vinst.conf)

These are simple UTF8 format file with contents similar to:

;!@Install@!UTF-8!
Title="P4Win Custom Installer"
BeginPrompt="Do you want to install P4Win?"
RunProgram="setup.exe"
;!@InstallEnd@!

October 31, 2007

Robert Cowham
Robert Cowham's Weblog
» Subconf 2007 - Subversion and the Enterprise

This article is a result of my visit to Subconf in Munich a couple of weeks ago.

The first thing to say is that Subconf was very enjoyable and interesting and my compliments to the organisers. There were some excellent speakers and it was good to chat to various vendors too. I also had a very good time at the dinner that evening - the only slight fly in the ointment was the S-Bahn strike the following morning which meant a rather earlier taxi to the airport than I might have wished!

The two keynotes by Brian Behlendorf and Karl Fogel respectively were particularly good. Indeed it was good to get them at the conference since both guys seem to be scaling down their Subversion involvement - Brian has stepped down as CTO at Collabnet and Karl is running QuestionCopyright.org (although remains as President of Subversion Corporation.

Some of the slides are now available online.

Brian's Keynote

Brian mentioned his involvement in Apache and some of the lessons learnt about creating a community that worked. These lessons were then applied by Collab.net to get Subversion going. If you create a high quality community, you will create high quality software.

  • Need to be nice to people to avoid a "fork" - Development leaders exhibit good communication skills, and can bring different ideas together.
  • Conscious effort to bring new developers along the path: from ?consumers?, to bug reporters, to patch submitters, to active contributors. 

Brian was asked the question in his talk about Linus's somewhat inflammatory video about Git where various sideswipes about brain-dead CVS and Subversion people are made. Brian had no particular axe to grind about this and said that the Subversion camp bore no ill will to the distributed crowd and were rather surprised at some of the venom coming back - a very measured and mature response.

Brian mention the difference between a centralised model and a distributed model - there is a lot that enterprise like about the control of a centralised model. For example, laptops get stolen (he lost his a few weeks ago), and a laptop might contain a whole distributed repository. Enterprises prefer more control!

Martin Doettling's Intro to Karl

The key points of this were:

  • Estimated user base now exceeds 2 million
  • 10x growth since 1.4
  • Large numbers of enterprise users

Karl's Keynote - How Stuff Happens

Karl picked up on some of Brian's points about the community, and showed various ways in which they have been able to get hackers to create software that the enterprise can use!

  • There is a very comprehensive guide to how to contribute - 43 pages of it! This is where people start.
  • Some principles:
    • Make it easy to do things right
    • Make it rewarding to do things right
    • Influence proportional to effort
      • Moving from contributor to partial-committer to commiter
      • Tracking no of patches etc by user - all automated with links (see Contribulyzer)

Other Notes

I couldn't stay for the second day, but some other points of interest:

  • There are increasing numbers of tools based around Subversion - Collabnet, Polarion, CodeBeamer etc
  • Subversion command line is increasingly irrelevant as people use it through Tortoise, Eclipse etc.
  • Merge tracking there at long last with 1.5 just around the corner
  • Really being used in the industry
  • SVK provides an interesting distributed method

I also had some chats with Collabnet around the idea of perhaps the BCS CMSG working with them to put on an event in London next year - definitely something to look into.

Well worth a visit as a conference.

July 10, 2007

Robert Cowham
Robert Cowham's Weblog
» Directing vs. Managing Configurations

In a recent CM Crossroads article on defining Agile SCM I included the quote from Charles Handy that I recently picked up on my travels through an airport:

Go to the theatre and look at the programme. Everyone connected with the performance is listed, no matter how small their contribution. People like to be recognised as individuals. The word "manager" is reserved for those in charge of things, not people, the stage manager and the lighting manager. The people who are in direct communication with the customer, the actors, are directed, not managed, by someone who actually leaves the scene once the project is under way. He or she trusts the cast to go it alone, and as often as not, they improve on the production once the director departs. Trust inspires. One more thing - at the end of each performance they receive an expression of appreciation from their audience, direct feedback from the people who matter. No waiting for the annual performance appraisal.

Charles Handy, Myself and Other More Important Matters, Arrow Books, 2007 - ISBN 9780099481881

This obviously applies to more than just the directing vs. managing debate (e.g. feedback/appreciation), but really hit home for me as an expression of what those of us in the Configuration Management arena need to be doing. We should be directing our colleagues and developers rather than managing them (although we can of course manage actual configurations as before!). We need to get out and about amongst our congregation of developers and others, and infect them with enthusiasm and skills for the delights and benefits of good CM.

There is a huge danger, which I come across fairly frequently, of sitting in your ivory tower wondering why developers aren't doing what you know is "the right thing". They haven't got the message, they aren't doing what they should, and management isn't helping you to beat them up with a big stick to force them into doing it. Sometimes it seems we are just on a hiding to nothing...

In CM we need to aspire to the director (servant-leader) mold - we provide a support service for people to get on with their jobs and we help them do it, but it is they that do (most of) it, not us. There is obviously some level at which CM is still responsible for certain things - in particular processes and systems, and for auditing to ensure things have been done correctly.

Thus I am more and more convinced that, particularly for Agile teams, CM should be the responsibility of everyone not just the CM person(s). This means that as CM professionals our job is:

  • to sell the ideas, principles and tools of sound CM while supporting people in getting their jobs done and delivering value to the business (not just ticking CM boxes)
  • to solve people's day-to-day issues through education and tool/automation support
  • to show how it is easier to do "the right thing" than it is not to do it - a little discipline is worth a ton of lost time due to bugs introduced, or things suddenly not working
  • to get out and about and work with developers, pairing with them and demonstrating good CM practice, discovering those little niggling things that cause lost time and productivity or pain and solving them
  • to ensure that our own working practices are suitable role models (avoid "do as we do not as we say...")
  • to focus on continuous process improvement - there is almost always something more we can do (but again focus on value)

There are many challenges, and in an Agile world, it is more about working with people to define just enough process to be useful, and not too much to get in the way. I was fascinated to hear a vendor representative complaining that some of their customer had developers who were downloading and using Subversion for their day to day work as opposed to using the corporate tool. I wondered what had happened to make the tool or the configured process so unfriendly as to force the developers down this path.

June 19, 2007

Robert Cowham
Robert Cowham's Weblog
» Perforce Branch Naming and Repository Structure Guidelines

One of the the things that comes up just about every time I go in to do some Perforce related consulting for a client is that of repository structure - how do you map some particular branching scheme (combination of branching patterns) onto the repository?

This of course necessary since Perforce uses the repository structure to represent branches as well as other directory structures. This way of recording branches is feature one might occasionally wish were implemented otherwise, but it does have its attractions, and is the same model as used in Subversion and also Microsoft's new Team Foundation Server product (as an aside, it is perhaps not surprising that Microsoft chose this model since it is a not very well kept "secret" that their in house SCM tool called SourceDepot, introduced during the development of Windows 2000, was a re-badged copy of Perforce. Team Foundation Server is obviously a total rewrite on top of SQL Server, but they seem to be comfortable with the "branch in path space" model).

Some clients need help working out a good branching scheme, and some have relevant experience with other tools so are comfortable with that part, but many need help with the mapping. I have covered a little of this in my article Introduction to Perforce Branching, but this article expands on that.

For their flexibility, I would generally recommend people use branch specs (e.g. created by "p4 branch" command) for branching since you can add view mappings to address not propagating renames, and branch extra components. However, if you can keep the branch spec to a single view mapping then that is great! (Note that the P4V GUI allows integration via drag-and-drop - sometimes convenient, but can be somewhat dangerous...).

Guidelines

A repository branch structure is about creating an appropriate information hierarchy (designing an information architecture). So here are some guidelines that I have found useful over the years and seen successfully used in many organisations (and I certainly didn't invent any of them!). Maybe in the future I can come up with some appropriate pattern like descriptions, but for now, here they are:

  • Standardise naming conventions - make branch directories obvious
  • Keep branch spec mappings and client workspace spec mappings as simple as possible (and permissions)
  • Make it as easy as possible for users (new and experienced) to visualise the branching scheme as they click down through the hierarchy
  • Find the right balance between broad & shallow vs. narrow & deep hierarchies
  • Keep branches of a similar type at similar depths in the structure
  • Make branch directories sort sensibly
  • Plan for the future: 2, 3, 5 years down the line - include things like the year (and optionally month) somewhere in the path to provide a natural sorting.
  • Provide Guidelines and Delegate!
  • Remember - there is no one best solution, but there are many good enough solutions!

Some of what comes below is also covered by Laura Wingerd in her recent book "Practical Perforce" (O'Reilly) - you really need to get this book if you are responsible in any way for Perforce usage at your company! Her quote which sums things up nicely is:

"There is no reason that the repository structure should match the release lineage, but when it doesn?t it causes confusion to new users [and cognitive dissonance to all users]."

Now while I agree with the intent of what Laura has written, I find her suggested option is perhaps a little too restrictive, and it doesn't take into account organisations with many branches at a particular level (and indeed many different types of branches).

Standardise Naming Conventions - Make Branch Directories Obvious

Laura gives a very simple example in her book of naming convention - just use uppercase for the branch component. Thus she has:

//depot/MAIN/...
//depot/REL1/...
//depot/SOME_PROJECT/...
//depot/ANOTHER_PROJECT/...

This is simple and can work very well (although it rather suggests that you don't use uppercase for any other paths which might be unnatural in certain companies for certain items). Another alternative is to append (or prepend) something to the name indicating that it is a branch directory, e.g. ".branch" or or "-branch", or ".br" for those who don't like typing! Thus the above becomes:

//depot/main.br/...
//depot/rel1.br/...
//depot/some_project.br/...

I am happy to have other suggestions recommended!

Sensible Sorting

Directory (and thus branch) names should sort sensibly at any point in the hierarchy, thus

  • use leading zeros for numeric components, such as 01 instead of 1 if you are going to have more than 10, or indeed 4, 5 or 6 leading zeroes where necessary
  • reverse the order of dates using yyyy-mm rather than mm-yyyy

Examples:

//depot/main.br/...
//depot/rel_01_00.br/...
//depot/rel_02_00.br/...
//depot/rel_02_01.br/...
//depot/rel_02_02.br/...
//depot/dev/2007_06_some_project.br/...

Broad & Shallow vs. Narrow & Deep

At any one level the number of choices (sub-directories/branches) should be appropriate and naturally limiting (note that the structure needs to be as useable in several years time as it is now). For example, a structure which ends up with thousands of branches in a single directory will become unwieldy over time. Divide these branches into sub-directories using appropriate naming (and sorting). For example if you wish to create lots of branches for individual change requests then the obvious structure is very wide and shallow at some point in your repository:

~/rel 	/CR000001/...
	/CR000002/...
	:
	/CR000099/...
	/CR000100/...
	:
	/CR009100/... 
	:

Instead of that, try:

~/rel 	/CR0000xx	/01/...
			/02/...
			:
			/99/...
	/CR0001xx	/01/...
			:
	/CR0091xx/	/01/... 
			:

This naturally splits things up and also sorts at all levels.

As a corollary of the above you should split things up even if they don?t have a ?natural? dividing component. One alternative is to use the date as an element of the path (or even a prefix) as YYYY or YYYY-MM to give some sort of natural sorting by date. Examples being:

~/2006-01/some_project/...
Or without introducing another level but using a prefix (which is often just a matter of personal preference):
~/2006-01-another_project/...

In 2 or 5 years time it should be obvious to any user which are the which are the ?new? ones where they are likely to be spending most of their time, and which are the ?old? branches which can typically be ignored.

Hiding or Retiring Old Branches

It is possible ?hide? old branches by removing read permissions on those repository paths (leaving them visible to superusers or perhaps an "archive" user). There are some potential performance implications to doing a lot of this (the more lines you add to the protections table the more work the server has to do for commands).

Often the easiest way is just to make them easily ignored via an appropriate hierarchical naming convention as shown above.

Provide Guidelines and Delegate Responsibility

Larger organisations may have a number of teams or units each of which is responsible for products with sometimes quite different lifecycles and thus branching schemes. In these cases it makes sense to set the guidelines and principles and delegate to the teams the work of defining the precise conventions to be used.

Automation

If your repository structure is simple enough and regular enough then it is easy for tools or scripts to:

  • create them (perhaps with input from a user), including adding a mapping to the current client workspace view
  • have triggers that validate that they are being created in the correct place (e.g. don't allow a branch with wrong naming convention to be created in certain directories or levels of the repository). This prevents user error.
  • work out the relationship between branches automatically and thus be able to produce simple reports for things like changes that need to be propagated etc. While this is often done by means of some configuration file (stored in the repository), and indeed this is sometimes necessary anyway, it is easier if it can be automatically deduced from the structure (so that a new branch "popping up" is automatically included into the reporting mechanism).
  • make your protections table easier to maintain

There is No One Best Design!

And remember that this is always true - there are always many ways that are good enough, and the final choice often comes down to personal preference.

However, every type of branch should have a ?home?, so that when a user creates a type of branch your CM Plan should have simple guidelines so the user is never at a loss as to where to create one. This requires identifying the types of branches likely to be used.

You need to do enough thinking and planning to get at least 80% of it right. It is of course possible to re-structure your repository (by branching from old name to new name and deleting the old within the same atomic changelist). However, this can cause you quite a few problems if the branch you are re-structuring has relationships with lots of other branches (e.g. release branches being maintained).

People who are aware of the issues and have experience, can slot in a new branch type fairly easily (though they can still end up making sub-optimal choices that they regret).

June 8, 2007

Robert Cowham
Robert Cowham's Weblog
» P4Web Usage and Action Codes

People ask from time to time how to browse a Perforce repository, and the easiest answer is to use p4web.

Some advantages:

  • It's free, and in browse-only mode doesn't consume extra licenses - which means that you can "publish" changes via your Perforce server to managers and other people who may not normally have a Perforce license.
  • It will "serve up" things like Word documents such that they are typically opened in Word by your browser
  • It can give "canoical" or fixed URLs for documents or other configuration items stored in Perforce, so that fixed references always get the latest versions (as opposed to a file share where it is often unclear which is the latest and if there are any other versions etc)
  • The default URL to get the head revision is very simple:
    • <p4web-base-url>/<document path within perforce without double slash at start>, e.g.
    • http://p4web-machine/depot/some/path/to/document.doc

Example of Accessing a Document

A test document has been setup in the Perforce Public Depot.

The direct URL to load the document is http://public.perforce.com/guest/robert_cowham/test/testdoc.doc 

Alternatively it can be accessed for example with the history via p4web as:

http://public.perforce.com:8080/guest/robert_cowham/test/testdoc.doc?ac=22

Bear Security in Mind

If you setup in browse only mode as a user with access to the whole repository, you may be bypassing other security requirements and protections.

Consider setting up different p4web instances with different access permissions. Putting a separate layer on top of p4web with usernames and passwords, while possible, is likely to lead to maintenance problems.

DNS Alias to keep the URL fixed

If you start referring to a Canonical URL which then becomes used inside documents, make sure that the machine where p4web is running is accessed via a DNS alias so that it can be moved easily without breaking the references.

Also consider having a setup (e.g. via Apache or IIS virtual server settings) where the default web port of 80 is used to avoid ugly port components of the path such as :8080 or some other port  being required.

Parameters

The following were culled from personal experience and postings on perforce-user (by Noah Salzman  and Nick Levine )

Action Codes
10	Changelist Detail
http://p4web.example.com:8080/12345?ac=10

14	Branch Specification
http://p4web.example.com:8080/foo_branch?ac=14

16	Label Spec
http://p4web.example.com:8080/build_tag_123?ac=16

22	Revision History
http://p4web.example.com:8080//depot/main/foo.c?ac=22

64	View Head Revision
http://p4web.example.com:8080//depot/main/rel1/foo.c?ac=64

69	Submitted Changelists
http://p4web.example.com:8080//depot/main/rel1/foo.c?ac=69
http://p4web.example.com:8080//depot/main/rel1/...@build_tag_123?ac=69

142	View Annotated
http://p4web.example.com:8080//depot/main/rel1/foo.c?ac=142

143	View Fully Annotated
http://p4web.example.com:8080//depot/main/rel1/foo.c?ac=143

Note: the single forward slash (versus a double forward slash) is  
important in URLs that do not include depot paths.
107	List Jobs  (p4 jobs)
http://clrfi.alu.org:8080/?ac=107
http://clrfi.alu.org:8080/?ac=107&jsf=Job&jsf=Status&jsf=Date&jsf=Title&jsf=Area&jsf=Priority&ft=status%3Dopen

111	Describe Job (p4 job)
http://clrfi.alu.org:8080/job000005?ac=111
URLEncode the "@" (%40) or "#" (%23), for example, show contents of head rev

//public/jam/src/Build.com#2 is
http://public.perforce.com:8080/public/jam/src/Build.com%232?ac=64 

For #1
http://public.perforce.com:8080/public/jam/src/Build.com%231?ac=64 

For label @jam2-2-0 it is:
http://public.perforce.com:8080/public/jam/src/Build.com%40jam2-2-0?ac=64 

 

March 19, 2007

Robert Cowham
Robert Cowham's Weblog
» Vinterleir 2007 with Suganuma Sensei

This is a personal review of the Norwegian Aikido Federation (NAF) Vinterleir (winter school), 15 - 18 March in Tromso. Jessica Loeb and myself formed the London contingent - unfortunately Francis and Craig who were due to come too were not able to in the end.

Tromso is a very nice small town in the north of Norway, well inside the arctic circle. At this time of year there is plenty of snow and yet the daylight is a good 12 hours (it is far enough north to range from almost no daylight mid-winter to no real darkness in summer). There is not a vast amount to Tromso, but one of its attractions is that it seems to take about 10 minutes by car to go anywhere, including being out of the town and in some lovely nature.

The Vinterleir is traditionally taught by Bjorn Eirik Olsen (6th dan), Technical Director of the NAF, but this year, in honour of 30 years of aikido in Tromso (which was the first club in Norway at all), they invited Suganuma Sensei (8th dan). He has been 11 times to Norway but always in the south for the summer camp, and I think he very much enjoyed the different surroundings of snow and nature.

This also worked out very well as they have the Doshu Moriteru Ueshiba leading the summer camp in July this year. That promises to be a rather larger affair than usual!

I have actually been half a dozen times to the Vinterleir - there was a period when I went every year, but the last time was 5 years ago - Benedict who is now 4 and a half years old changed life a little!

As ever, the welcome in Norway is always extremely friendly and the atmosphere is a huge part of the success of the camp. Most people stayed with dojo members which was very nice, and in my case, together with Geirr from Oslo, we were kindly looked after by Ingelill.

It was great to see that Suganuma sensei had brought a large party from Fukuoka - nine in all. As was mentioned at one point in the speeches, it was also very nice that the Japanese group were not just fairly senior teachers, but ranged from a 17 year old to a 70 year old, and beginners to a 6th dan.

Suganuma Sensei

The training itself was excellent as usual - very clear and precise. Suganuma sensei's teaching seems to vary very little from year to year, and is very unflashy on the surface and always seems to be oriented towards the basics in these courses. He is always very clear about techniques and gives lots of smaller pointers to particular things to watch out for ranging from use of atemi to positioning for maximum safety during nikkyo.

There are times when this can almost seem a little boring - and it certainly contrasts with Inaba sensei who spends quite a lot of time talking/lecturing during courses can also give some very interesting food for later thought and reflection (although some people find that very boring!).

Suganuma sensei is very attentive during the practice and frequently helps beginners with great patience, including taking ukemi for them to make sure they have understood.

Over the years it has become an interesting exercise to tease out more from what Suganuma sensei is demonstrating, and I was starting to get a better sense of how he uses his body and energy, and feels the connection with his partner. I still have lots of questions about the "shape" of some of his techniques which I can not replicate without expending a lot more muscle in comparison with other ways of doing the same techniques. Something to work on next time.

One very interesting point for me was when he mentioned how often we tend to provoke a negative response from our partners by inappropriate or hard techniques. This is something I am finding very fruitful for my current practice.

I will certainly be back again, although am interested to know if his teaching varies for example in his home dojo(s), although an interview with him implies that it doesn't. It was very useful to practice a little with Tokuda sensei (6th dan) and get an alternative feel from someone who has trained with Suganuma sensei for many years. A very clear use of energy and sense of sticking and controlling in some techniques. I would love to arrange a visit on a future trip to Japan. For me a lot of the interest is because of the contrast with Inaba sensei - two very inspiring, but very different, teachers.

My Class

Due to the number of people on the course and the size of the main dojo not being sufficient to accommodate everyone, people were divided up into groups and roughly a third went off for a separate class at various points. I was asked to teach one of these and had some fun doing it.

I taught a somewhat relaxed and very much feeling based class rather than technique based, mainly because this is what I am finding most interesting in my personal practice at the moment. We started with some breathing exercises, breathing out and sucking the belly in, and then rotating the hips to drop the tanden/belly forward and using this to allow a fast in-breath, but with no hint of force or strain. It is important to feel as if breathing first into the belly and not fill the upper chest, or at least not let stiffness in there. Also there can be a temptation is to "sniff" which tends to stiffen the body - I find it much easier to do with an open mouth. This exercise certainly seemed a little strange at first, but I then built on it to show some simple techniques, e.g. from katatedori, where I find that an out-breath to absorb your partner's attack/energy, followed by a quick in-breath seems to release some energy upwards which makes it much easier to lift their arm and indeed their energy.

I then focussed on keeping very soft and relaxed arms and shoulders while performing simple koky-ho from ryotedori (2 hand grab) or tenchinage, and allowing the arms to drop in a very relaxed manner without any sense of strain or pushing.

People seemed to enjoy it (after the initial confusion perhaps on the breathing and where I might be going with it!) and I had several appreciative comments afterwards which is always nice. My sources are mainly Inaba sensei's teaching regarding the breathing, and Peter Ralston's book "Zen Body-Being" for a fascinating approach to relaxation and effortless power which I am finding very rewarding to study.

Grading

I missed the kyu grading, but watched the dan grading on Saturday with interest: three shodans and two nidans, of whom I knew 4: Vegard, Steinar, Haakon and Kai Hare. The shodan grading took a good 20 minutes or more - any number of techniques being required to be demonstrated. Somewhat unnecessary in my opinion, but then Inaba sensei does rather tend to an alternative approach others find not rigorous enough. Interestingly the nidan gradings were shorter, since the requirement to work through the basic techniques was dropped in favour of a more free style approach. The candidates all did very well and deservedly passed. A nice surprise at the end was when Stein-Are Engstad was called out - he seemed very surprised and didn't know quite what was going on only to be awarded his yondan.

The Party

As always the party on Saturday night was a great success, and this year was held in Polaria (the Polar Centre) - one of the main tourist attractions in Tromso. The food was delicious and there were some very good speeches around the celebration of 30 years. Birger Sorenson was present who started the club in Tromso as a 4th kyu back in 1977. In that first group were Bjorn Eirik and Kore (both present) so we were treated to a few reminiscences of life back then (including some slightly alternative views of exactly what did go on!), and a little earlier in the evening a slide show of some photos over the years.

Suganuma sensei also spoke, and offered his congratulations. He also mentioned that in the summer camps a few years ago I had asked him the question "What was the most important thing in Aikido?" He answered that it was musubi or connection. A couple of years later, I asked him the same question, not wishing to catch him out, but genuinely interested if he had changed his mind or not. When this was pointed out to him at the time, he saw the funny side, and this year saved me the trouble of asking it again by saying that he still thought musubi the most important thing!

I realised that he had demonstrated the importance of "connections" off the mat as well - it is one of the qualities that make him the very impressive man that he is. You don't build an organisation of 100 or so dojos over 30 years if you can't establish and more importantly maintain good connections with people. There are lots of aikido teachers who are technically very capable, but very few who have the personal qualities and lack of ego of Suganuma sensei. I am sure that his many years of study of zen alongside his aikido has had a large influence.

The dancing afterwards was ably lead as ever by Bjorn Eirik and Birger Sorenson! Having been kicked out at 1am those of us who were left walked through the centre of a still buzzing Tromso, and while the main diehards went to the "Fun Pub", a smaller group of us enjoyed a beer at a slightly quieter local. The morning did however roll around rather early!

Touristy Things

The hosts had arranged a visit on Sunday morning for the Japanese, and Jessica and I tagged along. We took the cable car up the mountain on the mainland (to 420m), and were met by a couple of Sami (Lapp) hosts who gave us a reindeer sleigh ride and some coffee in a "lava" or tepee like tent. It seemed rather touristy and kitsch at first, but thanks to the really friendly and genuine guides turned out to be a lot of fun. The reindeer sleigh rides were only a quick circuit of perhaps 200m, but my ride became a touch more interesting when in response to my question "do you ever race?", much encouragement was given to the reindeer who promptly got over excited and veered off course heading for his mates down the hill. It turns out that reindeer don't like to be touched, and aren't terribly easy to control - starting out consisted of pulling the reindeer forward with them initially resisting, and then turned into a quick sprint and a leap on to the sledge as the reindeer suddenly bounded off!

Our hosts were young guys whose family apparently has some thousands of reindeer in their herd who spend the summer around Tromso (from April onwards), but they winter about 100km further south and inland where the climate is drier and the snow easier to dig for lichen/moss in. They were very friendly and enthusiastic with their explanations on details of dress and life, and it was very enjoyable. The only problem was that our time was a little limited and it would have been great to have a walk after lunch given the great weather up there.

Conclusion

A great course, leaving me feeling rather well exercised, and with some more happy memories to add to the store associated with Norway. Thanks to Bjorn Eirik and all the other members of Reimeikan.

March 13, 2007

Robert Cowham
Robert Cowham's Weblog
» 3rd BCS CMSG Conference - 15 & 16 May 2007

The 3rd BCS CMSG Conference is now open for registrations with a good lineup of speakers, and an excellent looking venue.

A number of people commented on problems getting to Cambridge (not so convenient from various parts of the UK), so hopefully Oxford will fit the bill a little better. In addition, we can up the budget a little for a slgithly more commercial venue.

Some other differences this time around:

  • PB Projects is doing the bulk of the heavy lifting in terms of organising details etc - yes it costs us, but it seems to be working fairly well. and takes some (though not all - e.g. sorting out timetable/speakers) of the load off the committee.
  • Only just realised that offering a group discount might help bring in a few more people - we more interested in having more people there than making a vast profit (haven't yet managed to get the committee fact finding trip to the Caribbean off the ground to spend our current surplus, but am working on it...)

So, the first registrations are already in, and look forward to plenty more!

Meanwhile, am hopefully getting a little breather to allow rather more blogging than of late - have a stack of topics piled up awaiting that spare moment!

 

» Microsoft, SCM and "Large" Projects

There have been some interesting posts recently on the challenges faced by Microsoft in developing Vista. It has of course fairly recently made it out of the door which in itself is a major achievement. However, it has become clear that many features originally scheduled for inclusion have been dropped (WinFS for example). The sheer challenge of developing such a vast system is immense, and a key part of any such development effort is the change, configuration and release management.

Some fairly recent blog postings have highlighted some of the issues involved, and seeing as SCM is right in the mix, I though it worth a review.

Overview of Vista - Paul Thurrott's Supersite for Windows

The sheer size and scope of Windows Vista makes it difficult to review, to digest, and to understand. If you step back too far, it doesn't look very impressive at all: It's like XP with a spit-shine. But if you get too close, it's easy to get lost in the seemingly never-ending lists of new features.

A previous article by Paul bemoaned various lost features.

Joel Spolsky wrote an article How many Microsofties does it take to implement the Off menu? reflecting on UI choices and how less is usually more.

This attracted a comment (or was it coincidental): Moishe Lettvin's Windows Shutdown Crapfest posting:

The most frustrating year of those seven was the year I spent working on Windows Vista, which was called Longhorn at the time. I spent a full year working on a feature which should've been designed, implemented and tested in a week..

The key SCM-related part of Moishe's post is:

In small programming projects, there's a central repository of code. Builds are produced, generally daily, from this central repository. Programmers add their changes to this central repository as they go, so the daily build is a pretty good snapshot of the current state of the product..

In Windows, this model breaks down simply because there are far too many developers to access one central repository. So Windows has a tree of repositories: developers check in to the nodes, and periodically the changes in the nodes are integrated up one level in the hierarchy. At a different periodicity, changes are integrated down the tree from the root to the nodes. In Windows, the node I was working on was 4 levels removed from the root. The periodicity of integration decayed exponentially and unpredictably as you approached the root so it ended up that it took between 1 and 3 months for my code to get to the root node, and some multiple of that for it to reach the other nodes. It should be noted too that the only common ancestor that my team, the shell team, and the kernel team shared was the root.

Joel Spolsky then chimed in:

Of all the things broken at Microsoft, the way they use source control on the Windows team is not one of them...

When you're working with source control on a huge team, the best way to organize things is to create branches and sub-branches that correspond to your individual feature teams, down to a high level of granularity. If your tools support it, you can even have private branches for every developer. So they can check in as often as they want, only merging up when they feel that their code is stable. Your QA department owns the "junction points" above each merge. That is, as soon as a developer merges their private branch with their team branch, QA gets to look at it and they only merge it up if it meets their quality bar.

I think there are various interesting things about this discussion.

Managing Complexity and Reducing Dependencies

From the days of The Mythical Man Month: "Adding manpower to a late software project makes it later", it has been clear that the challenges of managing large scale systems are many. The whole book is a joy to read, and the Wikipedia articles referenced give a good highlight.

I particularly like the idea of Conceptual Integrity to help keep things simpler both in development and control.

Microsoft's products would appear almost by design to have a very high level of interdependencies between them (although this is changing: "And what happened is as the projects got larger and larger, we introduced too many complex interdependencies on early software, more so than we could really digest throughout the system," said David Treadwell, corporate vice president of the .Net Developer Platform group).

One advantage of the dependencies is that if your customer buys one product then they pretty much have to take a slew of accompanying products too.

Branches and Sub-Branches - Your SCM Tool

Back to Joel's comment that the best way to organise things is branches and sub-branches. This is a fairly classical SCM approach to the problem which has many benefits.

It is interesting to note that Joel mentions SourceDepot which Microsoft uses internally, and which is a re-badged version of Perforce circa 1999 (when win2k was in development), with some Windows specific improvements such as memory usage, but from what I can gather no fundamental algorithm improvements. Perforce of that era only had the ability to easily propagate changes between directly related parent-child branches. Things like grand-parent <-> grand-child propagation skipping over intervening parent were possible but tricky - the tool didn't default to handling it. (Note that I am interested in the original comment about there being a tree of repositories, as opposed to a tree of branches - if the former, then it makes things much more difficult to improve merging across separate repositories).

Microsoft's new Team Foundation Server tool (uses a similar branching model and terminology but totally re-architected it would appear), does not yet support good common ancestor detection either.

Over the years Perforce has addressed this in various ways, and from 2005 it is working well and with reasonable performance at large sites. Thus in Perforce you can pull changes from one sub-branch to another without going via the parent, and changes propagated back to the common parent will not cause major problems for either sub-branch. There still remains a very large question as to whether "uncontrolled" propagation of changes is a good idea (as Laura Wingerd puts it - "why we don't allow driving through hedges" - Ch7), and my advice is certainly to think carefully and plan your normal strategy (with tightly locked down exceptions maybe permissible). It still comes back to managing complexity: the more you allow changes to be propagated all over the place in an ad-hoc fashion, the harder it is to track what has gone one.

As mentioned by Joel, Accurev is a very interesting looking tools and their stream browser certainly looks attractive, and has raised the bar in terms of features for the SCM vendors. My impression is that Perforce still has the advantage in terms of scalability (see Google paper) and performance, but I would certainly put Accurev on my evaluation list if advising a client in this area.

March 10, 2007

Robert Cowham
Robert Cowham's Weblog
» 2nd BCS CMSG Conference (21 & 22 June 2005)

The 3rd BCS CMSG Conference is now open for registrations with a good lineup of speakers, and an excellent looking venue.

A number of people commented on problems getting to Cambridge (not so convenient from various parts of the UK), so hopefully Oxford will fit the bill a little better. In addition, we can up the budget a little for a slgithly more commercial venue.

Some other differences this time around:

  • PB Projects is doing the bulk of the heavy lifting in terms of organising details etc - yes it costs us, but it seems to be working fairly well. and takes some (though not all - e.g. sorting out timetable/speakers) of the load off the committee.
  • Only just realised that offering a group discount might help bring in a few more people - we more interested in having more people there than making a vast profit (haven't yet managed to get the committee fact finding trip to the Caribbean off the ground to spend our current surplus, but am working on it...)

So, the first registrations are already in, and look forward to plenty more!

Meanwhile, am hopefully getting a little breather to allow rather more blogging than of late - have a stack of topics piled up awaiting that spare moment!

 

December 22, 2006

Robert Cowham
Robert Cowham's Weblog
» Agile SCM Column December 2006

I had some fun with the Agile SCM Column at CM Crossroads this month:

A Christmas (Configuration) Carol (Abridged)

We have endeavored in this Ghostly little tale, to raise the Ghost of an Idea, which shall not put our readers out of humour with themselves, with each other, with the season, or with us. May it haunt their houses pleasingly, and no one wish to lay it.
Their faithful Friends and Servants
C.D., R.C., B.A & S.B.
Charlie's Ghost

In which the ghost of (Configuration) Charlie appears to our heroine SCM Sally to help her in her dealings with VP of Development Ben ('Benezer) Scrooge.

Continued...

Sometimes inspirations strikes, though it is interesting what turns out in the end - had a couple of different ideas (and a lot more quotations from the original), but it evolved into what you see. Not quite as provocative as a previous column:

National Treasures of Agile Development

Introduction

Recent research has discovered a very interesting cache of papers about a little known Tribe called the Agile Developers. The first document is a draft dated July 4th (the year is illegible).

(Document starts)

When in the Course of human events, it becomes necessary for some developers to dissolve the bands which have connected them with others, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.

We hold these truths to be self evident,

Continued...

October 23, 2006

Robert Cowham
Robert Cowham's Weblog
» Test Harness for Perforce Scripting

Have been meaning to do some work on P4Python recently, and the first thing I realised I should do is to update the test harness.

This is based around the unittest module so is fairly standard Python. Does some fairly standard things with test suites etc, and provides a reasonably good example of how to use the code itself, so acting as some level of documentation of P4Python.

The old version assumed a pre-existing Perforce server installed and running with some known content. This was fine for my own personal testing but had a couple of problems:

  • it assumed the training repository which is fine for Perforce Consulting Partners but which isn't available to ordinary mortals (so they couldn't run the test harness locally)
  • was a snapshot with existing users and client workspaces so required a license to use - not good for everyone
  • it required too much manual setup before running

Obviously a candidate for tidying. So recent work done:

  • Change to use the new Perforce Sample Repository which anyone can download and install (works without a license too)
  • Change to automatically create and run a new server instance on a fresh install from the sample repository download (automatically unzip etc)

The end effect is fairly nice and automated, and provides a much better ongoing resource to anyone wishing to do work on P4Python. Please note that it currently assumes Windows environment, but will insert a few checks to make platform independent shortly.

To have a look you can either:

Note that credit for various aspects should go to Ravenbrook from their work on P4DTI - I stole various techniques from their implementation of a test harness.

It also serves as a reasonable example of a test harness for Perforce scripting, and am very happy to receive comments and improvements (there are no doubt some Python gurus out there who can suggest some improvements at points in the code). I use something fairly similar as a test harness for the VSStoP4 scripts which are actually written in Perl (too horrible to write such a thing in Perl was my feeling)!

Would like to encourage people to take their own copies and give back some improvements in the general framework at least...

October 8, 2006

Robert Cowham
Robert Cowham's Weblog
» Gravity and Budo

This is a personal review of the week in Wales with Inaba sensei and other teachers organised by Tetsushinkan dojo. It is not a blow by blow account of everything that took place, but a personal summary with various highlights of things that were important for me.

First a quick overview:

  • An advanced guard of the Japanese contingent arrived on Thursday (not including Inaba sensei)
  • Udegawa san taught the normal Friday evening class at Tetsushinkan
  • Saturday everyone decamped the 4-6 hours to Wales (this included Inaba sensei and Endo san who only arrived that day from Poland where they had been teaching the previous week). Various other overseas visitors flew in that day and also drove down.
  • Sunday was the official opening and first practice
  • Standard schedule during the week
    • Morning practice 9.30 - 12.30
    • Afternoon activities in groups ranging from canoeing to horse riding to cliff walks
    • A couple of Instructor's sessions 6-7.30pm for invited people (others free to watch)
    • A couple of meetings with instructors to review links with Shiseikan, including future courses
  • On Tuesday evening we greeted Toyama guji (the Chief Priest of the Meji Jingu shrine who was "on tour" in Europe) with his entourage to Wales. He came to watch practice on Wednesday and kindly hosted a meal on the Wednesday evening for an invited group.
  • Final practice on Friday, with an excellent party that evening in a restaurant in Newport
  • Return to London on Saturday
  • Japanese all left Sunday

Summary

It was a truly excellent week. I came away with masses of food for thought and further reflection regarding both life and budo studies. It was a great pleasure to renew old friendships and to make some new friends. I also feel a renewed personal challenge to do more with these insights!

The teaching started with a formal opening ceremony conducted by Mori san, a priest from the Meiji Jungu shrine and included prayers and harai tachi - cleansing of the dojo and the spirits of those present. Inaba sensei talked about his wish to demonstrate the importance of this aspect as part of his teaching. Shinto is a relatively recent name for practices which go back thousands of years. The gods and spirits come and sit in the branches of the himorogi, part of the small shrine set up during practice. He does not view this as a religious practice and certainly has no wish to convert everyone to Shintoism! However, he wished to demonstrate the importance of setting the tone for the week - in a way a sort of misogi for all those present.

Those of you who have seen Inaba sensei will know that talking and lecturing form quite a part of his teaching. This can be a bit frustrating at first for those used to other styles of teaching where much more emphasis is placed on practice. Many of Inaba sensei's messages keep recurring over the years, but as often happens, various things made more sense to me this time - maybe I have changed in the meantime and so now hear them differently!

Why Are you Practicing?

As with the initial ceremony, this was something Inaba sensei returned to at several points. We should understand why we are practising - what do we wish to protect or defend? What are we prepared to do in the face of attack, for example on our loved ones? Answers to this question inform all the rest of our studies, and thus really are fundamental. Study and thought will be well repaid.

Relationship with Kunii Sensei

Inaba sensei gave some more details on his relationship with Kunii Sensei (the 18th generation Headmaster of the Kashima Shinryu school). This relationship was fairly short - 17 months I believe as a direct student before his death in 1966. Inaba sensei started studying aikido in the early 1960s at Hombu (and with Yamaguchi sensei). Being allowed to study with Kunii sensei was not easy, but was enabled by an introduction from Ashizu sensei, a Shinto philosopher that Inaba sensei was also studying with (and indeed that association went on for nearly 30 years). The relationship with Kunii sensei was obviously very close and Inaba sensei mentioned being asked by Kunii sensei to pass what he learnt through direct transmission and also the results of his own independent study. Inaba sensei obviously feels a very strong obligation as a result.

Developing the Body

This has long been a key concern - developing our bodies as a foundation for studies in budo. This time around there were copies of Inaba sensei's book on the subject (English and Japanese in the one volume). The English title is "Researching Japanese Budo". It originated in a series of articles for a Japanese magazine in the early '90s, and I was involved in helping revise the translation - I hasten to add that this meant reviewing the resulting English and offering some suggestions for clarifying unclear points or reworking phrases which appeared a little clumsy. The real work was done by Endo san and Annika Hansen. The original translation was done by Yamada sensei and Diane Zingale, of which a number of people received photo copies nearly 10 years ago.

Anyway, pressure of time meant that it still isn't quite where it might be in the English translation, but certainly full of very useful and interesting information - perhaps will review separately. I suggest contacting Shiseikan directly if you wish to get a copy.

Energy Inherited from your Ancestors

This was a new concept for me, and something I did not find easy to get a handle on in a physical sense. Intellectually to me it makes sense to talk about what we inherit from our direct ancestors and also the energy of the place we are in, but how to do anything physical with such energy was not clear. The exercise demonstrated was basically kokyu-ho (e.g. both hands grasped and lifting your hands and thus your partner in front of you). The difference as shown between this and normal ki energy was not clear to me in any practical way, so very much one for further research.

Kenjutsu Kata

I was very interested to see demonstrated some more dynamic ways of studying kenjutsu. Inaba sensei discussed the various kata and went through the details of the Uradachi (second set of 10 techniques) in some detail. However, as well as the classic form of the kata with uke and nage, we were also shown more dynamic forms, for example where uke more or less attacked in a kendo style and the response need to be much more freeform than in the set version of the kata. I was particularly interested as I had been thinking prior to the course that my own kenjutsu risked being a little formulaic - falling into the pattern of anticipating the technique and only being able to respond in the fixed form - even getting annoyed when the attack wasn't "just right"! Some teaching of the ukemi side were also very useful for me and others who are in positions to teach. It is important to ensure that you are applying appropriate pressure to your partner to draw their best out from them.

He did mention the requirements to study each set of kata (Kihon, Ura, Aishin, Jissen, Kassen) properly before moving on to the next, but also to place them in a context of being able to react flexibly.

Inaba sensei went through the Uradachi in some detail as part of one of the instructor's courses. Particular points of note included the requirement for both sides to treat the shinai as a real sword and thus heavy contact by uke is not appropriate - you can't lean on the blade of a katana! Normally nage leads by taking the first step, although it may appear that both people start together. It is uke's responsibility to respond to nage.

Kesa Giri Footwork

I don't think anyone present will forget the outdoors practice of kesa giri - the location was at Clydey Cottages - a place with a great lawn and stunning view over the valley. Inaba sensei really demonstrated the difference between short, restricted, inhibited kesa giri and the more expansive cuts which really started to connect with the nature all around us. The difference that this made in everyone's kesa giri was very marked, partiularly the following day back inside - you could see the expansiveness still present.

This moved on also to the use of the feet. For practical reasons of the ground being perhaps unstable, or uneven, or on a slope, he emphasized the need to step with the feet fully before completing the cut. It is easy in the clean confines of the dojo to get used to cutting with the landing of the foot and the focus point of the sword cut occurring at the same point. Outdoors however, this doesn't work - as soon as you step you risk slipping and sliding, so it is important to have fairly firm footing before completing the cut.

Battojutsu (Sword drawing)

This was another memorable day outside (though slightly more rainy) focussing on the drawing of the sword. We were all using bokuto (wooden swords) for practice but he did discuss the use of katana (live blades). He does not recommend practice with iaito (metal swords with blunt blades), as these tend to give one a false sense of security. A katana will "teach" you if you let it. It requires much more focus and attention which is good for practice and concentration. In a way, you should not use a katana too much since familiarity may breed contempt and thus lack of focus.

Various exercises and cutting to different sides (front, back and both sides) were demonstrated and practiced. In practice you shouldn't just get into the habit of thinking "OK, now I will pretend someone is attacking from behind, so let's react". It should be a much more instinctual response - wait and be alert, and the right moment will arise. Remember also flexible footwork - react off either foot.

Stepping off the Line

Figure 1 - Stepping off the line. Step1 in is the "invitation" (or sassoi) - a half step. Then you step 2 to move left foot around an finally step 3 moves the right foot again. Note that both feet are now off the line but the focus (shown by heavy dashed arrow) is on a line pointing at uke's centre which forms a triangle with the initial focus line.

This was another exercise completed outside. It is one I have seen and practiced several times over the years, and yet realised this time around that I really need to focus much more on it for my own practice.

You start a couple of paces away from uke, and then step in so as to "invite" an attach (standard punch to the stomach). As the punch comes in you need to step off the line so that you end up in a triangle with feet pointing at your opponents centre. If uke attacks right hand (and foot), then it is quite normal to initially half step with the right foot to "invite", and then step off the line with the left foot closest to uke. See Figure 1.

The alternative which we practised involves "inviting" with a half step forward of the left foot. The result is similar, but left and right feet are reversed.

This appears quite simple but is difficult to do well, and in particular getting the timing correct and being able to move your body appropriately is key. Watching Inaba sensei, he doesn't necessarily appear to move particularly fast, but it is at just the right moment and with appropriate speed.

Further developments of this exercise shorten the distance to one step, then half a step, then no steps (with uke's hand on your belly). Challenging stuff!

The Grading

This happened at the end of the week during the last session ranging from shodan to yondan (errrr that would be me...). Most people were from Tetsushinkan but some others from London and Greece, and everyone generally did very well. Results are on the Tetsushinkan events page.

As is typical with Inaba sensei's gradings they were relatively short. In each case various attacks were specified and the actual technique was not specified - indeed it was perfectly acceptable to perform the same technique several times in a row to the same attack. The important factors are spirit, body movement and effectiveness, rather than necessarily technical beauty. People also demonstrated some of the Kashima Shinryu kenjutsu kata according to the knowledge and experience of the person being graded (e.g. Kihondachi or Uradachi). This approach is certainly different to what is normally done within many standard aikikai  dojos. I did discuss this afterwards over a beer with a couple of people - "but how do you know they know all the appropriate techniques?" was one question. I obviously can't speak for Inaba sensei but I think that it is usually clear from a demonstration the sort of level a person has reached, and it is perfectly in tune with his general approach of studying relatively few techniques but in great depth. In addition, I think that this sort of approach works well in smaller organisations where the people being graded are known to the examiners - I think there might be more problems with larger scale organisations.

Instructor's Meeting - The Future

There were a couple of interesting discussions with vairous instructors and countries present, e.g. Bjorn Eirik Olsen and others from Norway, Pascal Durchon and Joel Roche from France, Anita Kohler from Germany and various others from UK etc. As with all good discussions, some heat and light at times was eventually turned into some good results and conclusions.

Inaba sensei was keen to get an understanding of how he could best support those present, and what sort of future courses and training was both sought and might be possible. He did mention that his preferred method is to teach a small group of people at the Shiseikan where he could give a lot of time and attention to them, and he made a generous offer for those prepared to arrange visits. Pascal and Anita in particular expressed a desire to give more people the opportunity to get to know Inaba sensei and experience his teaching. The rough plan for 2007 is for a seminar in France to be organised primarily by Pascal, followed by Norway in 2008 and Germany in 2009. Visits to Shiseikan are left to individual groups to arrange. I think the intention for Paris is to perhaps offer open sessions where anyone can attend, but also a more restricted session for those with previous experience and who have already made contact with Inaba sensei and are already studying his methods. This would allow for more advanced training, in particular for kenjutsu. I think it will be very interesting to see how this works, and certainly look forward to Paris next summer!

Conclusion

The course was excellent - lovely surroundings and a great atmosphere. Serious study and yet not in a heavy atmosphere. The number of seniors around (from Japan and elsewhere) made for some excellent training.

Inaba sensei is a fascinating and inspiring teacher on many levels, and it is worth taking any opportunity to get around him and learn from him. Some of his methods are a little extreme perhaps, and I am not clear in my own mind how easy it is to combine what he teaches with classic aikikai aikido. The differences go to quite deep levels. This is fine for those of us who primarily follow his teaching. Others who think it might be nice to spice up their traditional methods may find it distinctly more challenging. How much integration is possible and how much is it two separate approaches - I suspect it is going to take me at least quite a few more years to decide. In the meantime he is planning for the future. He is not going to go on for ever - seek him out and take advantage of him!

 

» Budo camp with Inaba Sensei in Wales, August 2006

This is a personal review of the week in Wales with Inaba sensei and other teachers organised by Tetsushinkan dojo. It is not a blow by blow account of everything that took place, but a personal summary with various highlights of things that were important for me.

First a quick overview:

  • An advanced guard of the Japanese contingent arrived on Thursday (not including Inaba sensei)
  • Udegawa san taught the normal Friday evening class at Tetsushinkan
  • Saturday everyone decamped the 4-6 hours to Wales (this included Inaba sensei and Endo san who only arrived that day from Poland where they had been teaching the previous week). Various other overseas visitors flew in that day and also drove down.
  • Sunday was the official opening and first practice
  • Standard schedule during the week
    • Morning practice 9.30 - 12.30
    • Afternoon activities in groups ranging from canoeing to horse riding to cliff walks
    • A couple of Instructor's sessions 6-7.30pm for invited people (others free to watch)
    • A couple of meetings with instructors to review links with Shiseikan, including future courses
  • On Tuesday evening we greeted Toyama guji (the Chief Priest of the Meji Jingu shrine who was "on tour" in Europe) with his entourage to Wales. He came to watch practice on Wednesday and kindly hosted a meal on the Wednesday evening for an invited group.
  • Final practice on Friday, with an excellent party that evening in a restaurant in Newport
  • Return to London on Saturday
  • Japanese all left Sunday

Summary

It was a truly excellent week. I came away with masses of food for thought and further reflection regarding both life and budo studies. It was a great pleasure to renew old friendships and to make some new friends. I also feel a renewed personal challenge to do more with these insights!

The teaching started with a formal opening ceremony conducted by Mori san, a priest from the Meiji Jungu shrine and included prayers and harai tachi - cleansing of the dojo and the spirits of those present. Inaba sensei talked about his wish to demonstrate the importance of this aspect as part of his teaching. Shinto is a relatively recent name for practices which go back thousands of years. The gods and spirits come and sit in the branches of the himorogi, part of the small shrine set up during practice. He does not view this as a religious practice and certainly has no wish to convert everyone to Shintoism! However, he wished to demonstrate the importance of setting the tone for the week - in a way a sort of misogi for all those present.

Those of you who have seen Inaba sensei will know that talking and lecturing form quite a part of his teaching. This can be a bit frustrating at first for those used to other styles of teaching where much more emphasis is placed on practice. Many of Inaba sensei's messages keep recurring over the years, but as often happens, various things made more sense to me this time - maybe I have changed in the meantime and so now hear them differently!

Why Are you Practicing?

As with the initial ceremony, this was something Inaba sensei returned to at several points. We should understand why we are practising - what do we wish to protect or defend? What are we prepared to do in the face of attack, for example on our loved ones? Answers to this question inform all the rest of our studies, and thus really are fundamental. Study and thought will be well repaid.

Relationship with Kunii Sensei

Inaba sensei gave some more details on his relationship with Kunii Sensei (the 18th generation Headmaster of the Kashima Shinryu school). This relationship was fairly short - 17 months I believe as a direct student before his death in 1966. Inaba sensei started studying aikido in the early 1960s at Hombu (and with Yamaguchi sensei). Being allowed to study with Kunii sensei was not easy, but was enabled by an introduction from Ashizu sensei, a Shinto philosopher that Inaba sensei was also studying with (and indeed that association went on for nearly 30 years). The relationship with Kunii sensei was obviously very close and Inaba sensei mentioned being asked by Kunii sensei to pass what he learnt through direct transmission and also the results of his own independent study. Inaba sensei obviously feels a very strong obligation as a result.

Developing the Body

This has long been a key concern - developing our bodies as a foundation for studies in budo. This time around there were copies of Inaba sensei's book on the subject (English and Japanese in the one volume). The English title is "Researching Japanese Budo". It originated in a series of articles for a Japanese magazine in the early '90s, and I was involved in helping revise the translation - I hasten to add that this meant reviewing the resulting English and offering some suggestions for clarifying unclear points or reworking phrases which appeared a little clumsy. The real work was done by Endo san and Annika Hansen. The original translation was done by Yamada sensei and Diane Zingale, of which a number of people received photo copies nearly 10 years ago.

Anyway, pressure of time meant that it still isn't quite where it might be in the English translation, but certainly full of very useful and interesting information - perhaps will review separately. I suggest contacting Shiseikan directly if you wish to get a copy.

Energy Inherited from your Ancestors

This was a new concept for me, and something I did not find easy to get a handle on in a physical sense. Intellectually to me it makes sense to talk about what we inherit from our direct ancestors and also the energy of the place we are in, but how to do anything physical with such energy was not clear. The exercise demonstrated was basically kokyu-ho (e.g. both hands grasped and lifting your hands and thus your partner in front of you). The difference as shown between this and normal ki energy was not clear to me in any practical way, so very much one for further research.

Kenjutsu Kata

I was very interested to see demonstrated some more dynamic ways of studying kenjutsu. Inaba sensei discussed the various kata and went through the details of the Uradachi (second set of 10 techniques) in some detail. However, as well as the classic form of the kata with uke and nage, we were also shown more dynamic forms, for example where uke more or less attacked in a kendo style and the response need to be much more freeform than in the set version of the kata. I was particularly interested as I had been thinking prior to the course that my own kenjutsu risked being a little formulaic - falling into the pattern of anticipating the technique and only being able to respond in the fixed form - even getting annoyed when the attack wasn't "just right"! Some teaching of the ukemi side were also very useful for me and others who are in positions to teach. It is important to ensure that you are applying appropriate pressure to your partner to draw their best out from them.

He did mention the requirements to study each set of kata (Kihon, Ura, Aishin, Jissen, Kassen) properly before moving on to the next, but also to place them in a context of being able to react flexibly.

Inaba sensei went through the Uradachi in some detail as part of one of the instructor's courses. Particular points of note included the requirement for both sides to treat the shinai as a real sword and thus heavy contact by uke is not appropriate - you can't lean on the blade of a katana! Normally nage leads by taking the first step, although it may appear that both people start together. It is uke's responsibility to respond to nage.

Kesa Giri Footwork

I don't think anyone present will forget the outdoors practice of kesa giri - the location was at Clydey Cottages - a place with a great lawn and stunning view over the valley. Inaba sensei really demonstrated the difference between short, restricted, inhibited kesa giri and the more expansive cuts which really started to connect with the nature all around us. The difference that this made in everyone's kesa giri was very marked, partiularly the following day back inside - you could see the expansiveness still present.

This moved on also to the use of the feet. For practical reasons of the ground being perhaps unstable, or uneven, or on a slope, he emphasized the need to step with the feet fully before completing the cut. It is easy in the clean confines of the dojo to get used to cutting with the landing of the foot and the focus point of the sword cut occurring at the same point. Outdoors however, this doesn't work - as soon as you step you risk slipping and sliding, so it is important to have fairly firm footing before completing the cut.

Battojutsu (Sword drawing)

This was another memorable day outside (though slightly more rainy) focussing on the drawing of the sword. We were all using bokuto (wooden swords) for practice but he did discuss the use of katana (live blades). He does not recommend practice with iaito (metal swords with blunt blades), as these tend to give one a false sense of security. A katana will "teach" you if you let it. It requires much more focus and attention which is good for practice and concentration. In a way, you should not use a katana too much since familiarity may breed contempt and thus lack of focus.

Various exercises and cutting to different sides (front, back and both sides) were demonstrated and practiced. In practice you shouldn't just get into the habit of thinking "OK, now I will pretend someone is attacking from behind, so let's react". It should be a much more instinctual response - wait and be alert, and the right moment will arise. Remember also flexible footwork - react off either foot.

Stepping off the Line

Figure 1 - Stepping off the line. Step1 in is the "invitation" (or sassoi) - a half step. Then you step 2 to move left foot around an finally step 3 moves the right foot again. Note that both feet are now off the line but the focus (shown by heavy dashed arrow) is on a line pointing at uke's centre which forms a triangle with the initial focus line.

This was another exercise completed outside. It is one I have seen and practiced several times over the years, and yet realised this time around that I really need to focus much more on it for my own practice.

You start a couple of paces away from uke, and then step in so as to "invite" an attach (standard punch to the stomach). As the punch comes in you need to step off the line so that you end up in a triangle with feet pointing at your opponents centre. If uke attacks right hand (and foot), then it is quite normal to initially half step with the right foot to "invite", and then step off the line with the left foot closest to uke. See Figure 1.

The alternative which we practised involves "inviting" with a half step forward of the left foot. The result is similar, but left and right feet are reversed.

This appears quite simple but is difficult to do well, and in particular getting the timing correct and being able to move your body appropriately is key. Watching Inaba sensei, he doesn't necessarily appear to move particularly fast, but it is at just the right moment and with appropriate speed.

Further developments of this exercise shorten the distance to one step, then half a step, then no steps (with uke's hand on your belly). Challenging stuff!

The Grading

This happened at the end of the week during the last session ranging from shodan to yondan (errrr that would be me...). Most people were from Tetsushinkan but some others from London and Greece, and everyone generally did very well. Results are on the Tetsushinkan events page.

As is typical with Inaba sensei's gradings they were relatively short. In each case various attacks were specified and the actual technique was not specified - indeed it was perfectly acceptable to perform the same technique several times in a row to the same attack. The important factors are spirit, body movement and effectiveness, rather than necessarily technical beauty. People also demonstrated some of the Kashima Shinryu kenjutsu kata according to the knowledge and experience of the person being graded (e.g. Kihondachi or Uradachi). This approach is certainly different to what is normally done within many standard aikikai  dojos. I did discuss this afterwards over a beer with a couple of people - "but how do you know they know all the appropriate techniques?" was one question. I obviously can't speak for Inaba sensei but I think that it is usually clear from a demonstration the sort of level a person has reached, and it is perfectly in tune with his general approach of studying relatively few techniques but in great depth. In addition, I think that this sort of approach works well in smaller organisations where the people being graded are known to the examiners - I think there might be more problems with larger scale organisations.

Instructor's Meeting - The Future

There were a couple of interesting discussions with vairous instructors and countries present, e.g. Bjorn Eirik Olsen and others from Norway, Pascal Durchon and Joel Roche from France, Anita Kohler from Germany and various others from UK etc. As with all good discussions, some heat and light at times was eventually turned into some good results and conclusions.

Inaba sensei was keen to get an understanding of how he could best support those present, and what sort of future courses and training was both sought and might be possible. He did mention that his preferred method is to teach a small group of people at the Shiseikan where he could give a lot of time and attention to them, and he made a generous offer for those prepared to arrange visits. Pascal and Anita in particular expressed a desire to give more people the opportunity to get to know Inaba sensei and experience his teaching. The rough plan for 2007 is for a seminar in France to be organised primarily by Pascal, followed by Norway in 2008 and Germany in 2009. Visits to Shiseikan are left to individual groups to arrange. I think the intention for Paris is to perhaps offer open sessions where anyone can attend, but also a more restricted session for those with previous experience and who have already made contact with Inaba sensei and are already studying his methods. This would allow for more advanced training, in particular for kenjutsu. I think it will be very interesting to see how this works, and certainly look forward to Paris next summer!

Conclusion

The course was excellent - lovely surroundings and a great atmosphere. Serious study and yet not in a heavy atmosphere. The number of seniors around (from Japan and elsewhere) made for some excellent training.

Inaba sensei is a fascinating and inspiring teacher on many levels, and it is worth taking any opportunity to get around him and learn from him. Some of his methods are a little extreme perhaps, and I am not clear in my own mind how easy it is to combine what he teaches with classic aikikai aikido. The differences go to quite deep levels. This is fine for those of us who primarily follow his teaching. Others who think it might be nice to spice up their traditional methods may find it distinctly more challenging. How much integration is possible and how much is it two separate approaches - I suspect it is going to take me at least quite a few more years to decide. In the meantime he is planning for the future. He is not going to go on for ever - seek him out and take advantage of him!

 

September 21, 2006

Robert Cowham
Robert Cowham's Weblog
» Perforce European Conference 19 September 2006

Life has been just a touch busy recently having been flat out on various client projects pretty much over the whole summer (managed a week away but only just!). All grist to the mill for future blogging, so hopefully a variety of articles to come!

Meanwhile one of the things I was doing was preparing and then giving a presentation for the (first) Perforce European Conference on 19th September in central London.

I think the papers will be out pretty shortly on the Perforce site, but meanwhile a few highlights and personal notes. There were some big names present and it was good to hear about various practices and principles in operation.

Keynote

Christopher Seiwald did a variation on his slightly "aw shucks" style keynote. Some key points:

  • Perforce doing fine: 200,000 users and 4,000 companies
  • Company motto: "Aim low and hit!" (do one thing well, remain best of breed and wait for the analyst pendulum to swing back to best of breed rather than suite integration, which it seems to do on a regular basis)
  • Working on a variety of things for future world domination, but don't want to pre-announce as usual
  • Very pleased with the way things are happening in Europe, and obviously at the response to this event.
  • Next US Conference 9 - 11 May 2007, Las Vegas.
  • Sydney office now opened to give global timezone support coverage!

Symbian

Deepak Modgill did a nice presentation on the challanges faced by Symbian for their offshoring. Another in the Symbian series of how their business and vairous configuration management practices have evolved. Not deeply technical but interesting never-the-less.

SAP

Obviously a flagship site for Perforce. Thomas Kroll and Claudia Loff did a good presentation. Interesting how much process and tools they had wrapped around Perforce. A few key stats:

  • 4,800 users
  • 80+ Perforce servers (but all on same cluster hardware)
  • Fujitsu Siemens clusters with 32Gb RAM running SunOS 9
  • SAN (mirrored) for main data

They use a very structured process (repository structure and branching scheme) and a parallel (P4SAP) system with its own database to record things like changes and migrations (they call them transports) of releases between different servers. There is also a layer P4MS (Management System) to handle users etc.

Quite impressive.

Process Automation

Obviously my talk was wonderful! I was thought fairly pleased with how it went down and got some good comments afterwards. For anyone interested, the Ruby triggers framework and a couple of utilities are in my area of the Perforce Public Depot.

I will no doubt be blogging on various related aspects (that I haven't already touched on).

Bank of America

Good talk by Sean Cody and Kevin Breidenbach about different approaches with the bank. They have been replacing ClearCase with Perforce in various groups, mainly due to the performance for shared development between US, UK and India. Experience of Multisite sometimes taking hours to "sync up", vs. 10-20 minutes max in Perforce.

Another feature of the talk was the power of continuous integration.

Google

Dan Bloch discussed Google's use of Perforce and in particular how they manage issues around Perforce database locking and identifying and bumping off rogue commands.

Some more stats:

  • 3,000+ users
  • Single Perforce Repository
  • HP DL585 4-way Opeteron with 128Gb RAM
  • Linux 2.4 and NetApp filer

Sounds like it wins the contest for largest number of users against a single server!

The details of the lock identification was very interesting and Dan said he would be releasing the lock.pl script and some docs on the Public Depot real soon now!

Perforce 2006.1 Update

A very interesting and technical talk by Michael Shields regarding a variety of performance optimisations made between 2005.2 and 2006.1.

Summary: 2006.1 is quite a bit faster!

Read the slides for more details.

Laura Wingerd

Laura did another fairly technical talk on what has happened to the branching/merging algorithm, and more particularly common ancestor detection algorithm used in various releases of 2006.1. In her usual inimitable style she came up with some very useful ways of explaining things like convergence and divergence of branches over time. Things got decidedly more technical with discussions on common ancestors and I was left knowing I have to go through some of this in detail in a quiet moment just to make sure I really do understand it! The changes with 2006.1 look good, but I did get the impression some edge cases could give some slightly surprising results if you don't know what's going on behind the covers (and indeed the driving intentions behind the algorithm).

Summary

Venue worked very well for location. Networking with both Perforce people and various other delegates was as ever a highlight.

Unfortunately the room booked was not huge which meant the event sold out well ahead of time - a shame a more flexible venue wasn't chosen, but that was only quibble. Organisation well run.

An excellent day!

June 30, 2006

Robert Cowham
Robert Cowham's Weblog
» Perforce and Keyword Expansions

There are times when you receive a third party code drop which you wish to import. The classic method is documented in Tech Note 15 and its reference to working disconnected (Tech Note 2). The techniques mentioned work very well to find new files, deleted files and changed files.

There is sometimes a fly in the ointment to do with keyword expansion. This is things like a CVS code drop containing expanded keywords:

$Id: //depot/robertcowham.com/main/blog/data/scm/p4_handling_keywords.html#1 $
The Perforce equivalent of this might be:
$Id: //depot/robertcowham.com/main/blog/data/scm/p4_handling_keywords.html#1 $

The simple command to find differences is "p4 diff -se". If your local version has Perforce keyword expansion turned on then you will get a load of files spuriously identified as having changed where the only real change is in the keywords.

Thus we want a simple script to run through the diffs and exclude any diffs where only keywords are found (note that this includes where the keyword is embedded, such as in a static variable assignment).

The following simple script is a good base for this. It does the job, and performs pretty well, handling thousands of files in a few minutes. It makes use of unified diff format where changed lines have a prefix in the first character of the output.

# Script to import a set of changed files with existing keywords already expanded
# (either Perforce or CVS).
# Does "diff -se" and processes the output

# Args: current directory to check
  
require 'P4'

p4 = P4.new
p4.tagged
p4.connect

def process_file(p4, f)
  diffs = p4.run_diff("-f", "-du", f)
  real_diffs = Array.new
  diffs.each { |line|
    case line
    when /^====/
    when /^\@\@/ 
    when /^ /
    else
      if line !~ /\$Id|\$DateTime|\$Revision|\$Date|\$Author|\$Name|\$RCSfile|\$Source/
        real_diffs << line
        # puts f, line
      end
    end
  }
  if real_diffs.size > 0
    print "Editing #{f}\n"
    p4.run_edit(f)
  end
end

all_files = p4.run_diff("-se", ARGV[0])
print "Processing #{all_files.size}\n"
i = 0
all_files.each{|f|
  i += 1
  # print"Processing #{i}\r" if i % 10 == 0
  # print"Processing #{f['depotFile']}\n"
  process_file(p4, f)
}

It is pretty easy to run, e.g.:

diff_se.rb ...

The net result will be a list of files checked out (p4 edit) in the default changelist.

Note that one of the big advantages of Perforce branching and merging is that it handles merges neatly when keyword expansion is used between branches (and thus you don't get spurious conflicts).

CVS Imports

If you use the cvs2p4 scripts to import a CVS repository you can end up with a slight problem since the conversion copies the CVS archive files (in RCS format) and Perforce uses them unchanged. The problem comes about because CVS stores the keywords already expanded in the RCS archive. Perforce stores its RCS files with the keywords not expanded, which makes it easier for it to do the merging between branches (without keyword conflicts). While Perforce can handle a CVS archive with the the keywords "pre-expanded", it does lead to spurious merge conflicts. Note that this problem is only really present during the early merges after the CVS import. It will no longer be present as soon as the base file for any merge is fully in Perforce format (i.e. after at least one merge has been done).

» Perforce Triggers

Writing good Perforce triggers, and, more importantly, debugging them in live use, turns out to be one of those things that seems simple but has lots of tricky issues that can lead to lots of time being wasted.

In spite of thinking that I understood lots of the issues, I still spent a couple of hours recently debugging a problem that turned out to be a combination of environment and password issues. This was particularly annoying as I had rather though I knew about this stuff (and indeed have advised people over the years about it!), and yet was blindsided and caught out by some issues I had forgotten about or not thought through deeply enough.

I reserve the right to revisit this subject more than once in the future with further insights and news...

Assume Nothing About The Environment!

The classic approach to triggers is to write a nice script (Python or Ruby for me these days - no Perl, though just occasionally I miss it!) and debug it by running with the appropriate parameters from the command line (e.g. create a pending changelist and pass in the pending changelist number). This does indeed tend to turn up a number of issues, but the good thing is you can usually debug them with the appropriate command (<rant> why does python require you to execute pdb.py which isn't by default put in the path on Windows machines, and why does Ruby not learn from Perl and for example use -d as a parameter to debug things instead of "-rdebug" - very unobvious!</rant>).

The major problem turns out to be the fact that the trigger is executed by the Perforce server process and may have a very different environment to what you might think as you run a "login" session. One sort of expects this on Unix, but on Windows it can be particularly surprising how little is in the environment due to the username that the Perforce process is running in when it is running as a service (default installation on Windows).

Thus the first rule of trigger writing is "assume nothing about the environment!".

It is very easy to forget this and assume very simple things, like:

  • P4PORT is always defined
  • P4USER is always defined
  • failures of individual p4 commands within the trigger will be obvious

Thus immediate recommendations are:

  • Give full pathnames to executables. For example, "/usr/bin/ruby" or "C:\ruby\bin ruby.exe" as the initial parameter for the ruby script, rather than assuming that "ruby" or "python" or whatever will always be in the PATH of the user executing the command.
  • When in doubt (I'm generally always in doubt) give full pathnames to scripts too.
  • Pass in as parameters the p4port and any other parameters to be used rather than expecting them to be already present in the environment.
  • Within the script, explicitly add any extra directories to the search path for commands such as "import p4" in Python or "require 'P4' " in Ruby or any equivalent import-type statement, unless you are absolutely sure that the imported libraries are globally installed on the machine your are working with. Don't assume the same directory as the trigger script itself is in is in the path unless you can prove it.
  • Trap and print to stdout (or stderr which goes to the p4d server log file) any errors/stack traces including exceptions from your p4 interface to aid hunting out problems. This is much easier to say than to do!

Passwords Cause Problems

In the good old days, before "p4 login" was even a twinkle in Christopher's eye, you could write your trigger assuming super user privileges (says in Yorkshire accent "we had it tough - could only dream of admin privileges in those days") and everything would work.

Life became substantially more complicated with security level 3 and login being required. Commands failed due to not being logged in, and this turned out to be a bit of a bugger ('scuse my French) to work out (why it had failed that is).

Received wisdom is "run your triggers as a special trigger/admin user, put that user in a special group with timeout of some very large number, log them in manually and all will be sweetness and light".

The interesting thing about this approach is that it often works, but as I discovered recently, can flatter to deceive. The problem I had was that the super user was indeed in a special "long timeout" group, and logged in on the same box (generating a suitable ticket). However, as I discovered only after some hair was torn out, the P4PORT that the user was logged in under was different to that used by trigger and thus the P4TICKET file entry was also different and the existing "login" had had no effect and my trigger was unfortunately failing silently.

Thus P4PORT=localhost:1666 where localhost=some_server.some_company.com will not work if the superuser is logged in using P4PORT=some_server.some_company.com:1666, since the latter is what will be in P4TICKET and the former will not be found and thus commands will fail. Be warned and expect/check for this!

When in doubt print out the environment within your script (via some sort of debug parameter).

Belt and Braces

My current intentions on this front are to produce a trigger framework that helps detect the above problems, and helps both avoid them and, when necessary, debug them in a (relatively) painless manner. This, at the moment of writing, is a work in progress, but I hope to be able to share it with the wider Perforce community as it emerges into the glare of publicity. I do reserve the right to retain the right of surprise to add some slight spice to my upcoming presentation at the European Perforce User Conference on the 19th September (in London).

Update: hopefully will be able to share a rework/expansion of Tony Smith's P4Trigger.rb framework which addresses some of the above issues fairly shortly - seems to be working at a client - time will tell - but fairly quickly.

Future topics will include ideas on test frameworks etc.

» Review of Pratical Perforce (Part 1)

This is a partial review of Practical Perforce by Laura Wingerd, published by O'Reilly (ISBN 0-596-10185-6). The reason it is partial is that I intend to comment in more detail in future blog articles on some parts of the book, and wanted to post this without waiting for the whole thing!

As Laura mentions in the preface, the book is not intended for complete beginners, but more for readers with experience in other SCM (software configuration management) tools who are looking to understand how Perforce works.

To quote the introduction, there are two parts to this book:

  • Part I (Chapters 1-6) is a whirlwind technical tour of Perforce commands and concepts. It's not a tutorial, nor a reference, but helpful nonetheless.
  • Part II (Chapters 7-11) describes the big picture, using Perforce in a collaborative software development environment. It is particularly strong on branching patterns, how to structure codelines and tips and tricks in this area.

The real meat of the book for most Perforce sites is thus Part II, but there are definitely some goodies in Part I.

Chapter 1 presents some fundamentals about Perforce syntax and concepts. The diagrams on pages 6 &7 explain the relationship between revisions and changelists very well.

In Chapter 2, Laura discusses client workspaces and things like view syntax. She also describes basic check outs (open for edit in Perforce command line parlance), and introduces branching when she refers to cloning of files. She includes concepts of renaming and replacing content in files, reconciling changes made offline, and even introduces a couple of bits of undocumented syntax such as "p4 files @=1452". Quite a chunk of information in this chapter.

Resolving and Merging are the subject of chapter 3 and includes some very useful diagrams showing various scenarios. If you have ever had any questions about 3-way merging in Perforce - read this! On pp68-69 she gives examples of reconciling changes you have made to a file someone else renamed using the undocumented merge3 command - interesting if a touch esoteric (also referred to in "How to undo a merge" on p80. The recommendation on p74 to sync and resolve one changelist at a time is certainly worth considering, although I think it will depend on your environment as to how necessary that is.

The basics of branching are covered in chapter 4 including initial scenarios and how to track merge requirements across branches. She makes quite a lot of use of the interchanges command (not yet exposed in the GUIs) and explains the gory details of "yours", "theirs" and "base" nicely. Her approach of using filespec integrations for the initial examples is nice and simple, but I suspect more people are likely to use branch specs in real life. On p111 she gives a useful couple of commands to show how to find which changes have been merged in (more likely to be automated in scripts for most sites I would expect). Other subjects ocvered include all the gory details of what integrate actually does, as well as some very useful details as to what the interchanges command can tell us, particular with respect to cherry picked integrations.

Chapter 5 is quite short on labels and jobs and shows all the basics. A quick note on the final section where a job is used as a reference for a changelist - as of release 2005.2 there is an undocumented "dynamic label" option where a label can have an attached revision which probably makes the job trick unnecessary.

Chapter 6 gets into the subject of remote depots and proxies and also mentions the very useful spec depot option (automatic versioning of all spec objects). There is also some good advice on using p4web in browse mode to access your repository. The section on triggers and automation is a little light, but understandable.

Part II starts with Chapter 7 "How software evolves". This chapter is perhaps the highlight of the book, and introduces concepts that are totally independent of Perforce and apply to many SCM tools. Fortunately the chapter is currently available as a free PDF document from the O'Reilly website for the book. A firm understanding of the concepts introduced here will make it much easier for you to come up with suitable branching patterns for use in your organisations, and also, perhaps more importantly, give you some incredibly useful concepts for explaining your structure to other people within the organisation. Most SCM problems are due to poor communication rather than poor tools, or poor ideas. Laura relates the problems in the real world prevent us from an overly simplistic ideal world, and yet how some simple concepts allow us to manage this real world complexity. The "flow of change" and the "tofu scale" are classic concepts which should be in everyone's SCM vocabulary.

Summary

I am going to stop this post here, and will get to further chapters and some detailed comments on them as I have time.

But I will finish with the recommendation - buy this book!!

» Regular Expressions in Ruby and Python

A personal foible perhaps but I do find Ruby's regular expression syntax remains in my brain much more easily than the Python equivalent.

Maybe it's the Ruby inheritance from Perl that makes the difference. For simple scripts I can just write standard regexps without any recourse to documentation and they just work! For example:

some_var = "prefix interesting_match some suffix"
if some_var =~ /prefix (\w+) some suffix/
  interesting_bit = $1
  print "Match found: ", interesting_bit, "\n"
end

In order to do the same in Python I find myself faffing around with the documentation (using ActiveState Python it's great to have a proper help file, but I would really like more links between the class reference and real examples of usage to help me out) and trying to remember if I want re.search or re.match and how do I get a match group and use it, etc. I have sundry Python scripts floating around that I open up and copy relevant examples from, but it does rather take time.

import re
some_var = "prefix interesting_match some suffix"
pat = re.compile('prefix (\w+) some suffix')
m = pat.match(some_var)
if m:
    print 'Match found: ', m.groups(0)[0]

Now I have to admit that it's not a huge deal in terms of the resulting code, but it took me 5-10 minutes just now to code and debug the Python version as opposed to the Ruby version which I typed in and which worked first time.

The net result is that I am noticeably  more productive in Ruby for those little scripts that make life easier, or when I am under strict time pressure. Now this is not to say that I don't like Python, or indeed that when I have a little more time I don't get use it and enjoy it. Having done some reasonably significant work in Python, e.g. rework P4DTI for PVCS (now Serena) Tracker I feel reasonably qualified to comment.

I also took the time to get sufficiently proficient in Python extensions to enhance and maintain P4Python. Mind you I now feel somewhat humbled by the most recent efforts at a Perforce integration (PyPerforce) - shows a depth of Python extensions knowledge before which I can only bow in admiration! (Minor aside - Ruby extensions are much easier to write than Python ones due mainly to the different garbage collection models).

Finishing up, I am definitely Perl-averse these days. There are a few Perl scripts that I maintain and can't be bothered, or can't find a convincing "business" case to rewrite, but anything new is Ruby or Python.

» BCS CMSG News and Events for 2006

Tools Fair on 15th June - Cradle to Grave Support

As Chair of the BCS CMSG now (Shirley Lacy passed on the baton in November but fortunately for me remains in the background as vice chair), I have been feeling a little concerned about responsibility for the BCS CMSG Tools Fair - Cradle to Grave Support on 15th June. I am relieved that things are looking good now with good selection of sponsors supporting us:

Gold sponsors
  • Marval
  • Frontrange
  • Touchpaper
  • Square Mile Systems
  • MKS
  • Serena
  • Aldon
Standard Sponsors
  • Perforce
  • Telelogic
  • Axios
  • Accurev
  • Unicom
  • SpectrumSCM
     

Some newish names to me not being hugely up on the service management/ITIL side, but change, configuration and release management are a major focus in that field so look forward to meeting and hearing about them and the issues and solutions available.

A couple of big names missing on the software side which is a shame. A couple of organisations undergoing a re-org and thus no marketing budget to spend (well not now anyway). IBM/Rational aren't there because I can't find anyone to talk to that would appear to be able to make a decision. We used to have good relations with Rational but since they were acquired by IBM can't get anything out of them (I wonder what their responsiveness is if I were looking for support or trying to buy a produce?!). If anyone knows who to contact in the UK on marketing/events I'd be grateful for a heads up. The other one I have failed to get to anyone appropriate is Microsoft - would be good to find out about Team System etc, but all enquiries get passed from pillar to post and a deafening silence is the result.

Show me the money? identifying the value of Configuration Management

The other event we have on the 27th April is an evening event which is a relatively new departure for us. This will be at London South Bank University in their conference centre (which we have used before), and is in the form of a workshop lead by David Cuthbertson. This is going to focus on selling the benefits of CM and we hope will be a useful way of bringing together both new and more experienced people in the field to share experiences. Details to go up on the web site very shortly! David has spoken several times in the past and always gotten excellent reviews. He is also skilled at