Ever wonder if it was possible to use an SCM tool to check in changes through Email. Guess what? It is! There are many situations where a programmer may want to work on code but may not have direct access to the server. A VPN connection may not be available or just really slow. perhaps you just want to see how cool it is to use Email to manage your source code changes.
How It's Done
Plastic has a replication command which can be used to copy change history from one location to another. This can be done through a direct connection specifying your source server location and your destination server location. But; It is also possible to export a package file and this package file can be moved, copied or Emailed from the source server to the remote server and then imported into the main repository.
What You Will Need
1.) Install Plastic SCM client and server on your laptop, home desktop, or wherever your remote location is going to be.
2.) Use the replicate command to export your /main trunk, or other branches, or make a copy of the Plastic database (*.fdb in your PlasticSCM/server) directory. This step is necessary so you have a Plastic repository that resembles your central repository.
3.) Setup your home server repository using the import package from the central server, or copy in the .fdb files to start off with a copy of your work database. You can also use the replicate commands via VPN or network connection to get this initial step done.
After you have a copy of your repository from work you are ready to go. You can make changes offline, at home, wherever then export those changes and Email them to work where they can be imported by you when you get to the office or by a colleague.
It will be necessary to occasionally update your home server using the same method. You can export changes as a package and email them to yourself at home or if you are using a laptop you can occasionally synchronize your server with your work server using the replicate command (to simplify just save the commands as a batch).
Step By Step
So now your setup with a copy of your work repository and you are mobile. Let's run through the commands and steps to make this happen!
First, its recommended you use a branch at home to make your changes. It's going to be easier to create a package this way. Let's say you create a branch called Bug099 and you check in a bunch of changes and now you want to Email it to Earl in IT who can check it in for you.
Your command will look something like this:
cm replicate br:/main/Bug099@rep:default2@repserver:localhost:8084 --package=C:\bug099
Zip up the file at C:\bug099 and email it to Earl in IT. Once Earl receives the Email from you he can unzip the file to C:\bug099 and run this command to import the changes you made.
cm replicate rep:default2@repserver:localhost:8084 --import=C:\bug099
That's all there is too it. You can create a simple one line batch file to Export the package and another to Import the package. The command syntax for each is:
cm replicate srcBranch --package=packagename [--changeset=number]
cm replicate destinationRepos --import=packagename
srcBranch: a full branch spec in form
br:/NAME@rep:REP_NAME@repserver:SERVER_NAME:port
dstRepos: a full repository spec
rep:REP_NAME@repserver:SERVER_NAME:port
It can be a little tricky determining your srcBranch syntax. To verify this open up the Plastic GUI client and on the Branch Explorer right click on your srcBranch and select Permissions. The permissions window will show the full srcBranch path.
In addition to exporting a branch into a package you can also export a changeset, but I would still recommend using branches.
I've also recorded a video which walks through the process starting with setting up the home server and ends with importing the emailed change package. It's about 12 minutes long most of which is the initial setup and the export/email/import steps take about 5 minutes.
Full Screen Video
It would also be possible to automate the import of the package. All you do is setup a service that monitors a POP account or a mail server and then executes the import command on the attached package file.
In the beginning everything was flat.
Now, fortunately, it isn’t.
Background
Ok, why I’m writing this? Well, this week I had the chance to take a look at two great presentations from the OSCON 2008:
The first is from Mark Shuttleworth (Canonical, Ubuntu), titled Beyond Agile: Enabling the Next Wave of Software Development Methods, and talks about a number of very interesting topics such as Lean Software Development. You can find the slides here.
What I really liked was slide 17 where he says:
Branching and Merging
Keep trunk pristine,
Keep features flowing,
Release on demand.
And slide 20 where he mentions:
Pre-Commit Testing
I see you knocking
but you can't come in.
I also liked Code Reviews for Fun and Profit, from Alex Martelli (Google) which you can find here .
He mentions the following on slide 26:
ideally, CR mailing should happen BEFORE
actual commit/push of change-set to the
codebase -- upholds trunk/head/tip quality
Which also does look interesting, right?
So, I came up with this post.
The rules
Let’s try to find which ones are the key rules to make version control shine in the post-agile era:
Rule 1: Keep your main branch (trunk) pristine. Your main branch is your project! Don't use it as a garage (full of garbage) to park all the fixes and new features. Use branches instead!
Rule 2: Isolate changes in branches. It is a consequence of Rule #1. Place each bugfix and new feature inside its own branch. It will help keeping the main line clean, and it gives you all the power of real parallel development. It's also great for task switching and keeping track of intermediate changes.
Tip: Associate your branch with a task in your issue tracking system (Bugzilla, Mantis, Jira ...) and close the full cycle.
Rule 3: Commit often, more often than you do now! If you develop on the main branch you won't commit every 5 minutes, changes often take longer! If you have your own branch for the task, you're free to commit as many times as you want to... and then you get private change history.
Rule 4: Review changes before they go to the mainline. In main branch development (and continuous integration) you run the test suite after the changes are merged. Don't! This is too late, code is already broken! Take the best out of your SCM tool, review your changes before integrating!
The good news is: there are tools supporting this model both on the commercial and the open source worlds: plastic, accurev, git (bazaar?)...
The bad news is some of the most extended tools out there can’t.
A new era is coming... get ready!
After the previous posts describing selectors in detail, it's now time to enter the last selector frontier: multi repository selectors.
As you know plastic can manage multiple repositories. You can map each one of your projects inside a plastic repository, or go for more advanced practices like component oriented development.
Repositories can be totally independent from each other, but there're also situations in which they can be tightly related. For instance you can have shared libraries you’ve developed and you reuse between different projects. If this is the case, it can be useful to have one repository for each project, and one repository for the libraries.
But then, how can developers use the code from the libraries and the project at the same time?
Let's take a look at a very simple repository like the one on the figure. It has just a couple of files and an empty directory. Probably none of your projects looks so simple. Suppose this is the repository named “proj00”.
Then you have another repository containing your library code. It looks like the one at the following figure. This repository is named “lib_repos”.
Now we need to make the lib_repos repository available to the “proj00” developers.
Please note we’ve created an empty “lib” directory which will be used as mount point in “proj00” to plug “lib_repos”.
Take a look at the following selector:
repository "lib_repos" mount "/lib"
path "/"
branch "/main"
checkout "/main"
repository "proj00"
path "/"
branch "/main"
checkout "/main"
Remember how selector rules work:
from top to bottom, and you’ll see how we’re telling plastic: take everything from lib_repos at the main branch, but mount it at /lib. Later on it will need to resolve the /lib path, and it will be solved using the next repository rule (proj00).
If you run the following ls (with a format modifier to show the repository information) you’ll see the following.
>cm ls
br:/main#1@rep:proj00@local:8084 .
br:/main#0@rep:proj00@local:8084 file00.txt
br:/main#0@rep:proj00@local:8084 file01.txt
br:/main#1@rep:lib_repos@local:8084 lib
Note:
We used the following PLASTIC_LS_FORMAT environment variable:
LS_FORMAT="{3}@{8,-26} {4,-5} {5}"
Right, the lib directory is being loaded from the lib_repos repository.
Of course if you move inside the directory you’ll check that everything inside is from the same repos. Look at the following plastic screenshot showing the repository details of the files and directories.
Implementing a real working environment
The selector above showed how to go for main branch development on a multi-repository scenario. But you’ll probably need to implement a whole branching strategy. If so, then consider the following selector:
repository "lib_repos" mount "/lib"
path "/"
label “lib_00”
repository "proj00"
path "/"
branch "/main/task001" label “BL010”
checkout "/main/task001"
This resembles a branch per task pattern on a multi-rep scenario.
Please note we’re now mounting “lib_repos” as read-only because we’re just specifying a label and not a check out rule.
The way to use the mounted repository will vary depending on your project’s needs. It can happen than a different dev group completely manages “lib_repos”, then is ok to mount it read-only because developers at “proj00” will only use it as a “library”. Lib_repos will go through its own release cycle and the team at proj00 will only have to take care of changing the label of the mounted repository when a new release is available and approved for their project.
It can also happen that your team is responsible of both repositories. You’ve decided to split them because they’re clearly different components but there is only one release cycle for them.
Then it makes sense to follow a combined “branch and merge cycle” for the two repositories. If you’re working on task001 then it can happen you need to change code at both lib_repos and proj00. You’ll be probably using a selector like:
repository "lib_repos" mount "/lib"
path "/"
branch "/main/task001" label “BL010”
checkout "/main/task001"
repository "proj00"
path "/"
branch "/main/task001" label “BL010”
checkout "/main/task001"
Please note that:
There’re two branches /main/task001, one at each repos, and the same for labels, but you can use a naming convention (using the same name as I’m doing here) to enforce their relationship.
Going really advanced, configuring what you mount
So far we’ve seen typical mount scenarios. But, what if you need to mount inside the “/lib” directory at “proj00” something which is not at the root of “lib_repos”?
Then we’ll use the power of plastic branch inheritance to actually get the desired result.
Suppose we want to mount inside “lib” the content of the “/bin” directory. So, we want to use “/bin” as the root of the lib_repos repository. To make things easier we’ll assume there’s a “release” subdirectory inside “bin”. Check the following figure.
Let’s go to the lib_repos repository and create a branch named “/main/mount-point”.
Then let’s use regular rm and mv commands to correctly configure the “mount-point” branch as we need.
>cm co .
Checking out . ... Done
>cm rm doc src
Item doc has been removed.
Item src has been removed.
>cm co bin
Checking out bin ... Done
>cm mv bin\release .
bin\release has been moved to .
>cm ls
0 07/04/08 dir br:/main/mount-point#CO CO .
0 07/04/08 dir br:/main/mount-point#CO CO bin
0 07/04/08 dir br:/main#0 release
>cm ci bin
Checking in bin ... Done
Created changeset
cs:3@rep:lib_repos@repserver:CONRAD:8084
>cm rm bin
Item bin has been removed.
>cm ls
0 07/04/08 dir br:/main/mount-point#CO CO .
0 07/04/08 dir br:/main#0 release
>cm ci .
Checking in . ... Done
Created changeset
cs:4@rep:lib_repos@repserver:CONRAD:8084
Then we can set the following selector:
repository "lib_repos" mount "/lib"
path "/?"
branch "/main"
checkout "/main"
path "/" norecursive
branch "/main/mount-point"
repository "proj00"
path "/"
branch "/main"
checkout "/main"
Please note the following:
We’re using the /main/mount-point branch just as a way to “refactor” the directory structure, but all the contents will be loaded from the main branch of the “lib_repos” repository. Of course instead of the main branch we could be using a different one.
The purpose of the /main/mount-point branch is not being merged back into “/main”
but just hold a project reorganization. In fact we can even prevent it to be merged denying the merge permission.
Let’s set the following selector:
repository "lib_repos" mount "/lib"
path "/?"
branch "/main/task001" label “BL010”
checkout "/main/task001"
path "/" norecursive
branch "/main/mount-point"
repository "proj00"
path "/"
branch "/main/task001" label “BL010”
checkout "/main/task001"
And a little explanation:
Whenever you make changes to your code inside the “lib” directory, you’ll be placing the changes directly inside “/main/task001”, but using the directory reorganization from “/main/mount-point”.
When you merge back /main/task001 in lib_repos, you’ll be only getting the changes made on the task, and not the entire reorganization made inside “mount-point”.
That’s why plastic branch inheritance is so powerful and allows many different scenarios to be implemented.
Future work
We’re currently working on the design of new selector rules to allow different repositories to be mounted on different locations directly. Some people will find the “mount-point” branch solution helpful, but other will prefer to be able to do something equivalent just using selector rules.
We’re introducing new selector rules to be able to specify which one is the root item to be used in a mount point, for instance. This way you could specify that /bin is now the root at lib_repos.
Also we’re working on creating “workspace selector directories”, which are local directories not under source control but managed by the tool (created by the update process) and able to hold controlled code…
So, we’re open to suggestions… feel free to contact me by email if you have ideas about possible selector evolution.
Welcome back to the plastic dark side! Today we'll be moving deeper in branching concepts, and trying to explain how labels and branches work.
In the previous post I've introduced several important concepts like how selector rules are evaluated by plastic one by one, from top to bottom, and how the path specifier works.
Let's now go to the "label" selector rule modifier. But first, what's a label?
A label is simply an object inside a plastic repository. Just that. When you create a label from the command line or from the GUI, you're just creating a new object inside the repository.
Labels are useful once they're applied to revisions as you can see in the following version tree. In the tree you see that revision 20 has only one label (BL051) but revision 19 has a lot of them (which in this sample means it belongs to a number of baselines... it wasn't changed for a long time).
So to label a revision in plastic you first need to have a label, then apply the label to a revision. From the GUI the only option available is to label all the entire workspace content, which means apply the label to the revisions you have currently loaded. Why? Because users normally use labels to group together a set of revisions at specific moments like releasing a new version. And more often than not is the whole workspace what they want to label.
Let's take a look at the sample in the following figure.
You see we've a directory with a couple of entries, one of them labelled with label "BL001". So, what would be downloaded to our workspace if we set the following selector?
rep "default"
path "/"
label "BL001"
We're telling plastic:
download everything marked with BL001. So, what do you expect to be downloaded?
Nothing!
You'll get an error saying you can't load the root item.
Why? Well, remember how plastic resolves selectors: first it gets the root item and tries to find a revision for it using the selector rules. So it will try to find a revision for the root item labelled with BL001 and... yes, it can't... end of the story.
So, whether you label the root item too, or you provide a rule to load the root.
rep "default"
path "/"
label "BL001"
path "/"
branch "/main"
The previous selector solves the problem.
It will go down to the second rule to find a revision for the root item, so problem solved.
Now you understand why it is so dangerous to label separate revisions instead of the whole workspace tree, unless you know what you're doing.
Labels are very useful because they also introduce a lot of flexibility. For instance, in the following example, if you specify a selector like the previous one, you'll download the workspace a tree like the one at the bottom of the figure.
Enter the world of branches
So far we've played with revisions, changesets and labels. Let's start talking about branches.
In plastic a branch is just a revision container. When you create a new branch you're just creating a new empty object in a repository. You'll have to work with it to really put some content on it. Note this behaviour differs from systems like Subversion, CVS, SourceSafe, TeamSystem or Perforce, where branch creation actually means copying revisions to the new branch.
In plastic, you create a new branch, and it is totally empty.
How can we use it? Suppose we've just created a main branch /task001. Let's do it from the command line:
$ cm mkbr br:/task001
Note that from the GUI you normally create only child branches.
I'll discuss child branches in detail later on.
Ok, now you have task001, but, what can you do with it?
You're almost an expert in selectors, do you have any idea?
What if we set a selector like the following:
rep "default"
path "/"
branch "/task001"
co "/task001"
Yes, you're right...
The server will complain telling it can't load the root item.
So we've to find out a way to make it reach the task001 branch...
What about the following?
rep "default"
path "/"
branch "/task001"
path "/"
branch "/main"
Ok, it will work.
But we're loading the revisions from main... How can I actually create a revision on the new branch?
rep "default"
path "/"
branch "/task001"
co "/task001"
path "/"
branch "/main"
co "/task001"
Watch the selector above. Specially the second rule.
It is telling plastic: get the revisions from main, but if you've to checkout something... place the checkout at "/task001". So, as soon as you checkout a file or directory... it will go to task001, and you'll start using your newly created branch!
Please note if your second rule were something like path "/" branch "/main" co "/main" the revision wouldn't ever go to the task001... so the first rule would be useless.
So, now you've your first checkout on a separate branch... If you run a ls from the command line, or if you check the item's view on the GUI, you'll see the revision is at a separate branch.
Two important things here:
Branches and labels combined
With what we've learned so far... how would you implement a branch per task pattern? Easy?
Yes, just create a new branch for each new task, and then set up a selector like the one above. And you're done.
"Ok" - you may ask - "but what if I need to start working from an specific baseline?".
Let's go.
Suppose you want to work on task001, but your starting point must be a well-known stable baseline labelled BL059.
How would you set up your selector?
Remember: you need to take everything from BL059, unless you've something on your task branch, which will be retrieved first...
Take a look at the following selector.
rep "default"
path "/"
branch "/task001"
co "/task001"
path "/"
label "BL059"
co "/task001"
Done, right?
Then, what are child branches for??
If you think they aren't needed... you've become a real selector hacker!! Congratulations! :-)
Let's go again to the selector dealing with branches /main and /task001
rep "default"
path "/"
branch "/task001"
co "/task001"
path "/"
branch "/main"
co "/task001"
Here you're!
After some weeks using them for branch per task at the beginning of the project (long, long ago), we came up with the following:
what if we make /task001 inherit from /main somehow? It is what we're trying to do with the selector anyway...
And then child branches were born!
If you create task001 as a child of main you're saying if task001 has content, take the revisions from it, otherwise take them from main which is exactly the same as the previous selector.
But child branches make your live easier. To create a child branch you can go to the GUI or type
$ cm mkbr br:/main/task001
Then the selector to use it would be:
rep "default"
path "/"
br "/main/task001"
co "/main/task001"
Which actually saves you two lines!.
And that's basically the reason why child branches were born: to save some lines writing selectors!! (Ok, please note that by the time when child branches were introduced, selectors were still written in XML format, this happened in the plastic prehistory, before any customer have even heard about plastic... and saving some lines was very important to the lazy developers!!).
Child branches and labels
"Ok" - you say - "but what if I want to use a baseline, which is the regular way of working anyway..."
And you're right!
To use a baseline you combine the label rule inside the branch rule, like the following selector
rep "default"
path "/"
br "/main/task001" label "BL051;LAST"
co "/main/task001"
For shortness the label rule can be simplified:
rep "default"
path "/"
br "/main/task001" label "BL051"
co "/main/task001"
The good thing here is that it is very simple to write multi-branch selectors
rep "default"
path "/"
br "/main/release50/bug-fix490" label "stable040;BL050;LAST"
co "/main/release50/bug-fix490"
The branch per task rule
And now admire one of the least known plastic selector rules:
rep "default"
path "/"
branchpertask "/main/task001" baseline "BL009"
Which is equivalent to:
rep "default"
path "/"
branch "/main/task001" label "BL009"
checkout "/main/task001"
But even shorter!
Wrapping up!
Well, now you're familiar with all the core selector concepts, and you're ready to jump to our next big topic: multi-repository selectors!!
Previously we were discussing about the future of continuous integration, according to Duvall’s award-winning book and possible alternatives.
Today I’ll be focusing precisely on this topic: different alternatives to handle the integration phase.
Some will prefer to stick with the main-line development style, while others will gravitate through a more controlled (and maybe less agile if we go religious) approach.
Beyond the selected branching strategy, there will be also an integration strategy.
Let’s start with main-line development. This is probably the most well-known and spread technique in version control. How does it work? Simple, all check-ins go to the main branch, as you can see in the following figure: 
Of course it has its own advantages and disadvantages.
Main-line pros
Main-line cons
Continuous Integration standard practices try to solve all the above problems with two principles:
What you get in return is a project that evolves very fast, which is great. The problem I usually found is that teams have problems preventing the build to be broken, and they normally prefer to trade fast evolution for stability. This is not always true, of course.
Some alternatives
The alternative I’ll be talking about is the branch per task pattern. Some aliases are: Activity Branching, Task Branching, Side Branching or Transient Branching. It is probably my favorite pattern because of its flexibility, ease of use and its associated benefits. It also sits the basis for new trends in version control like stream management.
How does branch per task looks like? Take a look at the following figure:
Note you have a main-line, which has been labeled as “BL00” and then there’s a task branch there named task001.
There are three important considerations here already:
In fact you’re getting rid of the main-line style problems. The drawback here is that it is a bit more complex to understand (just a little bit!) and not every version control out there supports it (that’s why we developed Plastic!).
Rapidly you’ll be creating more branches to implement more changes, as you can see in the next image:
But the point here is not only when or how you create your branches, but when, how and who integrates them back into the main-line.
I’ll be talking about two different approaches.
Running mini-bigbangs
You’ve decided to go for branch per task and then you have your colleagues creating branches for several days. When you should integrate them all back into the main branch?
I’d normally say: no longer than a week. In fact if you let time between integration span longer than a week you’ll be normally hitting one of the biggest version control problems: big-bang integration!
Big bang integration is a very well-known and documented problem, one of the “roots of all evil” in software development, but it is still out there waiting for new victims.
How does it work? You’re working on a, let’s say, 6 months project. Then you plan 2 milestones and split your team in sub-teams. They work on separate features and they’ll be integrating their work together one week before each milestone. Sounds familiar? Well, I hope it doesn’t because it is a great recipe for disaster!
If you follow this approach instead of a week, probably your first “integration” will last much longer, and needless to say the second one won’t be better... You’ll get an amount of software which never worked together before, and you need to be sure it works... in a week! Crazy!
That’s why it is such a good idea to reduce time between integrations. I’d say integration frequency has to be inversely proportional to the amount of work your team can perform. I mean, if you’re running an small 5 developers group, maybe it is ok if you run an integration a week, but as soon as you get bigger, maybe you’ve to run them more frequently, even more than once a day!
The whole point here is avoiding integration problems. It is exactly the same rule introduced by continuous integration: if something is error prone... do it often to reduce the risk!
And remember the integration problem shouldn’t be actually merging the files and directories. If it is: switch to another version control!. The problem is that even when code compiles correctly, it can break a lot of tests or just hide an unexpected number of critical bugs. As I mentioned, the problem shouldn’t be merging. We were working with a company once which was running weekly integrations with CVS. They needed several hours to just merge the code together.
Then they switched to plastic and they’re using the “spare” time to run a whole test suite. That’s the point of integration, being sure your code works as expected, not being worried about how to merge it.
Being that said let’s take a look at how a short merge iteration looks like:
I named them “mini-bigbangs” because they’re actually big bang integrations: you take a number of separate developed code changes and merge them back together. The key is that they’re short enough so that the real “big-bang” diseases don’t show up.
Why then run this approach still, why not directly going to pure continuous integration on the main-line? Well, having your own branch for each task you develop still sounds as a very good idea, it gives you a great place to create changes, prevents mainline corruption and all the other advantages of branch per task.
You’ll then continue working and your development will look like the following figure:
Until you run again another integration and:
Is it now clearer? Remember you test suite is also a cornerstone for this integration approach. You must enforce a subset of the whole test suite to be run upon task completion (you can set up a build server to do that, polling for new finished tasks, downloading, compiling and unit testing them once they’re marked as finished).
There are some warning lights to watch here: I’ve been running this kind of integration very successfully on different projects, but it can come a time when, for some reason, integrations become a real pain. In my case it actually happened because of the test-suite: it grew too large that checking each task upon integration (something you have to do!) took too long. Then integration started to grow longer and longer, and they became a real pain, as I mentioned before.
Please note that the real problem isn’t at the version control field but at the testing ground: maybe the tests were so fragile or took too long, and they have to be somehow fixed. But anyway, let’s try to figure out some alternatives.
The first one was running staggered integrations: the developer running the “mini-big-bang” decided to group tasks together, integrate them into different intermediate integration branches, test them in parallel, and then merge them all together into the main branch.
Remember the whole point here is not speed up merging, which is already very fast, but be able to run the biggest number of tests in parallel while merging branches.
Merging branch per task and CI
If you’ve a situation with a very large test suite and you need to prevent integrations to last too long, or you want to avoid the integrator’s role (in the purest agile approach) or you go after an even faster release cycle, you can combine the branch per task technique with the continuous integration approaches.
With this approach each developer merges his own branch back into the main-line. Remember he must first (as it’s shown on the second branch) merge down from the main branch to take the latest changes, then run the tests (using a CI server, for instance) and only when they all pass merge the changes up (it will be just a “copy merge”) to main.
Then a build can be triggered on the main branch, even a long big one, and it everything goes right, a new release can be created.
Continuous integration with build branches
There are other options available. Suppose each developer merges back into a specific “build branch” like the following figure shows:
The build branch can be used to run integrations during a work-day, and then over night an automated nightly build process can run. If everything works as expected, the build branch with all the changes made during the last day can be automatically merged back into the main branch, a new release created and used as baseline for development the next day.
Wrapping up
As you can see there are several alternatives when running integrations, from the simplest to the most sophisticated ones. I’d always follow the KISS principle, and try to keep things as simple as possible. Of course this is not always the case, and then you’ve to figure out which is the best alternative for your team.
Do you want to try yourself?
If you want to try it yourself... why don't you download plastic here? There's also a Linux version available here.
What if you want to know which revisions have been created in the last days? Who made the changes, where? Or if you're looking for changes made at a certain component to locate an specific modification you'd like to review?
It is very easy now with the customizable views in Plastic 2.0.
Take a look at the following figure (click to enlarge):
I've customized the changeset view to actually display revisions. And I'm using the query system to retrieve the revisions I've created since April 1th at our main repository.
I'm then using the filter box to focus on the revisions at the directory 01nerva
The query is very simple:
find revisions where date >= '4/1/2008' and owner = 'pablo' on repository 'codice'
Of course I could go even further and try to focus on the revisions created on a given branch, or bigger than a certain changeset... and so on.
The following query will locate the revisions created at two repositories after a given date on a branch different than the main one.
find revisions where date >= '4/1/2008' and owner = 'pablo' and branch != '/main' on repositories 'codice','pnunit'
As you see, the query system is very useful to create activity reports, locate certain changes, inspect modifications and so on.
Hope it helps!
How can I set up plastic security so that developers can only see their own branches plus the main one and the maintenance?
I was asked this a couple of days ago and now I’ll try to give a detailed answer and explain, step by step, how to set it up with plastic.
Secure your rep server
The first step will be securing the repository server. Do you know what the rep server is? Well, it is just the server which is handling all the repositories, and as you know, repositories are the ones which actually contain the data (branches, revisions, items, labels...)
Why is such a good idea to secure the rep server? Plastic implements a whole security hierarchy. Everything inherits from the repserver, then the repository, then the actual objects inside each rep. Take a look at the following diagram for more information:
The first thing we’ll do is to allow access to the repserver only to our allowed users. In our case I’ll be granting access to management, development and the special users named OWNER.
To set up repository server permissions from the GUI go to the repository view, right click on a rep and go to repository server permissions....
Please note both the owner and the management group have all the permissions granted, but developers will have the view permission not set. Check the following figure:
This way the developers won’t be able to view any object they don’t have created, but we’ll grant access to the right branches, items and labels so they can work correctly. The permission granted to the owner ensures they can work on the branches they create, as the initial question asked.
Note: check you set up permissions with a user belonging to the right administrative group, management in my case.
Grant access to the code!
Right now no developer can access the code because all items inherit from the repository, and the repository from the repserver, and the repserver doesn’t have view access set for developers.
So, as an administrator go to your item’s view, right click on the root item, and go to item’s permissions as the following figure shows:
Once you’re there make sure developers get the view permission set!
Securing the branches
Now it’s the turn for the branches. In the example we want all developers to view the main branch and the branches they create. The second requirement is already achieved due to the owner permissions at the repserver level.
To actually allow developers to view the main branch and the maintenance (and any other branches you want to make available) go to branch permissions and enable the view permission.
Check everything works
Check the following figure. The plastic instance on the left is run as a privileged user, while the one on the right corresponds to a developer. Note that the developer can only list the main and maintenance branches and the one he has created, but not the one created by the admin user (/main/task001pablo) in the sample.
Wrapping up, don’t forget the labels
If you remember the security hierarchy introduced above, you’ll notice that labels won’t have view permissions either. So after creating a new one, make sure you grant the right access.
The plastic security mechanism gives users huge flexibility. The purpose of setting permissions doesn’t always need to be related with preventing unwanted access but also being to enforce certain development policies. I’ll be talking about it on a future post.
This is the true story of a break up: a sesioned SCM manager tells us why he finally broke up with is beloved system: clearcase.
It probably sounds just like a funny joke, but please read on carefully and find the hard facts behind his disappointment...
Here it goes:
After 12 years working and playing with clearcase I'm fed up with it.
The beautiful lady I fell in love with after my first months in a software development and R&D; company became a real pain for me. And a new mother in law, IBM, didn't make it any better.
At the time we met we were doing development for digital media on these nice systems made by Silicon graphics.
It was the time when a system at work was way more powerful, and much more expensive, compared to the silly 486 PC running some version of DOS I had at home. So I went to work with the idea that what I did really mattered for me and the company. After all it was before the bubble.
What I particularly liked about clearcase was the fact that it was completely invisible for the developer.
You set a view and you could start working with any tool you wanted, the sources were right there where you needed them. It was a fast and advanced environment. It had directory versioning, so you could change the structure of your code without trouble. You had advanced branching and merging, you had merge tracking and the branching strategy we used at the time was branching per task. It was possible to do this efficiently in clearcase since it was so good at branching and merging.
And it had all this 12 years ago!
And the stunning fact is that while the world evolves, clearcase chooses the completely wrong direction.
It tried with the ugliest implementation ever conceived for a SCM environment, UCM, Unified Change Management, trying to impose a ridiculous overloaded process on top of the old base. She completely ignored her old lover boy, the developer, while trying to impress those people who hated development. It broke down the whole idea of branching per task.
It gave up on having a consistent user interface on all the platforms it supports, using clearcase on the different flavors of UNIX, Microsoft windows or Linux really is a different thing. The clearcase GUI on UNIX and Linux is really bad. And why has the command line so much more features compared with the GUI? And why is Apple ignored?
The fact that there is no standard way of doing a recursive check-out or check-in, the error messages that seem to be completely unrelated with the real error, the fact that if line endings are changed in textfiles used in a crossplatform environment and the merge and compare tools that fail on these, the fact that the number of characters on a line, e.g. in an XML file is limited and makes the merger fail are all things I could handle.
But the real showstopper comes with the integrations provided to 3rd party tool like eclipse, WSAD, Visual studio etc... , these integrations are in fact only integrations to do a check-in/check-out, all other clearcase operations are a pain in these environments. Simply switching views in an IDE to a view on another branch in the same project is almost impossible, effectively making the entire branching power useless.
While at the old days, clearcase was invisible for the developer, a silent and helpful companion, nowadays it makes a developer's work difficult, slow and cumbersome. Some people even argue that going back to the stoneage of SCM practice, by using tools like CVS or Subversion, is better then using clearcase.
And right at the time when I really needed to feel young again, fresh and excited about software development, a beautiful, open product appeared on the scene, made by developers for developers, all the focus is again on how development and integration can be done in an efficient, fast and powerful way. With a consistent look-and-feel on all the platforms it supports. (it even runs on Apple, a brand ClearCase always ignored), with an easy to use branching and merging environment, with good integrations with your IDE, so switching workspaces to another branch is a very easy and fast task, with merge tracking and correct directory versioning.
In fact with all the good things of clearcase and a solution for most problems it has.
Please go and check-out plastic SCM 2.0 as fast as you can, you won't be disappointed.
We've just published a new article on DDJ, this time talking about distributed software development.
You can find the whole text online here: http://www.ddj.com/architect/208802468.
It covers several scenarios: from basic multi-site to full distributed, and it explains in detail how conflicts are solved when items are modified concurrently at different locations.
Hope you like it!
The folks at DDJ have just interviewed me about SCM Trends.
I tried to give my own view about the future of SCM and even talked about how threading and concurrency (which I've to admit is one of my favourite topics since long, long ago I first read Ben Ari's book, now there's a new edition available and also the superb Jeffrey Richter's book which is also renewed this year) have an impact in the version control field, or even better how I believe SCM can help there.
I've also focused on why C# was an important decision in the early Plastic development, and more specifically why Mono was actually the key which really opened the C# door for us!
I spent years programming in Delphi, from the first version 1.0, more than ten years ago, to the excellent releases 6 and 7. I also grew up as a developer using tools like TurboPascal so when a couple of days ago I had the chance to have a try on the latest Delphi IDE and check how it works with Plastic it was actually a very rewarding experience.
Installing Plastic plugin
When you install Plastic to be integrated with Delphi, remember to check the Visual Studio integration option in the installer. Plastic currently integrates with Delphi using the SCC API so if you don’t install the SCC plugin you won’t be able to make it run within Delphi.
Once Plastic is installed you’ll need to download and install SourceConneXion. You can obtain it from http://www.epocalipse.com/downloads.htm. I’ve tried version 3 and it works great. Remember to download SourceConneXion and not VssConnection (as I did the first time) because the latter only works with Visual Source Safe.
Open Delphi, and go to Source Control\Tools\Provider Configuration and select Plastic as your version control provider.
Now you have Plastic installed and working.
You can take a look at the Delphi configuration options for plugins (Source Control\Tools\Options) in order to set up whether you want automatic checkouts, and so on.
Adding a project under source control
I started with a very simple Win32 project containing just a form and a button as you can see on the following figure:
Then I just right clicked on my project and selected the add to source control option.
If you still don’t have a workspace at your project’s location Plastic will help you creating one. You need to place your Delphi projects inside a Plastic workspace. For normal (and complex) Delphi projects a Plastic workspace will be used to work on different ones. A workspace is just a place on your disk where you work with your projects and Plastic knows that it has to control all changes there.
Delphi creates a list of the files to be added and shows it in a dialog so you can decide which ones go in or are rejected. In my case I typed a short comment telling this was the initial code import.
Start working
Once your project is under source control you can start working. Another developer can already create a new workspace on his machine and download the code you’ve just added.
I played a little bit with the Delphi functionalities and created a DUnit testing project. I also placed it under source control. Then I decided to change on the sample test methods so I just typed a key on the file. The file was checked in so Delphi prompts you if you want to check out the file before making any further modifications.
From now on the process will be very simple: checkouts will happen automatically whenever you start typing on a protected file. To checkin your changes you just have to go to the Source Control menu or right click on your file on the Project Manager pane.
Project explorer
Under the Source Control menu there’s an interesting option: the project explorer. You can take a look at your entire project tree (very simple in my case) and check whether the files are checkout or not.
The project explorer is also very useful to select several files and run group checkins from there.
Changing a file’s type
Plastic recognizes Pascal’s file extensions but it can happen that you use a different Delphi version or that you have under version control a file which type has been wrongly identified by Plastic. I mean, maybe you have a binary file which should be identified as text or vice versa. It is important for running diffs and merges because text and binary tools are different.
If you need to modify a file type, simply go to the Plastic GUI (you can also do it from the command line, but the GUI is easier) select the file you want to change, right click on it and go to the type menu option.
You can also do it from the file history.
Labelling your source code
So far I’ve just made some modifications on the main branch so my branch explorer looks like the following figure (please note to display the branch explorer you've to go to the Plastic GUI). There are just some changesets available but no branches at all.
The first thing I’ll do is creating a baseline with the code I’ve made so far, so I have a good starting point to use during development. I mean, I have a version of my entire project which I know it works, it compiles, it passes the tests, and so on (yes, here I have just a very simple project, but you know, I describe how things should be).
To do so I go to the labels view and choose to create new label and then I type the name and comments of the new label.
Please remember in Plastic labelling is a two-step process: first you create the new label and then you apply it to your workspace. So here is the second step.
How does my branch explorer look like after the labelling? Just take a look at the following figure.
Wrapping up
So far we've seen the very basics of starting a project with Delphi and Plastic SCM. Now we know how to add a project under source control, perform basic operations, label a release...
In the next post I'll be introducing branching and merging using a Delphi project...
Creating branches
My preferred branching pattern using Plastic SCM is branch per task so that’s exactly what I’ll do now: I’ll create a couple of branches and then work in parallel, modifying the same code differently as two developers would do. I won’t care about where I make the changes and I’ll let Plastic reconcile the changes once I’m done. Hey, that’s why branching and merging is there, right?
What does it take to create a new branch in Plastic? Easy: it has nothing to do with arcane copy commands or strange instructions (well, you can type cm mkbr if you’re a command-line-aholic, of course), just go to the branch explorer, right click the branch that you want to branch off and select create child branch. Then the smart branch creation dialog will pop up:
What’s a smart branch? Well, it is just a branch which is able to remember where it is starting from at a certain point in time. To put it simple, you tell your branch where are the sources you want to modify, and you can modify this location later on. In my sample I told Plastic to create a branch named task001 starting from the label I created above: initial_version.
Note: no, you don’t need to write underscores instead of spaces in label names, you could have typed initial version and it would be a valid label name but, you know, having used old good DOS for years you never really trust it is going to work... ;-)
After your new branch has been created you can take a look at your branch explorer as you see on the figure below, and then use it to switch to it and start working on the branch.
You can also examine the branch properties and check a small scheme showing how the branch is currently configured.
Working on the first task
Let’s now focus on the first work assignment we have: modifying the form so that it has a better layout, a wider edit box and a green background. We’ll also modify the code associated to the OnClick button event to run a different action as the figure below shows.
Now I’ll run another useful tool from the Source Control menu: review changes. Running this option I get the following results which show how I have modified the Button1Click code.
So now I’m ready to checkin my first changes: I’ve modified both the .dfm and the .pas files.
I’ve just realized I don’t like the change I’ve just made in the code (I think it is not even working), so I modify again the .pas file and type the following code:
The good thing using a branch for each task is that you don’t have to worry about how many commits (or checkins) you make: just check in whenever you feel it is ok to save a copy, the code won’t go to your project’s mainline, so you aren’t affecting anyone even if your intermediate changes don’t compile, and you can still use the version control as your own versioning mechanism.
Finally I’ll be changing the form’s background to make it uglier with a money green color.
What has changed in the .dfm file? The good thing about .dfm (at least since version 2 if I remember correctly, when they were converted from a binary to a text format) is that the entire user interface is defined in a text format (yes, now we’re all used to it, but Delphi was able to do it... ten years ago!), so you can easily run diffs and merges on them.
How does my branch explorer look like after working on the same branch?
Placing a chart on the form
Ok, consider the following: before I’m done with the first programming task a second developer starts working on the application too. He has to introduce a chart in the form.
He will start working from the last known good version of our test application, which was intial_version. His branch explorer will look like this:
Please note I’ve activated the parent links for this screenshot, that’s the reason why you’re now seeing the yellow lines pointing from a branch to its starting point.
The developer adds a chart to the form which looks like this:
Please note that I’m not seeing the green background nor the other changes made by the other developer on task001 because I’m starting from the last known good baseline. Is that ok?
Well, I can hear some of you saying: “hey, but then you’ll have to reorganize the entire dfm manually after making your changes in parallel”. Believe me, you won’t!
I’ll show now a 3D version tree of the dfm file containing the GUI definition:
Yes, you clearly see that developer on task001 made two changes on the form, and the one on task002 only one... in parallel!
And to make things a bit more complicated I’ll also modify the OnClick event for the button writing the following code:
Merging changes back
It’s time to merge your changes back to the mainline. Right now your development looks like:
Now you just go to the branch explorer, right click the main branch and click on switch to branch.
Then select task001, right click on it and choose merge from this branch, and Plastic will display the merge dialog:
These are the two files you’ve modified on branch task001. There won’t be any conflict as it is the first branch we’re merging back and Unit2.dfm nor Unit2.pas have been modified on main in the meantime. In the contributors column you see the two of them are saying Source as contributor, which means there’ve been changes only in the task001 and not on main.
We run merge all and the merge is performed by Plastic.
After the merge is done the affected files are kept in checked out status so you can easily check them, compile your code and verify that everything went correctly.
You can also check the version tree of our Unit2.pas file:
It shows a new green arrow meaning a merge has been performed.
This is how merge traceability is stored.
I build the application and this is how it looks like after the task001 is integrated (ok, so far only task001 is on the main branch):
And here you can see how the branch explorer looks like after the first merge is finished and the files are checked in:
Merging task002
Merging task001 was pretty simple: the changes made on the branch were just copied into main, no real merge was done between the revisions.
But now things are different with task002 because it actually modified the same files as task001.
If you run the merge dialog right clicking on task002 branch and selecting merge from this branch you’ll see the following:
What’s different now?
Well, notice that the contributors column now says the files have been changed both in main (after merging task001) and task002.
We run the merge and we check how the dfm file is merged automatically, which is actually very good news.
The .pas file needs some help to be merged: if you remember we modified exactly the same method twice on two different ways, so Plastic doesn’t exactly know how it should solve the conflict.
The three way merge tool shows up:
What’s in there?
The source pane shows the file you’re merging from task002, the source then. The base pane shows your Unit2.pas as it was before changes were done: as it was on initial_version. And finally the pane on the right, the destination file shows the Unit2.pas as it is now on your workspace after task001 was integrated.
The pane on the bottom is the results and is proposing you to group the three blocks together, which is something you’ll need to modify.
Please note the three way merge tool is detecting 3 conflicts, but it was able to solve 2 of them automatically because they weren’t affecting the same code blocks.
I decide to stay with changes on the two tasks but get rid of the base:
Everything merged
How does my form look like after the merge? Well, as expected it has the background color of task001, the button and edit box modified there and the chart introduced in task002 as you can see below:
The great thing here is that both developers were able to focus just on the changes they had to perform, and didn’t have to worry about each other until the modifications were finished, and then Plastic took care of merging them together.
The version tree of the Unit2.pas file now looks like:
Conclusion
In my humble opinion Delphi is still one of the most powerful development environments out there, and of course it benefits from all the advantages of proper branching and merging. Plastic can be integrated with Delphi with the help of SourceConneXion and it really seems they make a great team working together!
Of course as a long time Delphi developer (I’ve even worked with Kylix for a while!) it is always a pleasure to take a look into how the folks behind the Delphi’s dev team handle the evolution of their tool.
We've published two new articles. The first one on DDJ Code talk, about Distributed Software Development, and you can find it here:
http://dobbscodetalk.com/index.php?option=com_myblog&task;=view&id;=526&Itemid;=85
And a second one introducing ways to automate Plastic actions from the command line:
http://www.plasticscm.com/support/cli.aspx
Enjoy!
I’ve been playing around with django , a pretty cool ~MVC framework in Python. To that end, I set up a Planet SCM , a feed aggregator about software configuration management. The SCM guys don’t seem to be very prolific, so I wanted one place on the web where I could go to quickly see if anyone has posted anything.
It’s using Feedjack, a django-based feed aggregator. I haven’t messed with it much, but, so far, it’s quite awesome!
If anyone knows of some other good SCM blogs, do let me know!