Planet SCM

We have just released Plastic SCM 1.5 Build 63.5 (internally BL063.5), and we have focused on our customer demands as a lot of you have been asking us to include Plastic SCM integrations with task tracking tools.

And that is what we have included on this new release:

• Integration with JIRA: Atlassian JIRA software provides reliable data mapping features that allow data type transformation with PlasticSCM. Companies can use JIRA to quickly improve the power of Plastic´s change management features.
• Integration with Bugzilla: This widely used open source tool reports and assigns each bug to the appropriate developer; it can be used to keep a list of “things to be done” as well as prioritize and schedule tasks. Bugzilla integration with Plastic SCM provides a strong ALM solution increasing a team´s productivity.
• Integration with Trac: Trac is an open source issue tracking system for software development projects. It uses a minimalist approach to web-based software project management. Plastic SCM integration with Trac assures minimal time and effort spent administering the system as well as complete traceability of all development work.
• Integration with OnTime: OnTime is a commercial tool provided by Axosoft which can be used to link defects, features and tasks to branches in Plastic.
  Each Plastic branch will correspond to an associated issue in OnTime: the result is a complete and powerful Application Lifecycle Management (ALM) solution.
• Integration with Mantis: Plastic SCM integration with Mantis will allow you to set up your optimum development process; it guarantees that the development team is synchronized with QA.

You can find more detailed information here.

This week we have downloaded Visual Studio 2008 Beta2, in order to evaluate new features and improvements of this famous IDE.

This new version includes a lot of interesting features, like:

• Code Analisys
• Code Metrics
• WPF designer
• Web design improvements
• ...

We have just released Plastic SCM 1.5 Build 63.3 (internally BL063.3), we have focused on correcting a few bugs but we have also introduced a couple of new features:

The first one is our Advanced Query System (AQS) which is available through the command line interface (cm query command).
If offers a more complete option to our Simple Query System: if it is in your repository, our Advanced Query System will be able to give you the information!
The new system language is SQL, so it passes SQL queries directly to your Plastic server database.
The queries can be done on one entity or several at the same time, the entities supported are any repository object such as items, branches, revisions, labels and changesets.

Have a look at this example of a complex query:

cm query
"SELECT BR1.NAME, R1.ITEMID as ITEMPATH
from REVISIONS R1, REVISIONS R2, BRANCHES BR1, BRANCHES BR2, LINKEDOBJECTS LO, LINKS L
where LO.SOURCEOBJECTID = R1.OBJECTID
and R1.BRANCHID = BR1.OBJECTID
and LO.DESTINATIONOBJECTID = R2.OBJECTID
and L.OBJECTID = LO.LINKIDand R2.BRANCHID = BR2.OBJECTID AND BR2.Name='main' AND L.NAME='merge' AND R1.ITEMID=SolvePath(c:\workspace)"
--solvePath=ITEMPATH

The result would be as follows:
NAME ITEMPATH
scm0205 c:\workspace
scm0831 c:\workspace
SCM0562 c:\workspace
SCM1371 c:\workspace
scm1755 c:\workspace


Example of a query: branches created by a certain user:




The other new feature we have included is actually an improvement on parallel development using a single branch.
Plastic support different development patterns, being “branch per task” our advised method, but if you want to work in a single branch you will enjoy this:

Every developer works from the last revision in main, if two developers check out the last revision, when the first developer checks in there is no problem, his changes are included as a new revision. But what happens when the second developer wants to check in his changes? As there as a newer revision than the one on which he was working Plastic used to advise that a merge was needed.
Now when this second developer checks in Plastic will automatically update his workspace and show a merge dialog; the result would be a new revision included changes implemented by both developers.
This improvement is not only available from Plastic GUI but also from Plastic integration with Visual Studio, so you will be faster and more productive!

Hope you enjoy our new release!

We're currently working on several open projects which will all end up creating the upcoming Plastic SCM 2.0 release.
Some of the changes and new features have been already highlighted weeks ago (what's next).
The feature I'm currently working on is the new GUI. We're about to start using it internally this week, so I expect some usability feedback coming from the team. The new appearance introduces a big change from what you've previously seen in Plastic, trying to hit several targets:

- Create a better interface to handle multiple data views
- Design it to make it attractive to users
- Simplify retrieving information from the tool, making most common operations easier than before
- Open up new integration possibilities with third party tools and even new Codice developments creating some sort of virtual canvas to place windows.

The results of a still really pre-release version can be watched here:

Linux has been a primary goal for us since the beginning of the project. We're still trying to put our GUI into the mainline (thanks to the mono folks), but the server has been up and running for months.

In fact many companies seem to be very interested on running Linux servers and Windows based clients for development. And this is exactly the scenario I'll be describing here.

Plastic 1.5 (BL063 for us) introduced several improvements in the Linux installer to make it easier to set up and configure. Now you don't need to install a separate DB backend or the mono platform, they are both included by default.


I've used a blank OpenSuse box (well, actually the vmware image at www.go-mono.org) and the process shouldn't take longer than 5 minutes, unless your hard drive is pretty sloooow.



The first step will be [obvious-comment]downloading the Linux installer package[/obvious-comment] which you can find here. The name will vary from version to version but currently it is PlasticSCM-professional-1.5.63.2-linux-installer.bin.

Once you have it on your machine adjust permissions to make it executable.



The installer is created with Bitrock InstallBuilder so it can perform both a command line mode and graphical mode installation. I don't want to have any sysadmin shouting at me so I'll follow the text based one...



So, just execute the bin file and it will start asking questions: choose the language, specify a directory, accept the license, choose the components you need to install (we're installing a server so just server components and the command line client will be enough) and then it will start copying the files. Optionally you can also install both the Eclipse and JDeveloper plugins (and if you're running on Windows the Visual Studio plugin too).





Once the files are copied (by default into /opt/PlasticSCM) you can set up both the server and the command line client, which will help us checking whether the system is up and running.



The first question will be choosing the language, and then it will jump to configuring the security working mode, which means the kind of authentication your server will work. This is a pretty important step because if you set it up wrongly your clients won't be able to connect to the server, and you will have to fix it...
So, take a look at the available options (check the image):

• Name working mode: the server will load the user list from the machine it is running on. If the clients connecting to it are running on accounts with the same name, the server will recognize them as authenticated users. As you can see it is a very easy to set up method but it will depend on how strong your network is. Identity hijacking is extremely simple if you're not a very careful sysadmin. On the other hand, it is quite useful when you just want to evaluate Plastic or need easy interoperation between Windows and Linux systems. Just make sure your user name is known to the server and you're done.
• NameID working mode: same as the previous one but matching is done using both Name and ID. On Windows systems ID is the user's SID. On Unix systems the user id. It is a bit stronger than the previous one on Windows systems but also weak in Linux groups unless you carefully control what can be attached to the network. It is ok for NIS based authentication.
• LDAP working mode: so, you have a LDAP server on your network? Well, it will be extremely easy to use for a mixed Windows/Linux scenario. You'll have to give credentials to connect to the server and be able to validate users and names. The internal ID is used for authentication. The server will need a user/passwd combination to be able to retrieve the user and ID list. This mode can also be used to connect Linux/Unix systems to Active Directory servers.
• AD working mode. AD stands for Active Directory. If you're running in AD mode on a Windows server you don't need to set up users or passwords, the server will be able to retrieve the known ones automatically. AD mode is compatible with LDAP mode: you can set up your Linux server in LDAP mode and your clients in AD, for instance.

We've just released a first preview of the upcoming Plastic SCM 2.0! You can download it here for both Windows and Linux, including a VMWare machine.

We'll be posting much more about what's new in Plastic 2.0, but right now we're interested in your opinions about the new user interface. Many already say it is one of the most beautiful Linux GUI tools ever, but now we need to be sure it is also easier to use and more effective than the previous one.

Enjoy!

We've just introduced Plastic 2.0 preview so now it is time to go through some of the new features, step by step.

The first thing you've probably noticed is the totally new GUI. We've rewritten the 1.x desktop tool trying to introduce a platform neutral look and feel.



The image above shows the Plastic SCM GUI introduced with 2.0 preview. So today I'll just focus on what's new at the GUI level.

If you download the VMWare OpenSuse machine you'll get an already preinstalled Plastic client and server which will help you follow this trip if you're not familiar with Plastic.

The first and foremost new feature is the ability to handle multiple views. The former interface was constrained to the workspace tree on the left and only one view on the right, so we decided to change it in favour of a multi-view layout, as shown on the following screenshot.

Our intention? Well, now it is a lot easier to keep the context while navigating through the information. Think about listing the available branches on a project and querying its contents. Previously you would jump from the branch list to the branch content, now the two views stay on your screen.

Now let's have a look at how to navigate the views. Well, to show the main ones you just have to click on the buttons at the top: items, branches, changesets, labels, checkouts and checkouts all users.

Suppose you're already on the branch view displaying the following content (as you would see using the sample included on the vmware image).



Right-clicking on a branch pop ups the context menu and you can view the revisions on the branch as displayed on the following image.



Another new feature is the changeset view. Changesets exist since release 1.0 but they were never first class citiziens for the GUI. Now there is a specific view to list them.



Do you want to see what has been modified on a certain changeset? Ok, it is quite easy: pop up the context menu and click on show changeset content.



A new view will be opened displaying the files and directories modified on the selected changeset.



And now let's introduce what I consider the most valuable new feature from a developer point of view: the code review helper tool. There is a handful of books and essays written about the subject, and all of them agree to some extent about the benefits of reviewing or inspecting the code. From formal inspections to peer review or even pair programming, they all seem to agree that code review is a great technique. I totally agree with them but normally the lack of proper tools make reviewing code such a boring process that we try to avoid it by all means. Well, we tried to make it easier, faster and less boring with this new review tool. Of course, if you're looking for a great book focused on code inspections take a look at Karl E. Wiegers' book. Or if you prefer a lighter but still effective reading, I would go for the classic Code Complete.

Ok, so, how do you launch the code review tool? Well, it is available from the changeset, branch and label views. From the changeset view you just have to select a changeset and click on compare changeset content.



And the code review tool will let you scroll through all the changes files or directories.



And the last (but not least) feature I'll be introducing today is the query system. Most of the new views are based on it and it eases selecting only the information you're interested on.

The sample below shows how to locate the changesets created by user hank only.



Well, and that's all for today. There are many, many other features in this new Plastic release, and I'll be introducing them in the coming weeks. Meanwhile feel free to try the new release, and please let us know what you think about the new GUI.

I've just finished setting up a Plastic BL081 (preview 2) server on Solaris.

  $ uname -a
SunOS atenea 5.11 snv_34 sun4u sparc SUNW,
Sun-Blade-1000
  $

#!/bin/sh
# all in one line
LD_LIBRARY_PATH=/usr/local/lib:/opt/firebird/lib 
  /opt/firebird/bin/fb_inet_server

  # inetconv
  # inetadm -d svc:/network/gds_db/tcp:default
  # inetadm -e svc:/network/gds_db/tcp:default
  # svcadm disable inetd
  # svcadm enable inetd
  #

<DbConfig>
    <ProviderName>firebird</ProviderName>
    <ConnectionString>ServerType=0;Server=localhost;Port=3050;
          User=SYSDBA;Password=masterkey;Database={0};
          Pooling=true;Connection Timeout=120;</ConnectionString>
    <DatabasePath></DatabasePath>
</DbConfig>

The folks at DDJ have just published one of our writings on software configuration management. This time our paper talks about the pros and cons of controlled vs continuous integration which is a hot topic for us.
Both strategies have benefits and downsides, so if you're interested... read on!

We're very proud to announce the second preview of the upcoming 2.0 release. There's a whole list of new features being unveiled in this version, so please take a careful look here: http://labs.codicesoftware.com



What would I highlight as the most important new features? Well, I'd suggest you to look into:

MySql backend: I'm specially interested on this one because it really shows one of the core's plastic features: flexibility. Plastic stores all your changes on well-known, rock solid backends instead of relying on hand-made and error-prone file based storage. So far, only Firebird and SQL Server (2005 and higher) were available, but now MySql (it doesn't need an introduction, does it? :-P) is also in the list. Why? Well, it is extremely easy to setup, it scales really well and it opens a whole new world for plastic based on its clustering capabilities... Now installing plastic in solaris or macos is also simpler.

Last week I found a piece of advertisement I found very funny: it was an ad telling readers 23K more programmers were needed each year... and of course becoming one of them was a good idea!
The ad dates back from 1971 and I found it in a very old comic book. I think it is quite nice finding such a reading!

Enjoy!

For the last three weeks I've been involved in the subversion importer development. It will be one the new features for the upcoming 2.0 release. AFAIK plastic will be one of the few commercial tools importing from svn. So I've been fighting against memory problems, performance and some svn features which don't really match with plastic concepts. Hopefully I'll be finishing it this week.

The last two days I started testing the implementation with big real svn repos. Ok, needless to say I made some testing with small ones while I was coding :-). So, it gave me some time to restart some pending tasks from the previous SPRINT. I normally prefer long tasks better than short ones, but having the chance to switch to something different while the big repos was been imported looked like having some fun.

I've been implementing some enhancements to the query system. Last week Borja finished the attributes support so now it was the time to really get the best out of it enabling some queries. So, I've modified the query system introducing the posibility to find objects associated to a certain attribute. Now you can run queries like the following one:

$ cm find branch where attribute = 'status' and attrvalue = 'open'

$ cm find merge where srcbranch = 'br:/main/branch1' and dstbranch = 'br:/main'

$ cm mkatt remark

$ cm statt att:remark rev:src\server\dispacher.cs#br:/main/task001#3 "Must be reviewed"

$ cm find revs where attribute = 'remark' and attrvalue like '%review%'

After a few conversations with some of our customers and different users of SCM systems, I have realized that in some cases it is not very clear which elements must be added under the version control. I will try to explain it in this post.

The usual thing would be to always include under the version control every resource which is part of a project, except those resulting from the compilation or generation system.

One of the questions that usually arises after a new customer starts using Plastic is ok, but what should I do with my data? Most development groups developing business applications use some soft of database backend (SQL Server, MySQL, Firebird, Oracle, whatever) and normally some sort of shared sandbox like the following:


So, when they decide they have enough problems breaking each others code, being locked by someone else’s changes, or continuously “trying to shoot a moving target”, they decide to implement some sort of version control system (or better version control system). Then they move to “stage two”, which can be depicted like:


Ok, but, the same troubles they were having with code are still present dealing with the centralized database: I make a change on the schema and then your “under development copy” stop working because it can’t handle my modification: you’re stopped until you change it (breaking mine again), or fix your code (losing your precious time)... Weren’t you supposed to work in isolation?

The answer is wonderfully explained at the book refactoring databases by Scott W. Ambler: get a private sandbox which includes the database too. If you manage to do so you’ll get rid of the database versioning hell.



Ok, but, if you’re used to directly make changes in the development database using the specific server GUI, how would you keep track of changes?

My recommendation here (and Ambler’s one, if you need to trust a real expert) is: keep your schema in DDL format! Yes, maybe it sounds too arcane for you, used to only delve into the rich SQL server user interfaces, but it will really help. Suppose your database is defined into a single (ok, or even more than one, it doesn’t really matter) file, then anyone making changes will register them in Plastic SCM, creating new versions of the files. Then merging those changes (add a new field, create a key, drop a table, whatever) will be just a matter of merging regular text files, and Plastic excels here!



Ok, but... how do I manage to set up the developer’s workspaces then? Well, my recommendation would be:

Then you get to step two: you’ve some sort of mechanism in place to upgrade from a version “A” to a version “B” without human intervention. Your software can have this feature built in (my favorite), or you can have your own “upgrade app”.



Ok, I’ll do that but, how will my software know the source and target versions? – you ask.

This question is greatly covered in the book I mentioned, and to be honest I was surprised to see there one of the things I’ve been doing for years, in different projects: you create a table inside your schema (you can name it “databaseinfo” for instance) which will hold versioning information. Then your system will be able to determine which the source database version is.

What about the destination one? Well, here my choice is: hardcode it in your software. Whether you do it embedding some sort of resource or with a simple constant, it doesn’t really matter, but it is very useful having your binaries know which is the database version they need to work.

You might be thinking now “yes, but creating a database upgrade system is not an easy task”. Of course it isn’t. If you need a really flexible one, take a look at www.red-gate.com. They have a powerful set of tools to cover all your needs.

But sometimes you don’t really need a wide-range solution, you just need to add some fields, perform some calculations or create some indexes from one version to the next: you can create your own upgrade system, which will be maintained together with the software, and which will be able to perform these very specific operations when it detects the source version is not the same as the destination one. Your upgrade system will have to check which one is the “starting point” and then apply all the rules until it hits the “desired destination version”.



So, you have a beautiful solution to move towards fully managed (in terms of version controlled) database development.

Ok, maybe some of you are actually disappointed because you expected some better integration, something you could just install and forget about all your data issues... I’m afraid there are no silver bullets and the solution will require some work on your side... The good thing is that in exchange you’ll get a better system, easier to modify and evolve...

by pablo at 10:35 AM under mono

The team have been working on the new layout algorithm for the plastic 3D tree. With long revision histories the tree currently has a cone-like behaviour, so it gets wider close to the end, which makes it more difficult to understand.
So they changed the algorithm to use some sort of espiral layout: branches are distributed spacially following an spiral-like path, which is restarted from the center of the tree on each loop to save space.
Daniel also asked to add (finally, because it is a long awaited feature) revision and branch location functionality. Date filtering has also been implemented: now you can select a date range and only the revisions in between will be created.
The new tree 3D will be integrated in BL067 and will be released together with plastic 2.0. I guess the GUI folks at Codice will first clean up the looks a little bit :-) but the new utilities will be there. Watch a preview at Codice's YouTube channel:

P.S: last friday one of the GUI 2.0 preview screencasts reached the list of the most viewed YouTube videos in Spain... among a huge amount of fútbol (soccer) material... :-)

We've just uploaded a new screencast of the new (still under heavy development) GUI system.

I was reading through some server logs last week when I detected a performance problem. One database query was taking much more time than expected. The strange thing being I was sure I'd double-checked it in the past. I repeated the test in my laptop and the same performance hit showed up (even clearer taking into account the laptop CPU difference with the server).
Ok, just to give you some numbers:

• The code took about 350ms (.NET 1.1) to execute the query and perform a reader.Read() loop until no more registers were read.
• It included both reading plus processing time: each row had about 15 fields which were read with reader["myfield"] sentences.
• The sample retrieved about 5000 rows.

Well, running exactly the same code using Mono the time reached 1800ms!! A huge performance loss!

Last week we were talking about Mono's string handling performance: while it outperforms .NET in integer based tests, it was clearly behind using strings. So I decided to make a obvious change: what if I accessed the data fields using the integer indexer instead of the string based one? I mean myfield = reader[0] instead of reader["myfield"]??

Well, then Mono took exactly the same time as .NET, which is slightly better than its string based counterpart.

So, until the string based operations get a bit optimized, you can count on the good-ol integer indexer to boost Mono based database access.

pablo

I've been focusing on performance for the last month or so. Basically I've been checking how Plastic works under heavy load.

It is very interesting to note how behavior differs from one single user accessing the server to several users running operations at the same time. I've been always concerned about speeding up plastic operations (as you've probably seen from my previous posts) and simulating hundreds of clients against a single server has been a good experience.

Ok, which was the testing scenario and how did we set it up? Well, we have several ''independent'' testing agents performing the following operations:

  $ cm mkwk -wk /tmp/-wk
  $ cd /tmp/-wk
  $ cm update .
  $ cm mkbr br:/main/task[ITERATION NUMBER]
  $ cm co file01.c
  # modify the file
  $ cm ci file01.c
  $ cm co file01.c
  # modify the file again
  $ cm ci file01.c
  $ cm co file01.c
  # modify the file (last time) 
  # (no check in is done this time)
  # go to another file and repeat the process

  # check in all the checked out files
  $ cm fco --format={4} | cm ci -
  $

Trying to reduce the number of database operations. Things get really interesting here: some optimizations which make no difference with 1 client, can save lots of seconds with a big number of client testbots.

And, what about the numbers? Well, we've tried with up to 40 machines (clients) and 1 server so far. All the clients were running Linux and have exactly the same hardware configuration. We have tried up to 200 simultaneous testbots but the regular testsuite was trying 1, 20, 35, 40 and 80 testbots against a single server. It is important to note that a testbot is not the same as a user but a number of them. I mean, a regular user doesn't create a branch, modify 30 files and check all the changes back in in less than 6 seconds, which is basically what a test bot does. So in reality we're simulating hundreds (even thousands) of simultaneous users.



How good are the results? and, compared to what? Well, my intention here is publish the entire test suite in a few weeks, so Plastic users can set up the testing environment to check how it performs on their environments before they make the buy decision, and then you'll be able to really check our numbers. What I can say right now is that we've created exactly the same tests for Subversion and some other SCM products (which I won't disclose yet) and right now (using BL079 as code base, preview 1 is BL081) we're faster than any of them in this specific scenario I described above. How faster? Well, from 3 times to 6 times depending on the load if you compare us with SVN, for instance. But, we still have to run other scenarios to gain a more complete view and provide better results.

by pablo at 10:35 AM under mono

In my previous post about performance I forgot to mention one important issue. It is an obvious one, but normally misunderstood: testbots are not real users. Even when you try to randomize their behaviour and try to make them look like humans... it is not possible, well, not at least in a so simplified scenario like the one I described. I mean, probably we could consider (hey! maybe we will in the coming months!) writing some AI code, but this wasn't the case yet.

Why I'm concerned about it? Well, a few months ago I had the chance to read an interesting book on software perfomance: release it!. It is interesting and easy to read, and it gives some good ideas about how to survive several race and performance conditions. Well, it even has a promotional video!

What did I like the best? Well, it is hard to say because as I said I liked the whole contents but I found really interesting the case studies and topics like: limit your caches, bots are not users (don't develop for QA but for real users), and all the patterns about performance, stability and so on. Yes, you probably think you already know all the tricks in the book, but I think it is good to read about them and learn a little bit more about not always so obvious topics.

And finally some good news about Plastic: we're currently in the Top Ten List at Component Source Best Sellers, fighting against well-known companies and products like SlickEdit, Telerik or DXperience... And still with 1.5!! We're excited about the upcoming 2.0 release. Part of the team is already working on the post-2.0 code already, including a new Eclipse integration...