Planet SCM

In my earlier post on upgradeing from Beta 3 Refresh, I noted that upgrading from the RC to RTM would hopefully not require a database upgrade. However, we have taken a change since RC that will require the databases to be upgraded. The process for upgrading from RC to RTM will roughly be:

1. Uninstall RC
2. Run TfsUpgradeRTM.exe
3. Install RTM.
4. Upload Reports.

While this is more complex than we had hoped, it should still be simpler than the upgrade from Beta 3 Refresh to RC has been, especially for dual server configurations.

There's a great write-up from Bryan MacFarlane on troubleshooting Team Foundation reports and the warehouse on the TF forum.

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=154526&SiteID=1

 

If you're using Beta 3 Refresh, you may be wondering what you can do now to make the server upgrade to RTM as painless as possible. Here are a few things that will help.

• If you customize work item types, don’t create work item field reference names that have more than 70 characters, or that begin with a number.
• If it’s not too late, deploy Team Foundation Server on the hardware you’re going to use at RTM.
• If you are going to deploy to new hardware, go ahead and install Team Foundation Server on the new hardware and try it out just to make sure you don't have configuration issues. You can uninstall Team Foundation prior to upgrading.
• Keep track of the customizations you’re making to the process templates; you’ll need to do these again before you’ll be able to create new projects with your custom process templates.

I’ve included the specification below so that you can get a better idea of what the upgrade will be like and let us know if you feel like we haven’t covered your scenario. Keep in mind that this isn’t final, but the actual upgrade will be similar to what’s described here.

Allen Clark

Program Manager, Team Foundation Server

Design Goals

When V1 ships, customers must be able to upgrade their data without support from Microsoft.

• Upgrading to RTM will not be effortless, but the work required must be reasonable.
• Customers should not get into a server-down state unexpectedly.
• Minimize the impact on the Setup team.

Scenarios

The section describes a few key upgrade scenarios.

Single Server

Sid is a senior developer on a small team that adopted Team Foundation at Beta 3 and upgraded to Beta 3 Refresh. He installed TFS on a server in his office and helped the other developers and the project manager get started. He has just received the RTM CDs, and he promptly begins to install this new version of TFS (the atdt single server installation). His only option is to uninstall the previous version, so he does that and runs setup again. Setup blocks on two conditions

• He is still using the evaluation version of SQL Server 2005 that was used with Beta 3 Refresh.
• The database schema versions don’t match. Here, setup directs Sid to the MS download center for instructions on upgrading his server.

He installs the non-evaluation version of SQL Server, downloads the upgrade package, and follows the upgrade instructions, which are outlined below.

1. Download any custom process templates from Team Foundation onto disk.
2. Save a copy of each custom report external to the Reporting Services database. (Ideally, these should be checked in to source control).
3. Extract the work item changed event data so that they can be recreated after the upgrade. A SQL statement will be provided in the detailed upgrade instructions that can be executed using SQL Server Management Studio
4. Uninstall Team Foundation application and data tiers.
5. Run TfsUpgrade.exe on the application tier.
6. Install Team Foundation application and data tiers, and at least one client.
7. Smoke test the installation using the instructions in the upgrade download package.

Several team members are aware that Sid is upgrading Team Foundation, so they go ahead and install the RTM client. During the time that the upgrade is running, some of these team members attempt to connect to Team Foundation. While Sid is performing steps 4-6, the clients fail to connect because Team Foundation is not installed on the servers. The team members with RTM clients are able to connect during Sid’s smoke tests. At the end of the upgrade process, TfsUpgrade.exe returns the databases to an accessible state and the clients are able to use the upgraded system.

Others on the team were already connected using the Beta 3 Refresh client. When they attempt to check in code, update work items, or otherwise work with Team Foundation, they are unable to do so because the Beta 3 Refresh client doesn’t work with the RTM server.
When Sid completes the final smoke test, he lets his team know that they should install the RTM client.

Dual Server Reinstall with Custom Process Template

Barb manages Team Foundation servers for an international corporation. When RTM ships, she has multiple servers in production at 3 locations. Although she considered traveling to each site to perform the upgrades, she has decided to execute the upgrades remotely. She has the CD in a drive on a machine that all three sites can access. She has notified her users that Team Foundation will be offline and that they’ll need to upgrade their client to RTM when it comes back online. She follows the instructions – she downloads the upgrade tool, uninstalls TFS, backs up her databases, installs and smoke tests the installation, upgrades the databases, and smoke tests the upgraded installation.
Barb has four custom process templates in use at all three locations. They contain custom work item types, work item instances, custom reports, and a customized portal template. Before upgrading, she downloads the custom process templates. She has some custom reports that are not included in a process template. Most of those are saved in source control, so she doesn’t have to do anything to save them locally. She scans the existing projects and finds a few reports that are not in source control, so she downloads those from Reporting Services and adds them to source control prior to the upgrade.

After the upgrade, Barb loads the custom reports in the Report Designer and modifies them to work on RTM. She uploads the reports to each project that uses them using rs.exe.

Next, Barb upgrades her custom process templates. Since she started with the MSF for Agile Software Development process template as her starting point, she downloads the RTM version and uses that to create her custom process template for RTM. She refers back to the Beta 3 Refresh version she saved prior to upgrading. She compares the work item types from the RTM work item types to hers and modifies her work item types so that the only changes between them and the RTM Agile work item types are her customizations. Barb adds the custom work item types, reports, and portal template to her new process template and uploads it. She creates test projects to make sure that the new process templates work properly. They don’t, so she debugs the problem, fixes the process templates and tries again. After she’s worked out the kinks in the new process template, she deletes the test projects.

Move to New Server

Takeya is an IT development manager working at a Japanese company with very specific rules about Beta software. Because of these rules, he deployed Beta 3 Refresh in a private domain instead of the corporate domain. Also, because the company doesn’t allow any trusts between the corporate and private domains, users used separate accounts in the private domain for accessing Team Foundation. Takeya is installing Team Foundation on new servers in the corporate domain.

Takeya begins by upgrading his Beta 3 Refresh databases on the existing servers, and verifies that the upgraded systems function properly. Having successfully upgraded, he is ready to migrate his TFS installation into the corporate domain.

He installs the prerequisites on the new servers in the corporate domain, then restores his backups of the upgraded databases, and installs Team Foundation.

The new server has a different name from the old server, so he uses TfsAdminUtil.exe ActivateAT to restamp the registration data to the new servers. It’s also running under a different service account – an account set up in the corporate domain.

Because he changed domains, he runs TfsAdminUtil.exe ChangeAccount. This will update each user account from the old domain to the use the account of the same name in the current domain if it exists. For example PRIVATE\nabuko is replaced by CORP\nabuko.

Takeya checks that basic functionality works and notifies his team that they should install the RTM client.

Retry

Ed manages the Team Foundation Server for a medium IT development team. He follows the instructions for upgrading to RTM. However, during the upgrade process, there is a network failure, so the upgrade fails. The upgraded databases are left in the state they had reached when the network failure occurred.

He can see in the output from TfsUpgrade.exe, and from the logs, that the failure was due to a network failure and he is aware that his network has intermittent problems. He runs TfsUpgrade.exe again, choosing to upgrade, and the upgrade picks up where it left off. The upgrade runs to completion, so he completes the upgrade instructions and sends notification to the team that they can connect with the RTM client.

Support and Recovery

Brenda leads a small IT development team. She installed Team Foundation Server Beta 3 Refresh and has just received the RTM CDs. She follows the upgrade instructions, but the upgrade fails because of an unhandled data inconsistency. She attempts to run TfsUpgrade.exe again, but it fails in the same way. She reports the problem to PSS. PSS looks at the problem and escalates it to the product team, who requests backups. She takes backups of the databases and posts them to an FTP site established by support. She restores her Beta 3 Refresh backups, installs Beta 3 Refresh Team Foundation, verifies basic functionality, and sends email to her team that the Team System was not upgraded, but that it is back up.

The Upgrade QA team copies Sarah’s backups to a server where they can be maintained (the support FTP site only retains files for 96 hours). The development team fixes the problem and a new version of TfsUpgrade.exe is prepared & verified by QA. This new version is sent to Sarah, and posted to the MS download site.

Sarah takes the new version of TfsUpgrade.exe and attempts to upgrade again. She runs into another unhandled data inconsistency. Again, she contacts support. In this case, the problem is something that’s already been encountered, and there’s a new version of TfsUpgrade.exe that fixes the issue recently posted to the MS download site. Sarah takes the new version of TfsUpgrade.exe and uses that to successfully upgrade her databases. She finishes the upgrade steps and notifies her team that they can begin connecting with the RTM client.

Design

TfsUpgrade.exe is a command line tool that allows the user to upgrade his databases from Beta 3 Refresh to RTM. It upgrades the data and schema, including the schema version for each database. It does not update the stored procedures of the operational stores. The stored procedures are updated by Setup. Other utilities are provided for specific operations.

Upgrade Download Contents

The upgrade download will contain the upgrade utility, instructions, and supporting utilities.

Upgrade Instructions.doc            The upgrade instructions.
Smoke Test Instructions.doc       The instructions for smoke testing the upgrade.
TfsUpgrade.exe                          The upgrade utility.
TfsChangeReferenceName.exe   Utility to change the reference name of work item field.
TfsBindDocument.exe                 Utility to rebind an mpp file or list in an xls to the server.
UploadAgileReports.rss              Sample script to upload Agile reports.
UploadCMMIReports.rss           Sample script to upload CMMI reports.

Components

There are two components – the command line utility and the scripts that perform the upgrades.

Command Line Utility

TfsUpgrade.exe is the command line utility that is used to upgrade the databases. It blocks access to the databases, calls the scripts to perform the necessary upgrades, then deletes the reports for the existing projects from Reporting Services.

1. Block access to the role used by the application tiers
2. Call the scripts
3. Delete the reports for each Team Project from Reporting Services.

Step 1 ensures that clients can’t access the databases until Team Foundation is installed again. (This is a safeguard to prevent access when the customer fails to uninstall Team Foundation before running the upgrade.)

TfsUpgrade.exe handles errors that occur in the scripts, logs them, raises Watson errors and outputs the specific information about the error.

Usage: TfsUpgrade [–s <data tier server>]

When TfsUpgrade is run on the data tier, the data tier server name can be omitted.

Scripts

The components that perform the actual upgrades are referred to as the scripts. Typically, these are SQL statements or batches, but they may be .exe’s as well. Each script performs a specific task. For example, a script might insert a column and some default values in a table. There are five groups of scripts, roughly associated with the databases. Some of the changes these scripts will perform are listed here. There are also two databases – TfsWarehouse and TfsActivityLogging – that are dropped and recreated by setup.

Integration Services – TfsIntegration

• Registration settings necessary as a result of moving Reporting Services to the application tier.
• Delete the reports from the folders associated with each Team Project.
• Change the name of the FIRE_EVENT permission to TRIGGER_EVENT
• Add the ATNetBIOSName extended attribute. This was introduced as a work item linking fix.
• Delete all Work Item Changed event subscriptions and update the SIDs associated with the rest of the event subscriptions.
  • Source Control - TfsVersionControl
  • Work Item Tracking – TfsWorkItemTracking and TfsWorkItemTrackingAttachments
• At RTM, Work Item Tracking will use display names instead of account names. The existing data for Assigned To, Activated By, Resolved By, and Closed By will be modified to use the friendly names instead of the account names. A standalone utility – TfsAccountToDisplayNames.exe – will be provided for users who wish to update additional fields. It will operate on a single field at a time.
  Stored queries that reference Assigned To, Activated By, Resolved By, or Closed By will be modified so that they use the display name instead of the account name. The TfsAccountToDisplayNames.exe utility will also update stored queries as well as the data in the fields.
• Between Beta 3 and Beta 3 Refresh, some fields in CMMI changed types. This resulted in a special utility to upgrade those fields for customers who had created a CMMI project in Beta 3 and then were unable to create a CMMI project in Beta 3 Refresh.
  The same changes must be applied to any databases in which the CMMI fields exist, but have not been upgraded.
  • New restrictions on references names have been introduced in RTM.
  • The maximum size of a field reference name changed from 255 to 70.
  • Reference names can’t begin with a number.
  • Hyphens can’t be used in reference names.
  • Periods and underscores are not used in determining a reference name’s uniqueness. For example Reference.Name and Reference_Name are not considered to be unique.

As a result, reference names created before RTM may be invalid. Rather than attempt to automatically rename the fields, TfsUpgrade.exe will simply fail and indicate which fields are invalid, and why. For example,

The following field reference names have more than 70 characters:
 Long.Reference.Name.1….
 Long.Reference.Name.2…
 …
Please use TfsChangeReferenceName.exe to rename the field and try again.

TfsChangeFieldReferenceName.exe will be provided in the upgrade download.

Usage: TfsChangeReferenceName.exe [–s <data tier server>] <old name> <new name>

If run on the data tier server, the server does not have to be specified.

Build – TfsBuild

Add schema version to the database.

Test Results – TfsBuild (the test results portion)

Because test results are only loaded if the correspond .trx files are present in the drop location, results will not get reloaded if the .trx files have been deleted.

Activity Logging – TfsActivityLogging

Activity Logging is dropped by TfsUpgrade.exe and recreated by setup. The historical logging information is not maintained.

Reporting Warehouse – TfsWarehouse

The reporting warehouse is dropped by TfsUpgrade.exe. Setup will recreate the database, and the first time the warehouse adapters run, they will update the tables and cubes using the RTM schema definition and then load the data.

Figure 1 - Upgraded Data

The scripts are coded to be resilient to errors and unexpected data conditions, and to handle errors and raise them in the way that the TfsUpgrade.exe command line utility expects. The command line utility will log the errors.

The scripts must also be written in a way that they can be called repeatedly. This allows a user to correct errors (like network connectivity problems) and rerun the upgrade utility, skipping over work that has already been done. For example, if there is a power failure during the third script, then the first two scripts should detect that they are not necessary and do nothing, and the third script should be able to complete the partially completed script. To be more specific, if the third script creates a column in a table, then calculates values and inserts them in the table, it should be able to deal with the fact that the column may already exist, and that some of the values have already been calculated.

The command line utility also deletes the warehouse relational and OLAP databases and the activity logging databases. During setup, these databases will be recreated and when setup is completed, the warehouse will be repopulated.

Decision Flow

Each script determines, based on the schema version of the database on which it operations, what action to take, as shown in Figure 2 – Upgrade Decision Flow. If the database doesn’t exist, then the script does nothing. If the database exists and is current, again the script does nothing. If the database exists and is an unsupported version, then the script returns an error indicating that the database is an unsupported version. The error includes the schema version if it’s known.

If the database is a supported version (Beta 3 Refresh), then the script upgrades the database. After the database is fully upgraded, the schema version is updated.

Figure 2 – Upgrade Decision Flow

For Setup, the decision is flow is simpler, as seen in Figure 3 - Setup Decision Flow. If none of the Team Foundation databases exist, then Setup continues. If they do exist, but are at the current schema version, then Setup continues. If one of the databases exists, but is not at the expected schema version, then Setup blocks and indicates that the databases must be upgraded.

Figure 3 - Setup Decision Flow

Schema Versions

Each database maintains a schema version used to determine whether the application tier can work with the database, and whether an upgrade is necessary. In most cases, the schema version is literally a version stored in the database, but TfsBuild and TfsIntegration don’t have this in Beta 3 Refresh. Those databases will some other indicator to determine whether a pre-RTM database is Beta 3 Refresh or some other version. The schema version will be added to both in RTM.

The RTM (and RC) version of each application tier will check the schema version and refuse to start if it’s not a compatible version. This prevents users from accessing Beta 3 Refresh databases that have been restored from backups after the RTM or RC application tier has been installed.
We hope that the RC and RTM versions are compatible. If not, then the RTM version will be prevented from connecting to RC databases in the same way it is prevented from connecting to Beta 3 Refresh databases.

Setup Interaction

Setup will call a stored procedure that will indicate whether the databases need to be updated. If they do need to be updated, Setup will direct users to the download center to obtain TfsUpgrade.exe and instructions for upgrading the databases.

After the databases are upgraded, the data tier setup installs the updated stored procedures in the operational store databases. The activity logging database and the warehouse are recreated.

Manual Upgrade Steps

Some of the data is not upgraded automatically. Each of those is described here, along with the mechanism for upgrading them manually.

Work Item Queries

In Beta 3 Refresh and earlier versions, work item tracking used account names instead of friendly names for fields like Assigned To. The RTM version uses friendly names. Queries that use those fields and are set to an account name will return empty result sets. This will not be updated by the upgrade utility. Instead, users will have to manually update these queries.

Work Item Changed Event Subscriptions

Work Item Changed event subscriptions store account names in Beta 3 Refresh. These will not work in RTM, so they’ll be deleted. The subscriptions can be created manually using the data extracted from the subscriptions table prior to upgrading.

Reports

The data warehouse changed significantly between Beta 3 Refresh and RTM, and so the reports that were created for Beta 3 Refresh will not work against the RTM warehouse. As a result, those reports must be deleted and replaced by new reports. For stock reports, the new version of the reports are taken from the new process template, and then uploaded to the server. To make this less cumbersome, a sample Reporting Services script will be provided that uploads the Agile reports into a project – UploadAgileReports.rss - and another for the CMMI reports – UploadCMMIReports.rss. The user will modify the script to specify the server and project and use rs.exe <script> to upload his reports in each project.

Rs.exe is delivered with SQL Server 2005 Reporting Services.

Custom reports will have to be modified to use the new warehouse schema. They can be uploaded with a modified version of the reporting services scripts, or they can be deployed to the server directly from the Reports project.

Documents

Excel lists and MS Project files bound to Team Foundation work items will continue to work after the upgrade, with the exception of those that are bound to work item queries that are broken. (See Work Item Queries, above).

If Team Foundation is moved to a new server, then the existing documents must be updated using TfsBindDocument.exe.

Project Portals

The Team Project portals are not upgraded. They can be modified manually to reflect changes made in the site templates between Beta 3 Refresh and RTM.

Process Templates

The old process templates that are delivered in the box – the two MSF process templates - will be replaced by the new version during setup. If these have been customized and saved by the same name, they should be downloaded prior to upgrading. 

The process templates will be manually updated. The approach will depend on the extent of customization. For example, a process template that has a few changes to MSF for Agile Software Development would best be handled by starting with MSF for Agile Software Development and making those changes again.

Variables

This section is intended to highlight variables that should be accounted for in the test plan.

Version

The following table describes which upgrade paths are supported. Upgrading from Beta 3 to Beta 3 Refresh involved no data upgrades. We hope that upgrading from RC to RTM will be the same, but there is a possibility that we'll have to take a change that forces us to upgrade the database schema or manipulate the data.

B3 to B3R                 Yes (reinstall)
B3R to RC                Yes 
B3R to RTM             Yes 
RC RTM                   Yes (should be a reinstall)
B2 to any                   No (blocked)
CTP to any                No (blocked)

Configuration

• Single server vs. Dual server
• Domain vs. Workgroup
• Team Foundation Workgroup vs. Team Foundation (full)
• Changing configurations

SQL Server

• Standard Edition: warehouse will not have perspectives.
• Enterprise Edition: warehouse will have perspectives.
• Standard Edition to Enterprise Edition: Should add perspectives
• Enterprise Edition to Standard Edition: Should remove perspectives
• Developer Edition (pri 3): Should be the same as Enterprise Edition

Other 

• Changing servers: see the scenarios for details
• Changing domains: see the scenarios for details
• Users accessing the system during upgrade: blocked during upgrade
• Locale: the upgrade may take place on servers that are not US-English
  

Join the Team Foundation team in another set of online chats!  Experts from the team will be available during this chat to answer questions on Team Foundation Server's suite of source control, work item tracking, Office integration, MSF, reporting, project portal, and build automation functionality.  A link to the chat is inline below.  See you there!

Visual Studio Team Foundation Server Beta 3 Public Chat Chat with members of the Team Foundation Server team. We'll be answering questions about Beta 3 and discussing our suite of source control, work item tracking, Excel and Project integration, reporting, WSS integration, and build automation tools. Add to Calendar

October 28, 2005 9:00 - 10:00 A.M. Pacific time Additional Time Zones

Enter Chat Room

Wow, this blog has four months of lost ground to cover.  There’s a lot to talk about, but I’ll start with the Beta 3 and surrounding events.  Our team’s been buried for the past quarter trying to deliver all the great improvements in Beta 3, but with those bits shipped we’re really turning our attention to your feedback.  If you haven’t done so already, download and install the bits.  If you have, post questions in our forum and log any bugs you find at the MSDN Product Feedback Center.  We’re closely monitoring these channels for Beta 3 feedback so now’s prime time to post.  For some of you this is all old news, as you have already started posting questions about Beta 3 at an unprecedented rate on our forum.  Our only message to you is that your excitement around the product excites us, so keep on doing what you’re doing. 

For partners itching for updated API documents and samples for Beta 3, these will be available later this month on http://msdn.microsoft.com/vstudio/extend/sdkdownload/, so keep your eyes peeled.  This SDK refresh will address a number of recent API and assembly name changes.

Additionally, there are two other recent events worth mentioning that aren’t directly related to Beta 3: the PDC and the MVP Summit. 

PDC 2005 Feedback

The PDC was the week before the Beta release and boy was I surprised at the level of public interest and knowledge around Team Foundation Server.  Attendees were not only keen on extending the Team Foundation platform, but adopting the tool broadly in-house.  There were a myriad of questions on topics ranging from plans around Windows Workflow to continuous integration support to ease of administration for the server.  However, I’m pretty sure the most frequent questions were “what’s your release schedule?” and “how much does it cost?” J  We’ve been seeing remarkably similar questions in our forum lately, so here are some helpful answers on both our schedule and licensing.  As we start planning around the next release of Team Foundation Server, we’ll be posting more thoughts here that address the many “down-the-horizon” questions we heard from the PDC. 

Team System MVPs

This past week, Visual Studio Team System opened up its MVP program and met with its first wave of MVPs at Microsoft’s 2005 Global MVP Summit.  It was a blast getting to know some of our new MVPs and hear about the latest progress on Beta 3 adoption and partner efforts.  If you are getting really involved with the community around Team Foundation Server or Team System in general, you may want to learn more about the MVP program.  With the doors just opening to Team System MVPs, we’re eagerly seeking new awardees. 

Well, I hope you’ve found this update useful.  We look forward to hearing lots from you in the coming days so we can address your feedback as we put the finishing touches on Team Foundation Server and get the final product into your hands.

Until next time,

Ling Bao,

Visual Studio Team Foundation Server

If you haven’t done so already, take a look at the VSTS Beta 2 Extensibility Kit.  You’ll need to register as a VSIP developer, which is free and totally worth it because the kit contains a bunch of goodies.  It has everything from Team Foundation Server overview slides to documentation on various components of the system to samples on customizing check in process templates and programming against our core services.  This is a great resource for those who want to customize or extend TFS.  It’s also a great source of general information about the architecture and components of the system.

 

If you have already downloaded the kit, check it out again because we updated some of our content this Monday, May 23.  The kit now includes these additional samples.

• A test tool extensibility sample on creating custom test types (Test Tool Extensibility\Extensibility Example.zip)
• A sample application that allows you to monitor a set of work items in real time for changes (Work Item Tracking\QueryMonitor.zip)
• A sample command line based interface for work items tracking that shows you how to leverage shared controls (Work Item Tracking\WIBrowser.zip)

Enjoy,

Ling Bao

Visual Studio Team Foundation Server

A common customization request is the need to change the list values populated in the Assigned To field on a work item.  By default, this field shows all users known to TFS (members of the TFS Everyone group).

What if you want to assign work items to groups or special values that don’t correspond to a user or group?  Well, the Assigned To field is just like any other field and you can fully customize its list values.

Here’s what you do.  Export the work item type from your project (see this post for instructions on importing and exporting types).  If you want to assign work items to groups, replace the definition of Assigned To with the following XML.  This expands all users and groups defined in the project contributors group in the Assigned To pick list.

<FIELD name="Assigned To" refname="System.AssignedTo" type="String">
<ALLOWEDVALUES expanditems="true">
      <LISTITEM value = "[Project]\Contributors" />
</ALLOWEDVALUES>
</FIELD>

Note how the list shows the Contributor group, as well as the child groups such as Developers and users in those groups.

To assign work items to special string values, just add them as list items.  The XML below allows work items to be assigned to the triage team.

<FIELD name="Assigned To" refname="System.AssignedTo" type="String">
<ALLOWEDVALUES expanditems="true">
      <LISTITEM value = "[Project]\Contributors" />
      <LISTITEM value = "Triage" />
</ALLOWEDVALUES>
</FIELD>

Until next time,

Ling Bao

Visual Studio Team Foundation Server

Migration of VSS database is a CPU and network intensive process. Hence, the performance of VSS Converter on a VPC image is very slow. We recommend users not to evaluate VSS Converter performance on VPC.

Thanks,

Akash

A frequently asked question about work item type definitions is: “How do I populate the list of allowed/suggested/prohibited values from an external source?”  This can be achieved by using global lists – named lists of values that can be shared by any work item type on the Team Foundation Server.

 

This post shows an example of how to use global lists.  In this example, you run an online retail shop with different pages dedicated to different product categories.  You use a work item type called “MyType” to track customer requests for changes to the site, additions to the selection, or other issues.  You want to classify your customer request based on the category of products that they’re associated with so that you can report on how many requests you’re getting per category.  To do this, your work items have a product category field with a list of allowed values.  The XML for your work item type looks like this.

<FIELD name="Product Categories" refname="MyCompany.ProductCategeroies" type="String" reportable="dimension">

<HELPTEXT>Product Category</HELPTEXT>

<ALLOWEDVALUES>

<LISTITEM value="Books" />

<LISTITEM value="Electronics" />

<LISTITEM value="Kitchenware" />

<LISTITEM value="Toys" />

</ALLOWEDVALUES>

</FIELD>

 

However, the product categories on your web site constantly change forcing you to manually update your work item type all the time.  Instead, you decide to harness the power of global lists to maintain the product categories list.  You author a tool that checks your product inventory database for updates and mirrors those updates in a global list on the Team Foundation Server.  Then, you change your work item type to use the values of this global list to dynamically populate the entries in the Product Categories field.  Now, when a new DVD category is added to your product database, your tool automatically updates the categories on the Team Foundation Server and your work items pick up these changes from the global list.  In fact, you can have as many work item types as you want leverage this global list.

Now, before you get started, make sure to download the Beta 2 Extensibility Kit and unzip “Authoring Work Item Types.zip” into the ~\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies folder on your client machine to get the tools you need.  Also, check out the documentation in the “Work Item Tracking” and “Work Item Types” folders for detailed info about the work item tracking object model, the type definitional language, and global lists.

 

Step 1: Create a global list

The first step is to create a global list that will hold all the product categories.  To do this, create an XML file like below and save it as “gl.xml”.

<?xml version="1.0" encoding="utf-8" ?>

<GLOBALLISTS>

<GLOBALLIST name="Products">

<LISTITEM value="Books" />

<LISTITEM value="Electronics" />

<LISTITEM value="Kitchenware" />

<LISTITEM value="Toys" />

</GLOBALLIST>

</GLOBALLISTS>

 

The GLOBALLIST element declares a new global list called “Products.”  The LISTITEM elements add string values to this list.  Note that you can add as many global lists as you want and that each list is a flat structure (no hierarchies).

 

Next, import the global list onto the Team Foundation Server.  You must be run this tool as a Namespace Administrator.

1. Open a command prompt and CD into your ~\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies folder
2. Run glimport to import the list

glimport /f gl.xml /t MyTFSName

1. Run glexport to make sure the list got uploaded correctly.  You should see the XML you uploaded on screen.

witexport /f MyType.xml /t MyTFSName /p MyTeamProject /n MyType

 

Now you have a global list on the server ready to be referenced by work item types and updated by your tool.

 

Step 2: Customize your work item type to reference the list

The next step is to customize your work item type to use this global list instead of hard coded list values.  Export your existing work item type and edit the XML to refer to the list for field.  The steps like this.

1. Open a command prompt and CD into your ~\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies folder
2. Run witexport to export your work item type

witexport /f MyTypeBeforeChanges.xml /t MyTFSName /p MyTeamProject /n MyType

1. Edit the XML for your “Product Categories” field to populate it with items from the global list “Products.”

<FIELD name="Product Categories" refname="MyCompany.ProductCategeroies" type="String" reportable="dimension">

<HELPTEXT>Product Category</HELPTEXT>

<ALLOWEDVALUES>

<GLOBALLIST name="Products" />

</ALLOWEDVALUES>

</FIELD>

1. Save the type xml as “MyTypeAfterChanges.xml” and reimport it.

witimport /f MyTypeAfterChanges.xml /t MyTFSName /p MyTeamProject

 

When you create new work items of this type, the Product Categories field will show the same list values.  However, these will be pulled from the global list.

 

Note: In the Beta 2 build, there is a bug where the global list name will show up in the drop down list – i.e. “*Products” will appear in the Product Categories field in the example.  This will be fixed in our next community drop.

 

Step 3: Update the global list from an external source

Now that everything is setup up, you can write a tool that syncs changes from your inventory database to the “Products” global list.  You write this tool using the Work Item Tracking Object Model (see “Work Item Object Model Sample.zip” in the Beta 2 Extensibility Kit) and run it under a TFS Service Account so that it can make changes to the server.

 

You set the tool to run regularly as an NT Service and include the following code snippet to update the global list.

//read your inventory DB for all product categories

//create a string “categories” that contains XML in the following format

//with a LISTITEM element for each category read from the DB

//

//<?xml version="1.0" encoding="utf-8" ?>

//<GLOBALLISTS>

//<GLOBALLIST name="Products">

//<LISTITEM value="Books" />

//<LISTITEM value="Electronics" />

//<LISTITEM value="Kitchenware" />

//<LISTITEM value="Toys" />

//</GLOBALLIST>

//</GLOBALLISTS>

 

//get the work item store from the TeamFoundationServer

TeamFoundationServer tfs = new TeamFoundationServer(server); //server – string for server name

WorkItemStore store = (WorkItemStore)tfs.GetService(typeof(WorkItemStore));

 

//import the global list defined in string “categories” to the work item store

store.ImportGlobalLists(lists);

 

Once this tool is running, changes in the inventory database are mirrored in the Product Category field on your work item.

 

Sample XML

Here are sample XMLs you can use for this exercise.

• gl.xml – The global list definition
• MyTypeBeforeChanges.xml – The type definition initially
• MyTypeAfterChanges.xml – The type definition that uses the global list

 

Keep in Mind

Here are some things to keep in mind while working with global lists.

• Global lists can only be modified by members of Namespace Administrators and Service Accounts groups.
• If a work item refers to a list item that is later removed in a global list update, the work item will still show the removed value next time it’s opened.  However, users will have to select a new value before saving updates to this item.  Note this is the same behavior with any type of list value or field constraint where previously legal values become invalid due to a work item type change.
• Global list names are unique to a Team Foundation server and can be up to 254 Unicode characters long.
• When a global list is updated, clients will have to refresh their metadata to get the latest values.  This is done by clicking the “Refresh” button on the Team Explorer in the IDE.

 

Until Next Time,

Ling Bao and Aliaksei Baturytski

Visual Studio Team Foundation Server

Join the Team Foundation team in another set of online chats!  Experts from the team will be available during these two chats to answer questions on Team Foundation’s suite of source control, work item tracking, Office integration, MSF, reporting, project portal, and build automation functionality.  Links to the chats are inline below.  See you there!

 

Visual Studio Team Foundation Server Beta 2 Public Chat Chat with members of the Team Foundation Server team. We'll be answering questions about Beta 2 and discussing our suite of source control, work item tracking, Excel and Project integration, reporting, WSS integration, and build automation tools. Add to Calendar	May 11, 2005 4:00 - 5:00 P.M. Pacific time Additional Time Zones	Enter Chat Room
Visual Studio Team Foundation Server Beta 2 Public Chat Chat with members of the Team Foundation Server team. We'll be answering questions about Beta 2 and discussing our suite of source control, work item tracking, Excel and Project integration, reporting, WSS integration, and build automation tools. Add to Calendar	May 12, 2005 8:00 - 9:00 A.M. Pacific time Additional Time Zones	Enter Chat Room

 

Hi

Have you been trying to get Team Build working and keep getting a 'Build Machine is not reachable' error ? If yes then make sure you have installed Team Build SKU from \vstf\bb folder.

Another point to note : In case you are trying to install Team Build on win 2K3 SP1, the setup will fail.

Its a known issue in beta 2 and we are working on fixing it. However here is a workaround for you to get going

      - Start the Windows Firewall/Internet Connection Sharing service which is disabled by default in win2k3 sp1

      - Trying installing Team Build again

You should be good to go now :) - have fun and let us know all that you think about the product ! 

-Khushboo

My name is Jiange Sun and I'm a tester in Team Foundation Version Control (TFVC) team. I am responsible of testing add, checkin, delete/undelete, get and permission features. With this post, I'll provide an overview of checkin policies in TFVC.

The checkin policy provides a mechanism for an organization to define checkin rules for a team project and enforce them by the source control client during the checkin process.  Checkin policies are implemented with the extensible checkin policy framework; third parties can implement their own checkin specific to their own application. Jim Presto has a post on how to implement a checkin policy.

We include 4 checkin policies in Visual Studio Team Foundation:

1. Clean Build (Requires that code is error clean before check-in) Note: This one will go away after beta2 and be integrated in to “Code Analysis” policy
2. Code Analysis (Requires that code analysis is run before check-in)
3. Testing Policy (Requires run check-in tests before check-in)
4. Work Items (Require that one or more work items be associated with every checkin.)

Next, I will be using the “Work Items” policy as an example to walk through a use case scenario.

Checkin policy Configuration

Let’s pretend that you are a Team Lead and you want your developers to associate at least one work item for their checkins. After multiple unsuccessful attempts on this message through team meetings and emails, you’ve had enough and decide to take advantage of the power of the checkin policy feature in Team Foundation:

   1.   You start VS, open the Team Explorer window.

   2.   On Team Explorer window, right click on your team project, select “Team Project Settings” | “Source Control …” from the context menu.  

   3.   On Source Control Settings dialog, select the Checkin Policy  tab. 

   4.    Selects the “New…” button to define a new checkin policy:

        

   5.   Select Work Items and click on Ok:

        
Note: Checkin policies include a mechanism for providing instructions to the user in the event that the policy plugin is not installed on their system. Since the work item policy is included in our release and will be installed when you install Visual Studio Team System, the above dialog will go away after Beta2.

   6.   Click on Ok button. You see the newly added Work Items policy is showing in the list. You can also “Edit”, “Remove”, “Enable” and “Disable” a policy with a single button click on this dialog:

        

   7.   Click on Ok and that’s it. You’ve just finished configuring the Work Items checkin policy for your team project. Next let’s see how this policy is enforced automatically when your developers try to do a checkin without associating a work item. 

Checkin policy Evaluation

Checkin policy evaluation process occurs during checkin process with the checkin dialog (both command line and VS). Let’s pretend that John Doe is a developer in your team and he is curious to see how the checkin policy you configured works:

a.   John starts VS and opens the solution. He makes some changes in his code and is ready to check in. He right clicks on his solution in solution explorer and selects “Check in…” context menu.

b.   John clicks on the “Check in” button and a “Policy Failure” dialog is presented since the Work Items checkin policy has not been satisfied:

       
Note: John really should’ve clicked on “Policy Warning” channel on the checkin dialog to see if all checkin policies are currently satisfied before he clicks on check in button. If John checks “Override policy failure and continue checkin”, the reason text box will be enabled and he can provide additional information about why he is checking in despite the policy failure.  The Ok button is then enabled and the checkin will succeed if he clicks on it.

c.   John clicks on the Cancel button on the “Policy Failure” dialog and he noticed that the checkin dialog has automatically switched to “Policy Warnings” channel for him. He sees the reason of the policy failure: “You must associate this checkin with one or more work items.”

       

d.   John clicks on the “Work Items” channel on the “Check in” dialog and checks a work item. He then switches back to the Policy Warnings channel and now he sees a description that reads “All checkin policies are currently satisfied.”

e.   John clicks on “Check in” button and this time it succeeds.

In this post, we looked at how to configure checkin policies, when checkin polices are evaluated and how to override a checkin policy failure. As you’ve seen, it’s pretty straightforward to use the new checkin policy feature in Visual Studio. Please let use know if you have any feedback on the checkin policy.

If you’ve already played with work item tracking in Visual Studio Team System, then you should be familiar with the notion of work item types.  For example, there are four types (Scenario, Bug, Task, and Quality of Service Requirement) in the MSF Agile methodology where each type allows users to track a different kind of work item.

 

Of course, you organization may have different workflow or different types of work items to track.  To accommodate this variation, Team Foundation allows you to customize work item types to your content.  You can add fields, rename fields, restrict the list of allowed value for fields, change the states and supported state transitions, make fields required or read-only, make one field dependent on another, automatically populate field values, re-arrange the appearance of information on the form, and much, much more!  You can also start fresh and build a work item type from the ground up.

 

How do you get started?  There are two ways.

1. Edit work item types in process templates and create new team projects using the updated template
2. Edit work item types in existing team projects directly

In each case, you edit XML definition files to specify the behavior of the work item type.  

 

 

 

 

Process Templates for New Team Projects

When you edit work item types in a process template, every new team project you create with that template will have the updated work item type.  Here’s how to do this.

1. Ensure that you have “Edit domain-level information” permissions on the Team Foundation Server.  This is administered via Team Explorer by right clicking the server icon and selecting Team Foundation Server Settings->Permissions.
2. Follow the instructions in Amy’s “Customizing Process Templates” post to export a process template.
3. Open the exported folder and go to the sub-folder where work item type definitions are stored (“MSF Agile\Currituck\TypeDefinitions” for MSF Agile in the December CTP).
4. Open the XML file for the work item type that you want to edit in your favorite XML editor.
5. Start editing away and make your customizations.  For reference information on the definition language, check out “Authoring Work Item Types Using the December CTP.doc” in the “Work Item Type” folder of the December CTP Extensibility Kit.
6. Save your changes and import the process template back onto the server by following Amy’s “Customizing Process Templates” instructions.
7. Create a new project with the template and start using your customized work item type.

 

Work Item Types on Existing Team Projects

Once a team project is created, the only way to edit work item types in that project is to export the type directly from the project, edit it, and re-import it.  These changes are scoped to the team project and won’t alter process templates or work item types in other team projects.  To edit existing work item types, you must use an administrative utility to export and import the XML from the project.  Step-by-step instructions are provided in “Authoring Work Item Types Lab.doc” in the “Work Item Type” folder of the December CTP Extensibility Kit.  Note that for December CTP and Beta 2, you will need to be a member of the Project or Namespace Administrator group in order to import work item types.  For the final release, we are planning to create a permission for administering work item types that can be granted to other non-admin groups.

 

Have fun customizing your work item types!  Also, keep your eyes peeled for an updated Extensibility Kit containing the latest materials on work item type customization when Beta 2 comes out.

 

Ling Bao

Program Manager

Visual Studio Team Foundation

As promised, I (Amy Hagstrom, VSTF Program Manager) am back this week to discuss customizing process templates.

What are process templates?

As mentioned last week, process templates are a type of blueprint for the Team Project Creation Wizard.  They provide a set of team project customizations that support the processes a team should follow.  VSTS will ship two process templates: MSF for Agile Software Development and MSF for CMMI Improvement.  Out of the box, process templates will include these elements:

·        Work Items

o       Type definitions (such as Defect, Task, and Issue) – stay tuned for a post in the next couple weeks that covers modifying Work Item type definitions in more detail

o       Queries

o       Instances – these make up a Project Roadmap (a predetermined set of tasks that must be done for every project, such as gathering requirements or writing a vision document)   

o       Mapping of Work Item fields to MS Project columns

·        SharePoint Site layout, theme, and content

o       Document Templates

o       Process Guidance

·        Version Control Settings

o       Check-in notes

o       Permissions

o       Multiple check-out

·        Reports (SQL Reporting Services)

·        Groups and Permissions

·        Iterations

Beyond this, 3^rd parties can write plug-ins for the Project Creation Wizard that could consume their own custom process templates elements. 

How are process templates customized?

1. It is best to start with an existing process template.  To do this first launch the Process Template Manager in Visual Studio by going to the Team menu > Team Foundation Server Settings > Process Template Manager.

         

2. Select the process template you want to base your custom process template on, and choose “Export.”  This will download the process template to a local directory you specify.  The process template is a cohesive set of folders and files.

        

3. To change the name and description of the process template, edit ProcessTemplate.xml:

...

<methodology>

     <metadata>

      <name>Name of my Custom Process Template</name>

      <description>Use this process template for projects that require light processes… </description>

...

4. Determine which modifications you want to make, and edit the appropriate xml files using your favorite xml editor.  In the below SCCTasks.xml I’m adding a new required check-in note called “Comments”, and making exclusive check-out required:

<?xml version="1.0" encoding="utf-8" ?>

<tasks>

    <task id="SccTask" name="Create Source Control area" plugin="Microsoft.Pcw.Scc" completionMessage="Source control area created.">

      <dependencies/>

      <taskXml>

<permission access="Read, PendChange, Checkin, Label, Lock, ReviseOther, UnlockOther, UndoOther, LabelOther, AdminProjectRights, CheckinOther" grant="allow" identity="$$PROJECTNAME$$\Project Administrators"/>

      <permission access="Read, PendChange, Checkin, Label, Lock" grant="allow" identity="$$PROJECTNAME$$\Contributor"/>

      <permission access="Read" grant="allow" identity="$$PROJECTNAME$$\Reader"/>

      <checkin_note label="Code Reviewer" required="true" order="499"/>

      <checkin_note label="Comments" required="true"/>

      <exclusive_checkout required="true"/>

      </taskXml>

      </task>

</tasks>

5. If you are including new files (like a new document or report), add those files to the appropriate place in the folder structure. 
6. To upload your customized process template, launch the Process Template Manager again and do so via the “Import” button.