Thursday, March 22, 2012

TFS2010 Gated Check-In and Multiple Build Agents

Most dev teams today improve their process by establishing the practice of Continuous Integration (CI; http://martinfowler.com/articles/continuousIntegration.html) and automating builds and tests. There are many products available to help perform automated builds. The range of products provide a fit for any size wallet. Microsoft Team Foundation Server 2010 (TFS2010) comes with these capabilities built-in. TFS2010 goes beyond check-in-build-test-report functionality and offers gated check-ins: build-test-report-check-in. This mode prevents broken builds appearing in the Source Control Management (SCM) system.
Gated check-ins are implemented on top of shelvesets. Once turned on for build definition, gated check-in will cause all check-ins to submit changes as shelvesets. The build server processes the shelveset and, if the build and tests are successful, the build server checks the changes into SCM and makes them available to other team members. Gated check-ins are subject to the same limitations as shelvesets though (http://blogs.msdn.com/b/buckh/archive/2006/01/10/511188.aspx ).
As the project grows in terms of team and/or source code size, scalability of the TFS builds may become the next challenge. One of the easiest steps to address this problem is to add more TFS build agents. Since each build agent can work on one build at a time, having multiple build agents should allow TFS to process multiple check-ins concurrently, right?

For regular CI, adding build agents improves the overall development process. Each gated check-in requires "exclusive" write access to the source code during build and test, so all build agents, except one, will be idle.

Most developers face resource sharing issues everyday (database locks and scaling-out challenges are the most popular). Performance improvement efforts usually go either into scaling up or into eventually consistent world. Gated check-ins and TFS scalability are the same challenges but in a new context. The solutions are the same as well: scale up the TFS server or move into eventually consistent old-and-proven CI (check-in-build-test-report).

It is easy to test the behavior mentioned above by using VMs with Visual Studio/TFS (http://www.microsoft.com/download/en/details.aspx?id=240). Add another agent, download a large open source project (I used Orchard CMS), set up build with Gated Check-In, and try submitting two changes one after another. Only one agent will be working.

3 comments:

  1. just found a trick to bypass this limitation:
    clone build definition, then you can have each build definition running in parallel.

    ReplyDelete
  2. @typ Each gated check-in should add NO_CI to check-in comment, which should prevent other automatic CI builds from triggering. If it is not happening after cloning gated build, then it sounds like a bug in TFS. I have to try it too...

    Two simultaneous gated builds can produce "phantom" builds. Consider two [almost] simultaneous gated builds, "dev1" & "dev2", on top of "base" changeset. Once first completes, it adds changeset to the repository and repository becomes base+dev1. Once second build completes and produces base+dev2 artifact, it merges results into repo and repo becomes base+dev1+dev2. At the end, we have build artifact from non existing changeset and unverified changeset in the repo (there was no build for base+dev1+dev2).

    ReplyDelete
  3. Andrey is correct. My recommendation now would be to use TFS 2012 and turn on "batching" for gated checkin. This article in Visual Studio Magazine describes it: http://visualstudiomagazine.com/articles/2012/06/13/batched-gated-builds-in-tfs.aspx.

    ReplyDelete