I work on a project where we have Live, RC, and QA. Individual issues go to QA when they're ready where the QA team tests the ticket's functionality. Once the ticket passes, it's ready to move to RC. Before a release we take all of the issues that are ready and put them all on RC together where we make sure that tickets don't cause problems with each other. Once that's done all of RC moves to live together.
What is a good way to handle this with Mercurial? Our current process has some problems.
Our current process:
Live runs default, RC runs a branch created off of default (live) and QA runs it's own branch (trunk) that branched from default in the past.
Issues are branched from default, worked, and then merged to trunk. Once a ticket is ready to go to RC it gets merged into the upcoming release branch. When that gets tested the release branch gets merged into default and default gets merged back into trunk. The problem we're running into is that after a while something happens and everything gets conflicts merging to trunk. If we resolve conflicts to trunk in that merge it tends to break trunk much faster, if we have conflicts we merge default into our branch and resolve them in that commit. This usually works but after a few weeks or months trunk seems to break and we can no longer resolve the conflicts.
Update:
Our process works such that ticket A might move into the QA environment and stay there for a while. Ticket B might be developed, moved to QA, QAed and on RC and released before ticket A ever leaves the QA environment. That is not something that can be changed right now. I'm looking for a solution that fits our needs. I have the ability to influence our implementation of the process at the repository level, but not the ability to modify the over all process in a significant way right now.
I think your best bet is reviewing Mercurial's suggested workflows. I can tell you from my own experience with a structured release cycle that had code (features, enhancements, and bug fixes) going to a QA environment for verification and prepping releases, then branching from those release tags for emergency patches, Mercurial fits nicely.
For my steps below, "trunk" is just what others might call mainline. There isn't any term "trunk" in Mercurial. You just have the repository you are working from, which is either just your local repository or a clone of another repository (still local).
trunk is where everyone generally works.
When a particular change is completed, it is pushed to a central server (still trunk and a central server is optional, just makes the workflow easier with continuous integration).
From the central server repository, a build is done and pushed to the QA server.
QA tests the functionality and should also generally test limited interaction outside the known change.
If change passes QA testing, a release manager may choose to tag that changeset as the version number to be released (hg tag "name of tag").
Once code is tagged and ready for a release, another build can be done to push to a user acceptance test environment (UAT), which mimics production configuration.
If UAT passes, a final build can be done to push into production. Note that you can save some build steps by using separate "build" repositories that hold only pre-built copies of your software. That way, only a single build needs to be done if QA and UAT testing passes before pushing to production.
If a defect is found in the production release, a branch can be created from the tagged revision that was released (say 1.0.0) using hg update "tag name" then hg branch "name of branch" (which should probably match the tag name).
While "on" the branch revision, fix the bug, commit, merge it to "trunk".
Do a build from the latest revision "on" the branch and push that to QA.
If testing is successful, that build can be moved to UAT and production as necessary and you can feel confident that you are only releasing that single bug fix.
All the while, new functionality, features, and non-critical bug fixes can be done in "trunk", continuing with the "next" version of the code to be released on a schedule.
Related
I've been trying to find actual documentation for Sourcetree without much luck, so I figured I'd ask here. I'm a relative newb when it comes to version control, with my current project being my first effort. I'm using Sourcetree on Windows 7 as a frontend for Mercurial, I've got my development code on my local machine in C:\inetpub, and whenever I do a Commit I then switch Sourcetree over to the cloned repository on my backed up network drive and do a Pull of the changes I just Committed so I've got the development history backed up.
What I'm trying to wrap my head around is using Sourcetree to set up a Debug branch so I can fix bugs on the version of the code running on the production server while simultaneously doing development. Obviously a common need, but I can't grok it. I am expecting there to be two code locations, so I can pause in mid-edit on the Development branch, make changes to Debug, and them come back to my changes in Development and finish them up before merging in the changes to Debug. If that's not how it works that's presumably part of my confusion. Any suggestions on where I can find clarity? Pointers to existing tutorials and the like would be fine, I just haven't been having luck searching Google, and I haven't been able to locate any actual Sourcetree documentation.
NOTE: Based on responses I've seen to other questions I've read about Sourcetree and Mercurial, I'll state upfront I have no interest in discussing outside repository hosting unless somebody can explain why it will help with this problem.
Two things here:
You do not need to change repository to pull, you can also push from your local repository;
You do not need 2 code locations for switching from one branch to the other. It might help, for larger projects, but I suggest you get comfortable with Mercurial before doing so.
So, for number 1, there is a default source or remote repository for every local repo. It is either defined by the user or it is the source repo from where it was cloned. Whether you push or pull, it will default to that same source. You can push/pull to multiple sources as well, but this is not your case at the moment. In the normal workflow, just issue a hg push every time you commit, and your changes will be propagated to the other repo.
For number 2, a Mercurial repo, as you already know, can have multiple branches. When you commit a changeset, it is automatically done on the current branch. In SourceTree, click on the Branch button, and enter a new branch name. The next commit you'll do will be the head (and the start) of your new branch. After that, you can update back and forth between the 2 branches, and your code will change accordingly. That means that you can update at any time to the head of any branch, make some changes, commit, and then jump to another branch, and so on. So no, you do not need multiple repositories.
Normally, the proper practice is to have a default branch (default name is rarely changed!) where you have your current development source. For every issue or feature you are fixing/implementing, create a new branch from the default branch to put your new code. Once that branch has been reviewed and tested, merge it back in the default and close the former.
For your current development, if you need an additional stable and safe trunk, you can create a Production branch, which would be the stable code that will run on your server. Once you are satisfied with the default branch tests, you can then merge it in Production to include your changes up to that point.
As a convention, make sure your server is always running the code from the Production branch, for the more stable code. At least, that is what I understood from your initial question.
Coming originally from SVN, I am still new to Mercurial.
I am interested in creating an experimental workflow to see if I can rewrite a troubled feature from scratch. If my attempt fails though, I wish to delete the experimental workflow - abandoning the work — with nobody else ever seeing it.
The problem is though I still need to push changes of this experimental workflow across laptops and PCs and keep working for a couple of weeks. But still keep the option open to delete that branch and fall back to the main branch, without having any trace of the experimental branch.
Is something like this possible in Mercurial and how could I achieve this?
FYI, I am using mercurialeclipse plugin on Aptana Studio 3.0. (so I
use a UI but commands should be fine too)
After changeset is pushed to the central server (assuming you have one) - there is no way to remove it from there.
So the possible (but terribly inconvenient) solution for you now could be to create a personal separated repository and synchronize your devices using it. And if you like the result - you push to the shared central repo then. Otherwise you just delete the temporary repository.
With a Distributed Version Control System like Mercurial you can sync between any clone of a repository, not just a "central" one that all users have agreed to use.
Therefore, you can:
Clone the repository to private a share that the systems "experimenting" can access.
Clone to a USB key and move that between systems.
Use hg serve to start a web server for a local repository on a system and clone and pull that history to other systems.
Use hg bundle/unbundle to package up new history and email it to another system.
To abandon work, just delete all these extra clones and clone from the common "central" repository again.
My dev team is just starting out with Mercurial and we're confused on the following point:
We are a php webdev team.
We have 3 developers. Most of what we are doing now is bugfixes on a very new product. Also doing some new feature development.
We have 2 QA people. Every bugfix and feature must be tested before it is allowed to go live.
So far, each developer has his own repository. We have a central sever called WebDev with it's own repo. A developer pulls from WebDev, then makes some changes (ie, fixes a bug), and pushes to Webdev. Then a QA tester will test the code on the central server (so testing on the code in WebDev) and if it works, he will push that code to our production server.
This does not work well, because... what happens when Developer-1 (dev-1) fixes a bug, and pushes to WebDev. At the same time, dev-2 fixes a different bug and pushes to WebDev. the QA person tests the code there, and approves the second bugfix but not the first. How would he push the second changeset to production without the first one as well? We seem to be losing all the advantages of a version control system.
I've read up a lot on branching, but I cannot figure out how to make this work for us... do we create a new branch for every bugfix and new feature, and the only after it is tested, QA will merge into the default branch on WebDev? Is that the best way, or is there something I am missing?
Thanks!!
----UPDATE----
thanks to everyone who answered so far. here is where i am holding now... i can think of two solutions.
1) dev-1 fixes a bug for bug-101. he pulls from webdev, merges and commits locally. he sets it in-testing. QA pulls directly from his repository, and test locally. if it passes, QA will pull from webdev ->merge -> push to webdev (and if its a big change, can review again there to make sure it is fine). so we are only testing one thing at a time, WebDev only contains changes that have been tested locally by the testers and is always stable.
2) create branches for everything. dev-1 creates branch "bugfix-101" then pushes to webdev without merging it. QA can test the branched code, and if it's approved, merge it with the default branch. I have four questions on this method - (a) is it possible to push an open branch to a remote repository? (b) if QA merges and closes the branch on webdev, the next time i pull, will my local repo also close and merge the branch? and (c) how do you test from the branched code? when i run the web app in the browser, how do i test from the branch?? (d) are there performance issues with creating so many named branches (assuming that most of them will get closed quickly)?
Thanks again.
Folow up to Bassam:
Your team is obviously missing the (really easy) branching and merging in Mercurial and working with a monolitic (?) default branch.
Just change your thinking and workflow slightly and you will see a big difference:
Each QA-member has a permanent clone of the repo and only pulls a developer's repo on request (it's faster, pulled changes are more visible); maybe branch QA also have sense
Use a separate branch for each and every big change (feature or bugfix)
When a dev has a changeset X in branch "Bugfix Y" in his repo finished and ready to test, he asks QA to "pull and test changeset X"
QA does that, maybe merges "Bugfix Y" to the "QA" branch in his repo (as a "test passed" sign?) and merges the "QA" branch to mainline ("Stable" or "default" branch), and finally pushes the results to needed destinations (WebDev and Prod?)
On every next request step 4 must n\be repeated
This way you never mix in one approve-cycle more than one development action
This is a good place to use tags. Have either the dev or qa person tag the release containing only the changes they want, and then the qa person can clone the repository at the tag level (or update the repository to the tagged changeset if that fits better for you... your preference). Then do the test using the tagged version of the code.
As a side note, it would be worth looking into these two answers on the Kiln stackexchange site to see Fog Creek's repository strategies (see how they eat their own dog food):
http://kiln.stackexchange.com/questions/354/release-repo-suggestions/355#355
http://kiln.stackexchange.com/questions/500/should-i-use-more-than-one-repository/504#504
Update
There is a good description in this post about why it is better to fix bugs in a stable branch and push them back to dev, while using a dev branch for features (which get pushed back to the stable branch as well... two way pushing/pulling). This is how we do it as well. It is not directly answering your question, but is related enough that I thought I'd include it.
Creating branches with Mercurial is a breeze, use that :) I would create separate branches for different feature and different bugs and have the QA person merge them to the main branch whenever the ensure that a bug is fixed.
Another alternative is to use named branches which is essentially the same thing but instead of seperate branches, your named branches is going side by side with the default branch until they are ready to be merged in.
Note: We have been alternating between both strategies at my work place.
Right now, we have a small team of developers using TFS for our version control. I'm evaluating the possibility of us moving to a DVCS, and am wondering if we'd need to give up some of the stuff we like about our current system if we moved to DVCS, or if we can find a way to support it.
Right now we a Stable branch, and 1 branch for each developer (you can think of each dev's branch as a feature branch that is reused from feature to feature). The stuff we like is:
1) Each time a dev checks in changes to his branch, we do a build and test of everything on the servers, followed by automatic deployment of all projects to that developers test environment.
2) Merging from any dev's branch to Stable is done by me, so that I can have 1 last check on what is happening to our stable branch before committing the changes.
3) If I want to help a dev with something, I can just grab latest from their branch and look at it on my machine.
I'm trying to understand how this could work with a DVCS (specifically we are testing with Mercurial).
I'm hoping to be able to pull off something like this:
1) We setup a central repository, and we create Main and Release branches in addition to 1 branch for each developer.
2) All devs clone the repository to their local machine.
3) All work is done in their personal branch against the local repository.
4) When they are done, the would pull from the central repository, and perform a local forward integration merge from Main to to their branch, to integrate any changes that have happened in the past to Main.
5) They would then push their changes to the central repository.
6) Some CI service would pickup this change, causing a build/test/deploy-to-dev of all our projects in that branch.
7) If everything was ok the dev would shoot me an email saying their branch was ready for merging to Main.
8) I could then merge in their changes, either by somehow connecting directly to the remove repository, or by doing a pull->merge->push.
So to deal with our requests:
1) I'm assuming there is some CI tools that can watch a branch in Mercurial, and kick off a build/test/deploy process (like CC.Net).
2) I can still manage the final merge process from DevX to Main either by connecting to the remote repo, or by pulling, merging and pushing through my local repo.
3) I believe I could either pull changes directly from another dev's repo, or I could just pull from the central repo, and then update my working directory to work on their code.
So do I have this mostly right?
Yep, you have all that right. Regarding you final assumptions:
1) There's a Mercurial Source Control Block for CruiseControl.NET. Another CI server I've heard of in use with Mercurial is Jenkins.
2) Correct. For integration with Main, I would prefer pulling (from either) and merging on my own machine before pushing, rather than merging on the server.
3) Exactly so.
It sounds like your developers are fairly disciplined, but just in case you need better control certain aspects of your operations:
You can use hooks to issue warnings when someone tries to merge their branch to Main. In-process hooks have to be written in Python, but they have access to the Mercurial API that way. You could also place hooks on the server that reject pushes containing a merge to Main not done by certain users.
One way some organizations control integration is a pull-only scenario. Only a few developers can push to the official repository and other developers send them pull requests. The Mercurial book's Chapter 6 covers this a bit, too.
A branch per developer is good. A branch per feature is also useful, allowing each developer to work on multiple things in parallel, then merging each to their branch when done. They just have to remember to close that feature branch before doing so, so the branch name doesn't keep popping up. This can be done with with clones as well, but I find myself preferring named branches since I have to keep work/backup/laptop development clones all synced so I can work on whateve, whenever. I still do expendable work as a clone first.
We got all psyched about from from svn to hg and as the development workflow is more or less flushed out, here remains the most difficult part - staging and integration system.
Hopefully this question goes a bit further then your common 'how do I move from xxx to Mercurial'. Please forgive long and probably poorly written question :)
We are web shop that does a lot of projects(mainly PHP and Zend), so we have one huge svn repo, with like 100+ folders, each representing a project with it's own tags,branches and trunk of course. On our integration and testing server(where QA and clients look at work results and test stuff) everything is pretty much automated - Apache is set to pick up new projects automatically creating vhost for each project/trunk; mysql migration scripts right there in trunk too and developers can apply them through simple web-interface. Long story short our workflow is this now:
Checkout code, do work, commit
Run update on the server via web interface(this basically does svn up on server on a particular project and also run db-migration script if needed)
QA changes on the server
This approach is certainly suboptimal for large projects when we have 2+ developers working on the same code. Branching in svn was only causing more headaches, well, hence moving to Mercurial. And here is where the question lies - how does one organize efficient staging/integration/testing server for this type of work(where you have many projects, say single developer could be working on 3 different projects in 1 day).
We decided to have 'default' branch tracking production essentially and then make all changes in individual branches. In this case though how can we automate staging updates for each branch? If earlier for one project we almost always were working on trunk, so we needed one DB, one vhost, etc. now we potentially talking about N-databases per project, N-vhost configs and etc. Then what about CI stuff(such as running phpDocumentor and/or unit tests)? Should it only be done on the 'default'? On branches?
I wonder how other teams solve this issue, perhaps some best practices that we're not using or overlooking?
Additional notes:
Probably worth mentioning that we've picked Kiln as a repo hosting service(mostly since we're using FogBugz anyway)
This is by no means the complete answer you'll eventually pick, but here are some tools that will likely factor into it:
repositories without working directories -- if you clone -U or hg update null you get a repository with no working directory (only the .hg). They're better on the server because they take up less room and no one is tempted to edit there
changegroup hooks
For that last one the changegroup hook runs whenever one or more changesets arrive via push or pull and you can have it do some interesting things such as:
push the changesets on to another repo depending on what has arrived
update the receiving repo's working directory
For example one could automate something like this using only the tools described above:
developer pushes five changesets to central-repo/project1/main
last changeset is on branch 'my-experiment' so csets are automatually re-pushed to optionally created repo central-repo/project1/my-experiment
central-repo/project1/my-experiment automatically does hg update tip which is certain to be on the my-expiriment branch
central-repo/project1/my-experiment automatically runs tests in its working dir and if they pass does a 'make dist' that deploys, which might set up database and vhost too
The biggie, and chapter 10 in the mercurial book covers this, is to not have the user waiting on that process. You want the user to push to a repo that contains possibly-okay-code and the automated processed do the CI and deploy work, which if it passes ends up being a likely-okay repo.
In the largest mercurial setup in which I've worked (20 or so developers) we got to the point where our CI system (Hudson) was pulling from the maybe-ok repos for each periodically then building and testing, and handling each branch separately.
Bottom line: all the tools you need to setup whatever you'd like probably already exist, but gluing them together will be one-off sort of work.
What you need to remember is that DVCS (vs. CVCS) introduces another dimension to versioning:
You don't have to rely anymore only on branching (and get a staging workspace from the right branch)
You now have with DVCS the publication workflow (push/pull between repo)
Meaning your staging environment is now a repo (with the full history of the project), checked out at a certain branch:
Many developers can push many different branches to that staging repo: the reconciliation process can be done in isolation within that repo, in a "main" branch of your choice.
Or they can pull that staging branch in their repo and test things out before pushing back.
From Joel's tutorial on Mercurial HgInit
A developer don't necessary have to commit for other to see: the publication process in a DVCS allows for him/her to pull the staging branch first, reconcile any conflict locally, and then push to the staging repo.