My company's product is module-based, meaning we ship with five base modules and users can purchase additional ones. We're using Mercurial, to which we are relatively new, for our source control, and since we've released 1.0 of our product, managing the separate module development has been a nightmare.
We want to be able to release minor bugfix updates without having to wait for particular module development to be complete, so one repo for everything doesn't work very well. I read about branching but the Definitive Guide seems to suggest that branching is temporary, and that merging with it is difficult.
Ideally, we'd have a base repo that is the product and then different repos (or branches) with the extra modules, so that QA could build the main product and the main+addons separately, while the developers working on ModuleA don't impact the developers working on BugfixB. I tried this with multiple subrepos but it ended up corrupting my repositories.
Should I be looking at using named branches? Or bookmarks?
I'm looking for suggestions on best practices on how we can take advantage of Mercurial's features to make this process easier.
Thanks!
There is a good tutorial about branching at http://nvie.com/git-model. The main point is to have
a release branch which contains only merges from completed release/bugfix branches
development branches for bug fixes or features
own branches for long-term features
Also there is a reference about the technical differences in mercurial branches at http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/
Branching is your solution. I consider named branches to be a Good Thing. However, you should be aware that named branches require a certain level of forethought and discipline in use.
I would suggest that each bug-fix gets its own branch. Developers will fork off that branch, do the bugfix, merge back into the feature-branch.
I would consider splitting your modules into separate repositories, one for each product. Possibly that's not very useful; you'll have to go over different use cases there and determine how the workflow/compile-flow would go.
I don't see why you'd consider having different subrepos for this when the file history is virtually the same throughout - this is a prime job for branches. The only complication is being able to cherry-pick patches for each branch - that may require you to export a patch (or set of patches) and apply them individually to each branch. It's a bit more awkward than it should be, but it's no harder than doing the same across different repositories.
I think the question blurs two different issues:
You have a modular product
You have separate development cycles for each module
For handling the modular product you should use different repositories for each module and bring them together using subrepos as appropriate for each customer configuration. It appears you're already doing this but are having corruption issues. This is certainly the correct way to go so you need to bottom-out whether the corruption is coming from a Mercurial bug or user error.
For handling separate development cycles then personally I'd go for module clones but named branches would also be fine.
Hope this helps.
I am new to Mercurial as well, but I think that your problem is not specific to it.
You need to think about the process of releasing code and the parties involved, and map this model to a branch layout that can support it.
Mercurial is nice because it can support developers well, by allowing them to maintain their own development "branches" without affecting a continuous build or other downstream processes (QA, installers, etc).
[Rel]
^
[RC]
^
[QA]----[QA]------[QA]
^ ^ ^
[Dev]---------------------------------------------------------
^ ^ ^
[Jen] [Paul] [Ken]
this is a possible scheme, where developers merge to Dev, and somebody merges regularly to the [QA] branch, and when that it baked nice goes to [RC] etc.
Every release stays isolated from other activity.
Good Luck!
Related
We are switching to Mercurial. We are have been using SVN for a number of years. I recall reading somewhere that Twitter only have two branches in their source control. A production branch and a dev branch. This I think will suit us because we offer a service and there is only one instance of it running and we control it completely so we dont have to worry about versions. Just bring able to do fixes for production and maybe merge that change into dev if the issue exists in the dev branch too.
Is this approach appropriate for Mercurial? Are there any hidden gotchas that we should be aware of? We plan do follow a more traditional layout with release branches for the one or two products that we have that we retail.
Yes, no gotchas. The most recognized branching model, for both Git and Mercurial, can be found here, which gives an outline of a more complexe structure than you are thinking of using.
And even when your environment continue to grow, mercurial will still be able to handle your changing needs.
Are there any hidden gotchas that we should be aware of?
Yes, they are, if you will have more than one simultaneous tasks in devel you'll get hardly readable and understandable history with a lot of anonymous branches in devel rather fast
Yes, Mercurial achieves fully your approach. May be the workflow you are proposing is this. The Mercurial code or the Python code are some examples of development with really few branches.
In Mercurial the default branch is used as default when cloning, so is usually used as the development branch. Other branches are used as feature or stable branches, but is up to you. Actually for not so long features and issues one may use bookmarks on the development branch.
And remember to commit often :)
I am currently using Mercurial, along with the Guestrepo extension, to manage and version the different components of a project. I have come to a quite stable workflow to manage the different versions of the components.
However, I can't come up with an effective solution when it comes to versioning subtle variations of a component. This is, for example, a slightly different embedded device driver (different serial port speed for example), or a GUI which is written in English instead of German.
I don't think stacking them in the Release/Stable branch is a good workflow, as the proliferation of different configurations (English,Spanish,Chinese,...) could lead to a serious and nonsense bloat of the Release branch.
On the other hand, creating a separate Release branch for each, would lead me to many, many branches, which is not the best solution IMHO.
Creating separate repositories for each of the configurations would suppose a quite tedious task whenever a structural change had to be made, as all of the repos would have to be updated with that change.
Any idea on this?
Thank you.
As #EldadAK suggests, creating a repo for each configuration and importing core functionalities from other repos seems a nice idea.
However, I still can't figure out how to arrange "same but slightly different" components, which differ in some subtle features but share their core.
Is it a code architecture issue? Should the components be refactored so that the main core and the differing features lay in different components, which are related using custom builds for each configuration?
IMHO, you should keep all changes in your main branch. Managing all branches or even multiple repositories is not scalable and will eventually get out of hand.
I don't think you should care about the size of your release branch. By keeping it all together, you will always know where you are, what is included in a release and when you really need to branch, have all the changes accumulated to that point.
It's my personal opinion that you should try and keep it simple to manage looking many revisions and years ahead...
Note - Project managers tend to think about next week. You need to think about next year...
I hope this helps.
effective solution when it comes to versioning subtle variations of a component
While I can't see any serious drawbacks from using named branches in one repo, you can use another ("default" de-facto) solution for configuration management inside Mercurial: MQ
You have only slightly adopt your workflow (same amount of branches, same amount of repos) to MQ
My development env: Windows 7, TortoiseHg, ASP.NET 4.0/MVC3
Test branch: code on test server
Prod branch: code on production server
This is my current branching model. The reason to branch out every task (feature) is because some features go to live slower. So in above graph, task 1 finished earlier (changeset #5), and merge into test branch for testing. However, due to bug or modification of original request, changesets #10, #12 have been made. While task 2 has finished testing #8 and pushed to live #9 already.
My problem is every time when modifying task branch (like #10, #12), I have to do another merge to test branch (#11, #13), this makes the graph very messy.
Is there any way to solve this issue? Or any better branching model?
It really sounds like you are trying to implement a feature branching strategy. However, based on your diagram, I think you're missing a few steps and/or are merging the wrong branches. In essence, you should probably have more like 4 lines of development there, plus 1 representing all feature branches. Unfortunately, I have not been able to find a nice diagram, except for one talking about Git and a workable branching strategy here. The diagram, though, better explains what you are looking for, even if you are using a different DVCS like Mercurial (my fav). Using Steve Losh's Guide to Branching in Mercurial and the Hg Book, you should be able to implement a good feature branching strategy that works for you. Steve's got pros/cons for each approach.
And, no, you don't need to clone to branch properly. Mercurial has named branches that allow you to switch between branches easily if you are commonly working on multiple unfinished items and/or performing code reviews/testing for other developers. With any kind of Web-based development with IIS, named branches are easier to work with as things don't move and different version can continue to work with the same configuration under IIS.
I must say, though, that feature branching or whatever name you give it is almost always a bad idea as branches that run too long (say, a year, which I have seen with disastrous results) can be almost impossible to merge back in unless you are frequently (daily) managing synchronization between the feature branch and its parent. This type of maintenance overhead is not worth the trouble and you're better off sticking to trunk-based development with release branches for bug fixes and fix your code to abstract code that is production-used and unfinished work.
When you want to work on a new feature, you better make clone from test repo. Branches in mercurial are supposed to stay unrelated... Imagine that you made a release v1.0 and going to work on v2.0 of your application (default branch). You will create branch v1.0 to keep it updated with bug fixes.
You can use separate repositories for each branch instead. Then rebase the changes on the latest changeset. This could reduce the number of merges.
We are looking to move our team (~10 developers) from SVN to mercurial. We are trying to figure out how to manage our workflow. In particular, we are trying to see if creating remote heads is the right solution.
We currently have a very large repository with multiple, related projects. They share a lot of code, but pieces of the project are deployed by different teams (3 teams) independent of other portions of the code-base. So each team is working on concurrent large features.
The way we currently handles this in SVN are branches. Team1 has a branch for Feature1, same deal for the other teams. When Team1 finishes their change, it gets merged into the trunk and deployed out. The other teams follow suite when their project is complete, merging of course.
So my initial thought are using Named Branches for these situations. Team1 makes a Feature1 branch off of the default branch in Hg. Now, here is the question. Should the team PUSH that branch, in it's current/half-state to the repository. This will create a second head in the core repo.
My initial reaction was "NO!" as it seems like a bad idea. Handling multiple heads on our repository just sounds awful, but there are some advantages...
First, the teams want to setup Continuous Integration to build this branch during their development cycle (months long). This will only work if the CI can pull this branch from the repo. This is something we do now with SVN, copy a CI build and change the branch. Easy.
Second, it makes it easier for any team member to jump onto the branch and start working. Without pushing to the core repo, they would have to receive a push from a developer on that team with the changeset information. It is also possible to lose local commits to hardware failure. The chances increase a lot if it's a branch by a single developer who has followed the "don't push until finished" approach.
And lastly is just for ease of use. The developers can easily just commit and push on their branch at any time without consequence (as they do today, in their SVN branches).
Is there a better way to handle this scenario that I may be missing? I just want a veteran's opinion before moving forward with the strategy.
For bug fixes we like the general workflow of Mercurial, anonymous branches that only consist of 1-2 commits. The simplicity is great for those cases.
By the way, I've read this, great article which seems to favor Named branches.
You're definitely thinking about this right, and it sounds like you're going down a good path. I'm a branches as clones person, but named branches have come a long way.
Having a central-ish repo to which all named branches are pushed is convenient for control and backups. Teams working on only branch X can easily create their own branch X only repo by doing hg clone -r X central-ish repo.
The best thing you can do to help the teams out is to let them do clones themselves somewhere that's sitting behind a hgwebdir.cgi instance (as, presumably your central-ish repo will be). You'll find not just teams, but sub-teams and pairs of teams will set up their own repos for mini-efforts you never new about. They'll put them on the named branches that make sense to them and merge back into central as appropriate.
I would make the decision if these three projects should go into one repository by the coupling between these projects (and how many patches are interchanged within them). The more independent they are the less are the advantages of having them in one repo (backup and management aside). There are some different kind of setups:
As you showed, one repository, with one branch for the shared code, and one branch for each project. When the projects itself are generated by forking the shared code base care must be taken when merging back to common (cherry-picking). When inside of each project-branch updates to the common-branch are generated as direct ancestors of the common-branch, and get merges into the project-branch, chances are good they can also be merged back into common. But if changes to common are developed on top of the project branch, merging back will require cherry-picking. I don't have experiences with such a setup, but I fear that the merges can get problematic.
one repo for the shared code and one for each project, connected by symlinks or as subrepo. Here care must be taken to not step on each others feed. In my experience this kind of usage has the potential to grow into a very big PITA. OTOH you seem to have this setup already and your fellow developers can work with it.
one repo for shared and one for each project, with the code from the shared one used as internal releases. I would go for this setup when there are not big regular changes on the shared code base.
All these situations can also be combined with customization-branches for each project within the common part. But I would try to minimize the number of currently active branches, since every new branch requires additional care of merges.
I'm sorry to not give a concrete answer, but "The right thing" (TM) depends to much on the local details.
I'm looking at the mercurial handbook, chapter 6 "Working with multiple branches". In there the author states that if you have separate versions/branches of the same software that it makes sense in an implied obvious way to host each branch of the software in a separate repository.
I know that Mercurial supports tags (which is the way that branches are done in subversion AFAIK). Why would you use different repositories instead of tags for branch managing?
Mercurial's tags are not the same as Subversion's branches. In Subversion, making a branch creates a separate copy of the code (with hardlinks at least, I think) while tags in Mercurial just point to a specific commit.
A while ago I wrote about Mercurial's branching models (of which branching-with-clones is one of four options), maybe you'd find it useful?
There are advantages in having different repositories in Mercurial, and problem in handling branches.
Branches mean multiple heads, the graphs are much more complicated then, and even the graphical representation may not be able to accommodate so many convoluted paths... and I don't even talk about the human brain!
On the other hand, having multiple repositories mean that each repository will have a much simpler structure, therefore easing the brain trauma of having to deal with multiple branches/merges (that you have anyway, since two developers working from the same changeset and onward develop concurrently).
Furthermore, with multiple repositories, you can read/edit any file on a given repository easily with whatever editor you are using (if you maintain the working dir up to date relatively to the tip).
On subversion you HAVE to deal with multiple branches, there is no other way around, and you have to use tags.
On Mercurial, tags are not supposed to move (it needlessly introduce changesets) and branches are handled off-the-shelf: you always have branches. However, since you can have multiple repositories, you are offered another dimension. It is your choice whether you use it or not, it made my life easier anyway.