How do I fork a continually-evolving project to customise for a specific client, while continually pulling improvements from the major branch without losing my customisations?
Should I just keep pulling from the main repository, merging appropriate changes and never push back?
I haven't been using Mercurial long, so I am concerned about integrating the two projects without making a mess of changed namespaces and different domain (client) logic.
Update: It looks like using separate repositories which pull from a main repository is the way to go.
The way you're using it:
Should I just keep pulling from the main repository, merging appropriate changes and never push back?
is precisely my preferred way of handling this. Any change that applies to multiple customers gets done in the main repo and pulled into the per-customer repos. Any per-customer changes are done in the per-customer repo and never pushed back to main.
We use named branches. Make a branch from the default with the customizations and merge the changes from the default into the named branch as needed.
Related
I have four devs working in four separate source folders in a mercurial repo. Why do they have to merge all the time and pollute the repo with merge changesets? It annoys them and it annoys me.
Is there a better way to do this?
Assuming the changes really don't conflict, you can use the rebase extension in lieu of merging.
First, put this in your .hgrc file:
[extensions]
rebase =
Now, instead of merging, just do hg rebase. It will "detach" your local changesets and move them to be descendants of the public tip. You can also pass various arguments to modify what gets rebased.
Again, this is not a good idea if your developers are going to encounter physical merge conflicts, or logical conflicts (e.g. Alice changed a feature in file A at the same time as Bob altered related functionality in file B). In those cases, you should probably use a real merge in order to properly represent the relevant history. hg rebase can be easily aborted if physical conflicts are encountered, but it's a good idea to check for logical conflicts by hand, since the extension cannot detect those automatically.
Your development team are committing little and often; this is just what you want so you don't want to change that habit for the sake of a clean line of commits.
#Kevin has described using the rebase extension and I agree that can work fine. However, you'll also see all the work sequence of each developer squished together in a single line of commits. If you're working on a stable code base and just submitting quick single-commit fixes then that may be fine - if you have ongoing lines of development then you might not won't want to lose the continuity of a developer's commits.
Another option is to split your repository into smaller self-contained repositories.
If your developers are always working in 4 separate folders, perhaps the contents of these folders can be modularised and stored as separate Mercurial repositories. You could then have a separate master repository that brought all these smaller repositories together within the sub-repository framework.
Mercurial is distributed, it means that if you have a central repository, every developer also has a private repository on his/her workstation, and also a working copy of course.
So now let's suppose that they make a change and commit it, i.e., to their private repository. When they want to hg push two things can happen:
either they are the first one to push a new changeset on the central server, then no merge will be required, or
either somebody else, starting from the same version, has committed and pushed before them. We can see that there is a fork here: from the same starting point Mercurial has two different directions, thus a merge is required, even if there is no conflict, because we do not want four different divergent contexts on the central server (which by the way is possible with Mercurial, they are called heads and you can force the push without merge, but you still have the divergence, no magic, and this is probably not what you want because you want to be able to checkout the sum of all the contributions..).
Now how to avoid performing merges is quite simple: you need to tell your developers to integrate others changes before committing their own changes:
$ hg pull
$ hg update
$ hg commit -m"..."
$ hg push
When the commit is made against the latest central version, no merge should be required.
If they where working on the same code, after pull and update some running of tests would be required as well to ensure that what was working in isolation still works when other developers work have been integrated. Taking others contributions frequently and pushing our own changes also frequently is called continuous integration and ensures that integration issues are discovered quickly.
Hope it'll help.
I have an active project with some sensitive files and directories. I want to hire an external contractor to do some simple UI work. However, I don't want the contractor to have access to some directories and files. My project is in mercurial on Bitbucket.
What is the best way to clean up the project and give him access to commit his changes? I thought about forking into a new repository, but I am worried about removing directories I don't want him to have access to.
How to I remove them so they don't appear the original changesets? How to I merge his repo back without it removing those directories in my main repository? Is a fork the way to go?
Naturally a repository needs access to its whole history in order to self-check its integrity. I don't know of a way to selectively hide parts of the repository (there's the ACL extension, but it is for write access only).
In your case, I would
create a new repository where all sensitive information has been stripped off (use the convert extension for that task).
Then I would let the external guy work with that repository.
Once his work is finsihed, pull his repository into a clone of the original one (using -f to force pulling of an unrelated repository), and
rebase his first changeset and all its children onto a head of your original repository.
Finally, push the rebased head to the original repository.
For steps 3 to 5 you don't necessarily have to wait until the external developer is done. Rebasing intermediate states of his repository is also possible.
Yet, it's an theoretical idea .. one has to see how it performs in practice.
Alternative: In case you frequently have external contractors who shouldn't see some parts of your code, I would second #Anton's comment to setup permission related multiple repositories.
There are multiple ways to do this:
Using sub-repositories
Using multiple repositories
???
Regardless, you need to restructure and split your existing repository, so this will create havoc if you have lots of people working on this project, they will all need to stop working, synchronize their work, destroy their local clones and clone down fresh copies after the restructuring.
One way using multiple repositories would be that you do the following:
Make 2 extra clones of the repository (keep one around for fallback if everything fails, you can always go back)
The first clone you need to run the hg convert command on to get rid of all the bits and pieces your contractor should not access
Then you fix that repository so that it works by itself. You might have to change code to provide hooks and events for anything not present but which you intend to inject into the project before you build
Then you need to run hg convert on the other clone to get rid of everything now present in the first.
Then you pull from the first (contractor) repository into the second (private) repository, merge, and do necessary fix-ups so that the code still works as intended
What you have now is two repositories:
Contractor-repository, with only the bits you want to expose
A private repository, that has pulled and merged from the contractor-repository, and contains all the other bits and pieces
From now on, whenever the contractor has pushed work to his repository, you need to pull from it and into the private repository and then merge.
Your repositories would look like this:
Contractor: ---97---98---99---100---102---103---104
M M
Private: ---91---92---93---94---95---96---101---105---106---107
/ /
/ /
---97---98---99---100---102---103---104
The two changesets with M above are merge-changesets that merge contractor-supplied code into your private repository.
Note that you too would have to commit code to the contractor-repository, to work on and fix bugs in the code there, but all the private bits you can keep private.
We've worked out a repository structure we'd like to maintain and under that structure will prevent RepoA from being able to pull from RepoB. How can I setup a repo so that it can only push to a certain repo but not pull from it?
You can technically push/pull from any location and would probably try to avoid mucking with that flexibility unless you're good at writing hooks. And, if anyone has write access to UAT, you cannot prevent any changes from being pushed to UAT as you will need to do that when some new bits need to enter the UAT branch for testing.
What it sounds like you are trying to do is preserve a "stable" while allowing work to continue on an "anonymous" branch (your alpha) that was cloned from UAT. Eventually, you have to merge that back into UAT, so I would really just give a few senior-level developers write access to UAT and trust that they follow the proper procedure when working with the branches.
I suggest reviewing the Guide to Branching and the Managing Releases section of the Hg Book before trying to invent a new way to seemingly protect your branches.
Edit: I did find a similar question for preventing the default push, but allowing pulls. It shows you the basics on implementing a preoutgoing hook, which is not what you want, but similar in nature.
As a former user of Subversion, we've decide to move over to Mercurial for SCM and it is confusing us a little. Although Mercurial is a distributed SCM tool we are using a remote repo to keep changes we make backed up on a server but we are finding a few teething troubles.
For example, when two or three of us work on our local repo's, we commit then push to the remote repo, we find that a lot of heads(?) are created. This confused the hell out of us and we had to do some merging etc to sort it out.
What is the best way to avoid so many heads and to keep a remote repo in sync with a number of developers?
Today, i've been working like this:
Change a File.
Pull from remote repo.
Update local working copy.
Merge? (why?)
Commit my changes to local repo.
Push to the remote repo.
Is this the best proceedure?
Although this has worked fine today, i can't help that feeling that i'm doing it wrong! To be honest i don't understand why merging even needs to be done at the pull stage because other people are working on different files?
Other than to tell me to RTFM have you any tips for using Mercurial is such a way? Any good online resources for information on why we get so many heads?
NOTE: I have read the manual but it doesn't really give much detail and i don't think i want to start another book at the minute.
You should definitely find some learning resources.
I can recommend the following:
hginit.com
Tekpub: Mercurial
As for your concrete question, "is this the best procedure", then I would have to say no.
Here's some tips.
First of all, you don't need to stay "in sync" with the central repository at all times. Instead, follow these guidelines:
Push from your local repository to the central one when you're happy with the changes you've committed. Remember, this can be several changesets
Pull if you need changes others have done right away, ie. there's a bugfix a colleague of yours has fixed, that you need, in order to continue with your own work.
Pull before push
Merge any extra heads you pulled down with your own changes, before you push, or continue working
In other words, here's a typical day.
You pull the latest changes when you come in in the morning, so that you got an up to date local clone. You might not always do this, if you're in the middle of bigger changes that you didn't finish yesterday.
Then you start working. You commit small changesets with isolated changes That isn't to say that you split up a larger bugfix into many smaller commits just because you modify multiple files, but try to avoid fixing more than one bug at a time, or implementing more than one feature at a time. Try to stay focused.
Then, when you're happy with all the changesets you've added locally, you decide to push to the server. When you try to do this, you get an abort message saying that extra heads would be pushed to the server, and this isn't allowed, so the push is aborted.
Instead you pull. This can always be done, but will of course now add extra heads in your local clone, instead of at the server.
Then you merge, the extra head that you got from the server, with your own head, the one that you created during the day by committing new changesets to your clone. You resolve any merge conflicts.
Then you push, and now it should succeed. On the off chance that someone has managed to push more changesets to the central repository while you were busy merging, you will get another abort and have to rinse and repeat.
The history will now show multiple parallel branches of development, but should always stay at max 1 head in your central repository. If, later on, you start using named branches, you can have 1 head per named branch, but try to avoid this until you get the hang of just the default branch.
As for why you need to merge? Well, Mercurial always work with revisions that are snapshots of the entire project, which means two branches, even though they contain changes to different files, are really considered two different versions of the entire project, and you need to tell Mercurial that it should combine them to get back to one version.
For one, you can pull at any time; pulling does just add changesets to your repo, but not change your local working files (except if you have enabed the post-pull update).
Merging is necessary if someone else has commited changes to the same branch you're currently working on. This created an implicit branch, and merging merely brings them back together. You can see this nicely with the "railroad track" in the repository view. Basically, as long as you don't merge, you stay on your own "private" track, and when you want to add your changes (can be any amount of changesets) you merge it back into the destination branch (typically "default"). It's painless - nothing like merging in older SVN versions!
So the workflow is not as rigid as you displayed it; it's more like this:
Pull as much as you like
Make changes and commit locally as often as you like
When your changes should be integrated, merge with the destination branch (can be a lower revision than the newest), commit and push
This workflow can be tuned somewhat, for instance by using named branches and sometimes by using rebase. However, you and your team should decide on the workflow to be used; Mercurial is quite flexible in this regard.
http://hginit.com has a good tutorial.
In particular, you'll find the list of steps you have here: http://hginit.com/02.html (at the bottom of the page)
The difference between those steps and yours is that you should commit after step 1. In fact you will typically commit several times on your local repository before moving onto the pull/merge/push step. You don't need to share every commit with the rest of developers right away. It'll often make sense to do several related changes and then push that whole thing.
I'm trying to determine how people use "branch repositories" while also using subrepos.
Let's say I have repo Main containing a solution file (.NET), and populated with subrepos A, B, C:
/Main
- A
- B
- C
MainSolution.sln
A, B, and C, while being shared between other "Main" repos, are very tightly integrated into Main project. Thus, a major feature to the Main repo will require modifications to the subrepos (i.e., they are shared libraries, but are very actively developed).
Now it is time to add a feature. This feature is too big for one person to handle, and thus the code will need to be pushed to the central repo so others can help. We'd also need to be able to go back to the last "stable" code before the feature development began in case a bugfix is needed. I believe I have two options at this point: (1) create a named branch in the Main repo, or (2) create a new clone of Main. Since there are subrepos, both of these options have repercussions not present typically.
Option 1) Creating a named branch will, I presume, allow modifications to the subrepos to be committed/pushed, but only other people who have also updated to that branch in their clone of Main will be affected, since the .hgsubstate file is tracked. However, the subrepos will get a new head, and thus the (possibly) experimental feature would end up getting pushed to the central repo. Am I understanding this correctly?
Option 2) There are numerous advocates for the "don't use named branches, use 'branch repositories'", which are literally clones of the main repo, but named differently and existing on the central server. This is a little appealing to me, as it seems to keep things separated (and thus detached from disaster as co-workers --and myself!-- are still learning Mercurial). But this workflow seems completely broken when subrepositories are involved, since creating a clone of the Main repo does not create new, separated clones of the subrepos. It's a new clone, but it's still pointing at the same subrepos, and thus changes made to them will find their way back into the subrepos! I realize this is by design, and it's one of the really cool things (to me) about Mercurial. But how on earth do people use this branch repository workflow with subrepositories? It is completely inconceivable that, for each feature/experiment/version/whatever, I'm going to create a new clone (on the central server) of the Main repo, AND create clones (on the central server) of the subrepos, AND modify all the .hgrc/.hgsub paths to point to the proper central repos.
At this point, I'm just trying to understand HOW people work on a complicated project and use subrepos with branch repositories. Any thoughts?
You have other options as well. You could use bookmarks, for example. Since version 1.9, bookmarks can be pushed and pulled, they're not just local anymore. Since you often don't want a development "branch" to stick around as a named branch after that new feature is completed, bookmarks are often a better choice for that kind of thing. I tend to use bookmarks for new development and save real branches for released versions.
You should also be aware that subrepositories don't have to be shared between multiple main repositories in the way you describe. You can actually have the subrepositories stored inside a main repository (as opposed to having them at the same level as the main repos, or stored in some other location entirely), which would make them unique to each main repository, except you can push and pull from the subrepos in other main repos when you want to share those changes. This is the way I usually do it.
Unfortunately much of this is difficult to explain without a whiteboard, so please let me know if this isn't clear.
I prefer named branches for features that will most likely eventually get merged into the default branch. It is much easier to switch branches than switch repos.
With named branches you never need to worry about accidentally pushing your unstable branch of development into the stable repo. The named branch is already there, but won't be retrieved via an update unless a developer asks for it.