My company is moving from Subversion to Mercurial. One of the reasons is with Hg we hope to be able to work more independently.
We are looking forward to using rebasing as our primary way to update from the main repository, at least in the beginning, to keep the history in one line, making the transition from Subversion easier.
Right now, if we need to work independently, we have two options: create a branch in Subversion, and commit there (aka merge hell), or not commit at all. With Mercurial we hope to be able to keep committing locally, and rebase every so often, thus gaining independence while staying free of the administration costs of bcreating named branches.
This all sounds cool until backing up comes into the picture. With Subversion it was obvious that if someone didn't commit, their work ccould be lost. But not committing became inconvenient quickly (no history, no log messages etc.), so people would commit time and again nevertheless.
With Mercurial it is going to be possible to go on committing and rebasing without pushing for extended periods of time, putting much more work at risk. So a question comes up: how to back up the stuff on developers' machines?
One solution would be to use some external backup software, but that doesn't sound like a very good idea.
We could also push to the main repo all the time (maybe even automatically?), but this would make it impossible to use rebasing, and would result in a lot of dangling heads in main.
We could push to a backup repo, and try to have only one head in the main repo. This sound a lot complicated.
Are there any other ways to do this? I'd like to find a solution that would let our developers use most of their Subversion knowledge in the beginning.
Just to throw this out there: I think you're making a mistake. A linear history isn't a big deal and pull/merge is the much more normal mercurial workflow. Embrace a non-linear history and leave rebase for special occasions.
You say "With Mercurial we hope to be able to keep committing locally, and rebase every so often, thus gaining independence while staying free of the administration costs of bcreating named branches.", but in mercurial one uses unnamed branches for this, so there's no administration costs.
See http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/#branching-anonymously for an explanation of how unnamed branches created automatically will give you exactly what you want with zero hassle.
I know it sounds like too-good-to-be-true snake oil, but your folks can just hg pull and hg merge and hg push when they're through and without anyone having to think about branch names, or who has what clone, or anything, you'll have a coordinated center with disconnected work.
You could also give each developer a repo:
https://example.com/repos/awesome-product
https://example.com/repos/awesome-product-wolever
https://example.com/repos/awesome-product-pintér
Where they can push to all they want.
It would mean a lot of dangling heads, but that's probably OK because nobody would look at them until it comes time to restore… Then you'd simply ask for only the ancestors of the current tip:
hg pull https://example.com/repos/awesome-product-wolever -r tip
If I were using this scheme, I'd set:
[paths]
backup = https://example.com/repos/awesome-product-wolever
in .hgrc/hgrc.
Alternatively, you could give each dev one overall backup repo by setting this in their global ~/.hgrc:
[paths]
backup = https://example.com/repos/wolever-backup
I think you're doing it the wrong way.
What you describe looks more like you're trying to put in place a tool to ensure/enforce a process, rather than to support the process.
Working in a decentralized way is quite a big shift compare to the centralized approach, and by trying to put the kind constraint you describe will more than likely cause more hurt than good for the experience perceived by your developers.
If you're scared that they won't push their changes on a regular basis to a main repository, then it would be good to ask yourself (and to ask them too) "Are we ready to change our process?"
Many people still claims that the DVCS will bring chaos to their software factory. It's true that the DVCS'es don't enforce a linear (single-point-of-failure) way to work, but what bring chaos is not the tool, it is the team spirit you built in your company.
Now, if you HAVE TO migrate to Mercurial (because it's already sold to the management level, or whatever), but feels more comfortable with the SVN approach, with a little "plus", try using HgSubversion first. Migration will still be easy later.
Related
I need to collaborate on a Mercurial repository (let's call it "foo") with some people who are novices at version control in general, Mercurial in particular.
I am trying to come up with a workflow that will enable us to use Mercurial without a lot of extra effort on either their end (confusion) or my end (cleanup).
My concern is that as novices I need to expect them to make errors, and I need to allow them to do so in a controlled way, otherwise they won't use the tool at all because they're too scared. But I don't want a bad change to pollute the repository unnecessarily.
I do not expect them to be able to merge properly or to use the mq extension. This is not
a matter of underestimating them, instead it is a realistic assessment given past experience with SVN and my own experience with Hg.
Which of the following approaches would make the most sense? Or if there's a better approach, what is it?
We have a repository foo-submit, read/writable by all, and a repository foo-trunk, readable by all but writable by admins. Users pull from foo-trunk, and push changes to foo-submit. Cleanup: If I find a good change, I let it through as is; if I find a bad change, I "bypass" it by merging with the previous version.
We have a repository foo-trunk readable by all, writable by admins. Each user is responsible for maintaining their own clone which is read-accessible to the rest of the team. When someone wants to push a change, they let me know and I pull it from their repository, with proper cleanup as necessary (same as in #1)
We have a repository foo-dev, read/writable by all, and a repository foo-trunk, readable by all but writable by admins. Users pull/push to foo-dev, and work in named branches if they need to do extensive development. I am responsible for performing merges and cleanup. The foo-trunk repository is merely for having a "clean" copy that has branches where the tip is always in a good state.
Good question, and one that I've never seen a great answer to.
That said, I like option 2. This is the "Pull Request" model used by the Linux Kernel and made popular by GitHub. It allows the admins to act as gatekeepers / reviewers, only allowing good change-sets to get past them when they're happy. If they decide a developer hasn't delivered something worthy, then the pull request is rejected (with reasons). Then the developer can go away, fix up their code / repo, and submit another pull request.
Running a server with something like RhodeCode on it can help keep on top of pull requests. As things grow you can have lower level gatekeepers that deal with subsystems, and higher level gatekeepers that deal with the whole project.
The bit I've never quite got my head around is what should happen to change-sets that are rejected, and that the developer decides to abandon rather than fix up and try again. They could be closed, but then could possibly appear by mistake as part of a future pull request. They'd be harmless, but possibly confusing. The alternative is stripping them, but that sounds like giving people tools they'll cut themselves on.
The other 2 options you give deserve a little comment.
1 is similar to 2. You're still doing a "Pull Request" type flow, but now you have server side branches which mirror the developer's clones. There's little difference and this is how a RhodeCode, GitHub, BitBucket server would let you work, except you don't have to go searching for changes. The server would tell you they're waiting for you to look at.
3 has the problem that everyone's changes are all merged together on foo-dev before you get to them. They would start becoming inter-dependent, and cherry-picking is going to be messy. You'd probably end up grafting change-sets on to foo-trunk which means you're creating new change-sets with new hashes. When the developers pull those they'll now have the change in two places; their original foo-dev version and your grafted foo-trunk version. This doesn't sound sustainable to me.
Best way i can think of if you don't want to use mq (understand with the least hassle for you) is to have your dev
create their own branch for the current feature being developped
merge it back to the main dev branch (or graft/transplant) when it's completed and validated
and then close the branch.
In the long term see for them to learn mq, it's not too hard to grasp.
3a - foo-dev has protected default branch (only some admins can push/merge-to this branch), users use named branches
I'm going through Bitbucket and I can't seem to find any Mercurial repositories that look like what I suspect our repository would look like, provided we switch to Mercurial.
As such, I'm wondering, is there a workflow that we're not considering here?
The thing I'm talking about is that I did a small automated test. We're 14 people that work on the same project, split into 4 scrum teams. To simulate 14 (I picked 10, round number) people working in parallel on the code, using Mercurial DVCS, pushing to the same central master repository, I wrote a script.
I created a new "master" repository, and then cloned it for 10 virtual people
I then ran a 1000 iteration loop, picking a random clone, and doing one of the following:
10% of the time, do a pull from master, merge, commit merge, and push
90% of the time, do a local change and commit
Note that I ensured that there would never be merge conflicts by simply making each virtual person work on his own file.
This would simulate people working locally by doing 1+ commits before pulling, merging, and pushing (to avoid 2+ heads in the master repo). It might be that this workflow is wrong.
This is a sample of what the repository now looks like (screenshot + link to repo):
The repository can be found here: http://hg.vkarlsen.no/hgweb.cgi/parallel_test/graph. Unfortunately this repository is no longer available and I no longer have a copy of the code due to an unfortunate backup incident, but this was just an example for people to visit, it should not be important any more
This looks awfully messy, and as I said, I can't seem to find any repositories that have similar history. By "messy", I mean that it looks like older history of the project will almost always have 10 parallel branches. Close to the top, it tapers off of course, but it will expand as people that are currently working in their local repository pushes to the master.
So I have two questions:
Can anyone show me a repository that has similar history? Since I can't seem to find any, I'm starting to wonder about what kind of conclusions I can draw from that...
Is there something wrong with our workflow (that is, the workflow I've laid out here)? Should we rebase/squash/transplant, delegate push responsibility to one person, other things, instead of the way it was done here?
Impressive preparation!
It always looks messy if you go back a bit and look at all old commits at the same time. It always tapers of, even looking at a small bit old history. See http://hg.intevation.org/mercurial/crew/graph/12402?revcount=120 for instance. This is not the most recent commit, but shows all history up to that commit.
Rebase helps quite a lot, especially if persons are working on separate areas. (I usually check the incoming commits to see if there are potential file or functionality conflicts, and if not, I do rebase.)
Rebase is not fool-proof though, so merge is the preferred "safe" action, but it leaves more "garbage" in the history. A trade-off.
Rebase is sort-of like the bog standard SVN update. The existing stuff is made baseline and your changes go on top, cross your fingers it still works. It's useful, but there are times when you feel safer having yours, theirs and the merge as separate commits in the history.
There is also commit-squashing as an option (histedit extension maybe), which squashes all in-between commits to one. This is useful when you're about to push and want to transferring many partials commits in your own repo as a single commit to the main.
I have 12 developers working in the same Mercurial repository at work, and our history looks nothing like that. There are occasional merge commits, but most merges are from merging actual branches, i.e there might be a merge in our main development branch bringing in changes from a bugfix release made on the production/release branch.
This is very easy to achieve, developers hack and commit to their local repository and when they have something stable enough to share with the rest of the team they push.
If nothing has been committed since they started committing the push goes through without problems.
If someone else has committed a change, Mercurial complains that the push will create remote heads. The developer then does a hg pull --rebase and retries the push. The push goes through and everyone is happy.
If you are using continuous integration with developers regularly pushing to a shared repository, this is the way to go. Knowing whether you have pushed changes or not is easy and you avoid lots of useless merge commits cluttering up your history.
I'm struggling to find the mercurial workflow that fits the way that we work.
I'm currently favouring a clone per feature but that is quite a change in mindset moving from Subversion. We'll also have issues with the current expense we have in setting up environments.
Using hg pull --rebase seems to give us more of a Subversion-like workflow but from reading around I'm wary of using it.
I think I understand the concepts and I can see that rewriting the history is not ideal but I can't seem to come up with any scenarios which I personally would consider unacceptable.
I'd like to know what are the 'worst' scenarios that hg pull --rebase could create either theoretical or from experience. I'd like concrete examples rather than views on whether you 'should' rewrite history. Not that I'm against people having opinions, just that there already seem to be a lot of them expressed on the internet without many examples to back them up ;)
The first thing new Mercurial converts need to learn is to get comfortable committing incomplete code. Subversion taught us that you shouldn't commit broken code. Now it's time to unlearn that habit. Committing frequently gives you a lot more flexibility in your workflow.
The main problem I see with hg pull --rebase is the ability to break a merge without any way to undo. The DVCS model is based on the idea of tracking history explicitly, and rebasing subverts that idea by saying that all of my changes came after all of your changes, even though we were really working on them at the same time. And because I don't know what your changes are (because I was basing my code off of earlier changesets) it's harder for me to know that my code, on top of yours, won't break something. You also lose the branching capabilities by rebasing, which is really the whole idea behind DVCSs.
Our workflow (which we've built an entire Mercurial hosting system around) is based on keeping multiple clones, or branch repositories, as we call them. Each dev or small team has their own branch repository, which is just a clone of the "central" repository. All of my new features and large bug fixes go into my personal branch repo. I can get that code peer reviewed, and once it's deemed ready, I can merge it into the central repo.
This gives me a few nice benefits. First, I won't be breaking the build, as all of my changes are in their own repo until they're "ready". Second, I can make another branch repo if I need to do a separate feature, or if I have something longer-running, like for the next major version. And third, I can easily get a change into the central repo if there's a bug that needs to be fixed quickly.
That said, there are a couple different ways you can use this workflow. The most simple, and the one I started with, is just keeping separate clones. So I'll have website-central, website-tghw, etc. It works well, especially since you can push and pull between them locally. More recently, I've started keeping multiple heads in the same repo, using the remotebranches extension to help manage them and hg nudge to keep from pushing everything at once.
Of course, some people don't like this workflow as much, usually because their Mercurial server makes it hard to make server-side clones. In that case, you can also look at using named branches to help keep your features straight. Unfortunately, they're not quite as flexible as Git branches (which is why we prefer branch repos) but they work well once you understand how to close branches, and why you can't really get rid of them once you start one.
This is getting a bit long, so I'll wrap it up by encouraging you to embrace the superior branching and merging that Mercurial provides (over SVN). There is definitely a learning curve, but once you get the hang of it, it really does make things easier.
From the question comments, your root issue is that you have developers working on several features/bug fixes/issues at one time and having uncommitted work in their working directory along with some completed work that is ready to be pushed back to the central repository.
There's a really nice exchange that covers the issue well and leads on to a number of ways forward.
http://thread.gmane.org/gmane.comp.version-control.mercurial.general/19704
There are ways you can get around keeping your uncommitted changes, e.g. by having a separate clone to handle merges, but my advice would be to embrace the distributed way of working and commit as often as you like - if you really feel the need you can combine the last few local commits into a single changeset (using MQ, for example) before pushing.
I'm working in a team of 3 developers and we have recently switched from CVS to Mercurial. We are using Mercurial by having local repositories on each of our workstations and pulling/pushing to a development server. I'm not sure this is the best workflow, as it is easy to forget to Push after a Commit, and 3 way merge conflicts can cause a real headache. Is there a better workflow we could use, as I think the complexity of distributed VC is outweighing the benefits at the moment.
Thanks
If you are running into a lot of 3 way merges it might be because you have too much overlap in what you and your team members are working on. Mercurial is pretty good at handling merges itself, so long as you all aren't editing the exact same lines of a file. If possible, you could divide up the work more clearly and avoid some of the headaches of large merges. Also note that this would still be a problem with CVS since it's arguably worse at merging than mercurial.
You also don't need to push after every commit. Your workflow could look something like this:
Commit part of some feature.
Commit some more of some feature.
Commit last part of feature.
Commit bug fixes for stupid mistakes.
Push full feature to repo.
To an extent, this looks like Going Dark, but that can be alleviated by making sure that the features in the above example are smallish in scope.
Forget all you know about CVS. Mercurial is nothing like it even if some commands feel somewhat similar.
Read http://hginit.com/. Follow the examples.
Forget all you know about CVS.
I mean it. This is the hardest part. Learn to trust your tool.
It sounds like you're all making your changes to the same branch. This has the unsatisfying side-effect that you're merging each others' changes on almost every single commit, which would be fine except that manually intervening for conflicts isn't something you want to do every time you push.
Here's the workflow I would suggest. The idea is to use branching more heavily, so you need to merge to the master branch less often.
Have every developer develop every feature in a separate branch. This way:
you avoid constantly merging changes from other people, and
you are free of the pressure to push incomplete work before the next guy, "makes it hard to merge."
When a feature is "done" and if the changes would appear to apply cleanly (a judgement call), merge the feature branch directly into the master branch and delete the feature branch.
If a feature falls way behind the master branch (many features merged), or if the merge otherwise appears difficult:
merge master into the feature branch.
Find and fix any bugs in contented isolation from other developers.
Assuming the feature is ready to go, merge it into master (notice: now the merge in this direction will be clean by definition). If not, you can just continue developing.
We are using Mercurial by having local repositories on each of our workstations and pulling/pushing to a development server.
That sounds fine to me. My team is about double the size and it works great.
I'm not sure this is the best workflow, as it is easy to forget to Push after a Commit,
You don't have to push after every commit; you push when you want to push. That's the big idea about DVCS: that Commit and Push are distinct!
and 3 way merge conflicts can cause a real headache.
Are you working on the same lines of code a lot? On my team of 5-6 programmers, pushing/pulling a few times a day, and committing up to a couple dozen times a day, I can't remember the last time I've had to manually resolve merge conflicts. Certainly not in the past month or two.
Is there a better workflow we could use, as I think the complexity of distributed VC is outweighing the benefits at the moment.
Perhaps you should describe your workflow in more detail, because the only complexity over centralized version control that I encounter on a typical workday is maybe one command, and the benefits are huge. Doing "hg blame" just once saves me more time over the centralized version than all the "hg push"es I've had to type all year!
For what it's worth, we're a similar size team working with Mercurial for the first time and we started with the same problem.
We persisted and things are now significantly better. I think most of the problems occurred when the codebase was tiny and people were all trying to work on the same thing. Now that it's a little more established people aren't treading on each others' toes quite so much and the Paris much reduced.
Hope you get it sorted!
Subversion shop considering switching to Mercurial, trying to figure out in advance what all the complaints from developers are going to be. There's one fairly common use case here that I can't see how to handle.
I'm working on some largish feature, and I have a significant part of the code -- or possibly several significant parts of the code -- in pieces all over the garage floor, totally unsuitable for checkin, maybe not even compiling.
An urgent bugfix request comes in. The fix is nice and local and doesn't touch any of the code I've been working on.
I make the fix in my working copy.
Now what?
I've looked at "Mercurial cherry picking changes for commit" and "best practices in mercurial: branch vs. clone, and partial merges?" and all the suggestions seem to be extensions of varying complexity, from Record and Shelve to Queues.
The fact that there apparently isn't any core functionality for this makes me suspect that in some sense this working style is Doing It Wrong. What would a Mercurial-like solution to this use case look like?
Edited to add: git, by contrast, seems designed for this workflow: git add the bugfix files, don't git add anything else (or git reset HEAD anything you might have already added), git commit.
Here's how I would handle the case:
have a dev branch
have feature branches
have a personal branch
have a stable branch.
In your scenario, I would be committing frequently to my branch off the feature branch.
When the request came in, I would hg up -r XYZ where XYZ is the rev number that they are running, then branch a new feature branch off of that(or up branchname, whatever).
Perform work, then merge into the stable branch after the work is tested.
Switch back to my work and merge up from the top feature branch commit node, thus integrating the two streams of effort.
Lots of useful functionality for Mercurial is provided in the form of extensions -- don't be afraid to use them.
As for your question, record provides what you call partial commits (it allows you to select which hunks of changes you want to commit). On the other hand, shelve allows to temporarily make your working copy clean, while keeping the changes locally. Once you commit the bug fix, you can unshelve the changes and continue working.
The canonical way to go around this (i.e. using only core) would probably be to make a clone (note that local clones are cheap as hardlinks are created instead of copies).
You would clone the repository (i.e. create a bug-fix branch in SVN terms) and do the fix from there.
Alternatively if it really is a quick fix you can use the -I option on commit to explicitly check-in individual files.
Like any DVCS, branching is your friend. Branching a repository multiple ways is the bread and butter of these system. Here's a git model you might consider adopting that works quite well with Mercurial, also.
In addition to what Santa said about branching being your friend...
Small-granularity commits are your friend. Rather than making lots of code changes in a single commit, make each logically self-contained code change in its own commit. Then it will be a lot easier to cherry-pick changes to merge between branches.
Don't use Mercurial without using the Mq Extension (it comes pre-packaged in the default installation). In addition to solving your specific problem, it solves a lot of other general problems and really should be the default way that you work (especially if you're using an IDE that doesn't integrate directly with Hg, making switching branches on the fly a difficult way to work).