I'm a big fan of Mercurial for my open-source projects, but I've never had the opportunity to use it at work.
I'm familiar with how Github pull requests work. What is the expected workflow in the Mercurial world for collaboration, assuming that pull requests don't exist (we don't use BitBucket or Gitlab)?
Say I have a public open-source repository, or a private work repository. How does someone propose non-trivial changes consisting of multiple commits? What options do we have for collaborating?
In the case of an open-source repository, I have to support both trusted and untrusted contribution requests. In the case of a work repository, I trust the contributor but the corporation mandate internal reviews for all contributions. Either way, it sounds as if we need a mechanism to notify authors of available contributions, some mechanism to discussion the contribution, and some mechanism to transfer changesets from the contributor to the official repository (given that the contributor might not have permission to push into the central repository but the central repository might have permission to pull from them).
Does code review software essentially handle everything I've mentioned above? Are pull requests really the only way to go?
What is the expected workflow without pull requests?
Push directly to the repository.
Does code review software essentially handle everything I've mentioned
above?
Yes, you should try Phabricator. See for yourself if they use pull-requests (cross repositories push).
Are pull requests really the only way to go?
No, but pull-requests fit well with most collaboration workflows. There are many ways. Pull requests favor pre-commit code-reviews, but you can also have a post-commit workflow.
If you are not using any code-review tool, only Mercurial and e-mails, you could do this:
For trusted contributors, you can set a fresh branch (and/or repository) to receive commits, do post-commit code-review. If they pass review, merge with the default branch.
For untrusted contributors, you can set a contributor repository to receive commits, do post-commit code-review. If they pass review, push it to the official repository, then merge with the default branch. Notify the review results using contributors e-mail (from the authors commit username).
If you find that only pull-requests fits your workflow or your tools, then the answer is: Yes.
Public forks
Ordinary and simple hg pull (each pull from fork will create at least anonymous branch, if not named)
MQ with MQ-Collab extension
Private forks
hg bundle + hg unbundle
hg export + hg import
Any media and any type of works
Exchange by shelves or queues
Related
I have three different Linux-based working places, each with a different computer. I need to have a repository synchronized to keep coding on the latest version each time I move from a workplace to another. You can always commit and push to, say, bitbucket and then pull from another computer, but this is not the purpose of a commit.
Other similar posts did not help, like Synchronizing a collection of Mercurial repositories.
Any suggestion?
Your two primary options for exchanging temporary work between repositories are Mercurial Queues and the evolve extension.
Mercurial Queues are documented fairly extensively here. To use them for your purpose, you have to put the patches under version control (explained near the bottom of the chapter) and can then push them to/pull them from a shared patch repository. Note that the book is a few years old and Mercurial has added some convenience features in the meantime. These days you can do operations on the patch repository directly via the --mq option (e.g., hg init --mq, hg commit --mq, hg push --mq) and don't need a bash alias for convenience.
Evolve is probably more intuitive; it provides a fairly straightforward approach to shared mutable history. You can commit changes in one repository, push the changes to a shared repository, pull from another and uncommit or alter them, then push them back.
In order to set this up, you need a shared repository somewhere that is declared as non-publishing. You do this by adding the following lines to its .hg/hgrc:
[phases]
publish = False
This prevents changesets exchanged through this repository from becoming public (at which point, they'd become immutable).
You will also need to install the extension first (unlike MQ, which is part of core Mercurial).
Note that Bitbucket currently does not support obsolescence markers, which are crucial for the functioning of changeset evolution, so you will need to host the shared repository in a different place. Evolve functions not by deleting outdated changesets, but by marking them as obsolete and hiding them (obsolescence markers also track how old and new changesets are related). Because Bitbucket does not support these markers, obsolete changesets will become visible again if pushed there. (Note that you can still use evolve locally or between evolve-aware repositories and use Bitbucket for public stuff.)
Slightly different ways:
Handwork
MQ with MQCollab extension
Commits with "classic" exchange between repos using MuliRepo extension (just don't forget hg pull on every workplace before pull - and add all remote repos into [multirepo] section on each workplace)
Automated way
Create additional "central hub" and use AutoSync extension
I am looking for suggestions on how to best organize this environment descried below. We are currently on Mercurial and I would prefer to stay there, however if a different version control system will help us achieve our goals, we will switch.
Short summary - Our company has been contracting another company to do work. I have been merging from our shared repository to our private repository where we have other modifications, and building a release product from there. Now we want to add 2 more contractors (who are entire companies, not just a guy) that can also contribute, however all contractors have private information that I can see, but needs to be private from the other contractors.
More details:
ContractorA started repository
I forked ContractorA to MyCompany
MyCompany now contains additions to ContractorA
The way I have been handling this so far is this:
ContractorA pushes to ContractorA
My Local machine has a clone of MyCompany
I have an alias to ContractorA on my local machine
I Pull from ContractorA, handle any merge conflicts, and then push to MyCompany
So the MyCompany repo has all changes from ContractorA and MyCompany
This has been working perfect for my needs, however now ContractorB is going to enter the picture
ContractorA has proprietary stuff that ContractorB cant have access to and ContractorB has proprietary stuff that Contractor A can’t have access to.
There is a common section that ContractorA will be contributing to and MyCompany and ContractorB will be using.
Everything needs to end up in MyCompany as we do 1 build that will include everything -
Common code that ContractorA modifies
Proprietary code from ContractorA
Proprietary code from ContractorB
Common Code from MyCompany
Any thoughts on how to handle this?
Since MyCompany is a clone of ContractorA, I cannot just give access to ContractorB to MyCompany Repo, correct? Or is there a way to restrict access on a directory basis to each user?
Is there a way to fork MyCompany into a new repository ContractorB and remove all ContractorA proprietary code such that ContractorB can never see any ContractorA proprietary code. If that is the case, could I put another alias in my local machine, pull from ContractorA, merge and push to MyCompany, then pull from ContractorB, merge and push to MyCompany?
Does this make any sense at all?
Thank you!
If you have a Mercurial repo, you will need to have full and complete access to the entire history of the working directory. However, you can hg clone a repo and only include certain branches in the clone and as long as you are careful about how those branches interact, you can achieve your goal. You do have a couple options and I've listed them in order of how I would rank them:
Each does their work on separate branches You will need four branches: core, branch-a, branch-b, composed-branch. All merges between the branches must go only one way: core -> branch-a -> composed-branch.
core: the common code base that is shared by all. All updates to this code need to be done at the tip of core since you will never be merging any other branches into it. If you make updates to the shared codebase, you then merge it into each contractor branch. If a contractor does anything on their own branch that should be put into core, manually make the updates to core via copy/paste or via a patch.
branch-a and branch-b: these hold the independent work done by the separate contractors. The public repo to which ContractorA has access (for pushes and pulls) will only have changes core and branch-a present since those are the only ancestors of the tip of their branch. Similarly for the ContractorB's repo.
composed-branch: this is where the magic happens. You pull from branch-a and branch-b and merge them into this branch as you receive updates from the contractors. You handle any merge conflicts here and can also do any additional changes to integrate the work done by the contractors. This is the branch that your build system pulls.
Break your current repo into several smaller repos If the risk of an incorrect push with one repo is too great or if the work each contractor is more like a dll or a library that can be built independently, it may make more sense to treat them as independent repos. Basically the previous suggestion, but independent repos instead of independant branches. You would again separate things into "used by all", "created by contractor" and "combined total". Also note that you may find hg's sub-repo useful, though it is considered a "feature of last resort".
Do it all via patches Contractors can send you updates as exported patches and you can apply and clean them up. This is a bad idea so I won't go into details unless pressed, but it is possible.
In all cases, you will likely need to start a new repo based on your current codebase that cleans the boundaries about what goes into core and can be seen by all.
Note: even if you hg delete files in the repo, they are still fully accessible in the history!
If every contractor must get another contractor work except "secret part", you have
Have MQ extension enabled on MyCompany repo
Move all secret parts of every contractor into separate MQ-patch (one patch per contractor)
(Clone|push to contractor's repo) with only his patch applied (hook can help with such condition-check)
Your integrator's work must be performed with both patches applied to MyCompany repo
I am investigating how to migrate our source control from SVN to Mercurial. One thing I am not sure how to deal with is usernames in commits. From what I've seen, there is no way to force an HG user to use a specific username, even if specified in Mercurial.ini, the user can override it in commits with the -u flag in hg commit.
How do companies deal with this? there is nothing to prevent developer A to commit something in his repository as developer B, and then pushing it to someone else.
Thanks.
I wouldn't say our company is large (4 developers), but it's never been an issue for us so far. I haven't seen any way to prevent that behavior either in my searching. I guess it comes down to an issue of trust amongst your developers.
Unrelated, we did successfully migrate from SVN to Mercurial about two years ago so I may be able to answer other questions you have.
EDIT: An idea:
I'm not sure how you were planning on setting up your topology, but we have a server that functions as the central repository for all our repos. It is possible to push changes between developers (bypassing the central server), but we never do that. We always commit locally and then push/pull from/to the central server. Additionally, we use https and windows authentication to authenticate with this central server.
If you're planning on having something like this, you could create a hook on the server (see repository events) (maybe the precommit event) that would verify that the user name in each commit being pushed is the same as the authenticated user from the web server.
Not sure if this would work, but it sounds plausable.
Another attempt(s)
Path-based ACLs in pseudo-CVCS workflow
If you'll use "controlled anarchy" workflow (p2p communications aren't controlled, resticted AND trusted and single authoritative source is common push-target), you can use "Branch Per Developer" paradigm. I.e - with ACL extension on central repo the following restrictions apply:
Nobody can push to default branch
Each developer can push only in his personal branch (under any name, name means nothing, auth-data for tracking is branch-name)
Only trusted mergers can work with repo-Central (merge dev-branches to default, NO rebase|NO history rewrite in dev-branches)
Each mergeset in default branch contain authentication piece - source branch
Signing branches
If you can't trust (and you must not trust) username in commits, you can trust strong crypto. Mercurial have at least two extensions, which allow digitally sign commits, thus providing accurate (so-so, see notes below) information about the authorship with own advantages and disadvantages in both cases
Commitsigs Extension Wiki and Signing Mercurial Changesets on Windows mini-HowTo are complete enough to understand and demonstrate all aspects of the start. Pro: no additional commits for signing, you can't (by design) sign old historic commits. Contra: not-so-nice output of needed commands (see screenshots in Damian's post for log and verifysigs), because it's GnuPG (no PKI), theoretically it's possible to create and use key-pair for any name-email and only "extra" comparison will show two different keys for one user
GPG extension and Approval Reports from wiki as quick-start. Pro: can use pgp-keys or openssl-certs (TBT!!!) (where openssl means one corporate source of issued certs), more readable and informative output of sigcheck command. Contra:
commiting changes to a .hgsigs file in the root of the working copy
and so it requires extra changesets to be made. This makes it
infeasible to sign all changesets. The .hgsigs file must also be
merged like any other file when branches are merged.
and at last file can be modified by hand by malicious user as any other file in WC
Edit and bugfixing
Openssl can be used in Commitsigs, not GPG extension
I would like to be able to review other developers code before they push it to central repository. The developers are at remote locations so going to their desk is not an option.
Currently they just push and if there are issues they would rollback. But this is not a good approach since someone can pull before they get a chance to rollback.
Mercurial is distributed, and as such should be able to adapt to any workflow. Try designating someone as an integration manager or use the dictator and lieutenants workflow.
How about review repository between the developers and the main repository? Only you push from there to main.
I upvoted kelloti's answer as this is just an expansion of it, but just used tiers of repositories. Have people push their un-reviewed changesets to a needs-review central-ish repository, and have reviewers push reviewed works from there to the needs-QA central-ish repository and have QA people push the release candidate central-ish repositories.
With a distributed version control system you can do a plurality of centralized repos as easily as you can do a plurality of developer repositories.
On my last project, we followed a very branchy development model - every task had a branch named with the task number. Code reviews were performed against the named branch. We explicitly wanted these pushed to the central repository and developers pulled them.
However, no task named branch was merged to the integration branch (in our case default, but it could have been any feature branch) until it had passed code review.
A lot of mercurial developers don't like to use short-lived branches that remain in the repository, but I find it makes it easier to follow the history, especially when looking at the history of a single change - you know that the changes for a particular task are on the associated named branch.
Perhaps using a shelve extension is a good solution? I'm not very familiar with Mercurial but this might work for you.
https://www.mercurial-scm.org/wiki/ShelveExtension
I'm in a small distributed team using Mercurial for a central repository. We each clone it via ssh onto our own linux boxes. Our intent is to review each others' work before pushing changes up to the central repository, to help keep central's tip clean. What is a good way to share code between developers on different linux boxes? I'm new to Mercurial. The options I can think of (through reading, not experience) are:
1: Author commits all local changes and updates working clone with central's tip. Author uses hg bundle, if there's a way to specify which local revs to include in the bundle. (an experiment showed me "bundle" only grabs uncommited changes, even if there are previous local commits that central doesn't know about) Author gets bundle file to reviewer. Reviewer creates a new clean clone from central's tip, and imports the bundle into that clone.
or,
2: After both author and reviewer fetch from central's tip, author uses patch and reviewer imports the patch.
or,
3: Author pushes to reviewer or reviewer pulls from author (but how, exactly? What I read is only about pushing and pulling to/from the original served repository, and/or on the same box instead of between different linux boxes.)
4: Forget reviewing the code prior to pushing to central; go ahead and push, using tags to identify what's been reviewed or not, and use Hudson (already working) to tag the latest safe build so team members can know which one to pull from.
If your team uses Mercurial and does code reviews, how do you do get the reviewer to see your changes?
Most of these are possible, some are more tedious than others.
You can use bundle by specifying the tip of the central repo as the --base:
hg bundle --base 4a3b2c1d review.bundle
Might as well just use bundle. That way, the changeset data is also included.
You can push (and pull) to (from) any repository that has a common ancestor(s). If you want to pull from one of your colleagues, they just need to run hg serve on their machine, and you will be able to pull.
This also works, but you will have to maintain multiple heads and be careful about merging. If you don't, it can become easy to base a stable change on top of an unreviewed changeset, which will make it hard to undo if you need to fix that unreviewed changeset later.
Of the options you presented, #1 and #3 are probably easiest, just depending on whether or not you can reach each other's boxes.
On a related note: This is the question that got my colleague and I started on developing Kiln, our (Fog Creek's) Mercurial hosting and code review tool. Our plan, and the initial prototype, would keep multiple repositories around, one "central" repository, and a bunch of "review" repositories. The review process would be started by cloning the central repo into a review repo on the server, and then running a full repo diff between the two, with a simple web interface for getting and viewing the diffs.
We've evolved that workflow quite a bit, but the general idea, having a branch repo to push unreviewed changes to and an interface to review them before you push them into the central repo, is still the same. I don't want to advertise here, but I do recommend giving it a try.
Half answer to this question is using ReviewBoard with Mercurial extention. It allows to push certain revisions for review by issuing the following command
hg postreview tip
I'll add a 5th option - do all development work on named branches, preferably one per task. Allow anything to be committed to a "development" named branch, whether it's in working state or not.
Push to the central repository, have reviewer pull the branch. Perform the review on the branch.
When the review has passed, merge the development work into the appropriate feature branch.
This workflow, which is (to me) surprisingly unpopular, has many advantages:
All work gets committed - you do not have to wait for a review to be done before committing.
You will not build off the wrong version. You only ever build from the feature branch.
In-progress work does not interfere with other developers.
From the development branch, you can either look at the latest changes (e.g. the changesets addressing review comments), compare with the branch point, or compare with the latest feature branch - all of which can give useful information.