I'm working on setting up a Hudson/Mercurial stack for development. One of the use cases I have is "As a developer, I want to update my local sandbox to a particular build number from Hudson, so I can [fix a bug, debug issues, create a branched version of code, etc.]."
So, if I see build #49 on Hudson, how do I update my local Mercurial repo to the same source code that was used to build #49?
Note: I have looked at Mercurial tags, however they don't seem quite appropriate. They require a commit, so it seems the commits will dirty up the history (each commit by a developer will show a subsequent commit from the tag operation). If this is the best there is, I guess I will have to live with it, but hoping for something better. Would probably still use tags for releases.
Ok, here's the solution I ended up with:
Using the Description Setter Plugin, I set both the success and failing build description to "Mercurial ${MERCURIAL_REVISION}". Turns out the current Mercurial SCM plugin sets this environment variable to the parent changeset id.
I can then look at a build on Hudson, and if so desired, grab the changeset id and do a "hg update " on my local repo to get that revision of code.
Note that in the Mercurial plugin issue tracker there is some talk of changing this to HG_REVISION instead and adding other environment variables, so this may break at some point in the future, but works for me for now.
You can use the keywords extension on the hudson system to update the nodeid into some aspect of the build, possibly including the artifact names. If the Hudson job output artifacts are like: myproject-2010-02-17-2dbf7575fa46.tar.gz you certainly know how to 'hg update' to that point in time.
The keywords extension and maybe a little ant-fu make that easy to do.
Related
How do I configure Hudson/Jenkins (we're still using Hudson) to build all branches with the mercurial plugin? I have seen that we can create separate jobs/etc for specific branches, but we branch often enough that it would be good to just build all branches.
The only way I can think of (and I haven't tried it) is to have our build script [executed by Hudson on the slave] somehow figure out which branch is the latest commit and then hg update -r itself, rather than the mercurial plugin doing it.
I would love to find out a more elegant/correct way to do this!
I've also been looking for this support in the Jenkins Mercurial plugin (partly because it's supported by its Git plugin) but the following Jenkins tickets seem to indicate that this is yet not implemented:
https://issues.jenkins-ci.org/browse/JENKINS-11102
https://issues.jenkins-ci.org/browse/JENKINS-10558
One option if you have branches that are used regularly (ie, named branches for milestones or developers) is to create copies of your master build and simply point it to that branch.
For example on my team's build server we have production ('default' branch) and development ('dev' branch). Our builds are run by feeding the branch name as a parameter to the build script (so it's a simple, one-field change from a copy of the main job).
Obviously this won't scale beyond a handful of branches, but it might be enough for your needs.
My question is essentially the same as here but applies to mercurial. I have a set of files that are under version control, and one save operation changes quite a lot of files. Some of the resulting changes are important for revision control, and some of the changes are just junk. I can "partition" off the junk into separate files. These junk files need to be part of a basic checkout in order for it to work, but their contents (and changes over time) aren't that important for revision control. Right now I just tell all our developers not to commit these files, but we all forget and it creates a lot of extra baggage in the repository. I don't really like the svn solution proposed because there are quite a lot of files and I want a simple clone to just work without all this extra manual work, so I was wondering if mercurial has a better alternative. It's kind of like hg shelve but not quite, and kind of like ignore, but not quite. Is there some hg extension that allows for this? Can git do it?
Mercurial doesn't support this. The correct way to do it is to commit thefile.sample and then have your developers (or better you deploy script) do a copy from thefile.sample to thefile if thefile doesn't exist. That way anyone can update the example file, but there's no risk of them committing their local changes (say their personal database connect string).
Aha! So TortoiseHG's repository and global settings have an Auto Exclude List where you can define a list of files that will be unchecked by default when the status, commit, and shelve dialogs open. So they still show up, but the user has to check them in order to actually do a commit. The setting is stored in hgrc, but it's under the [tortoisehg] heading so it's not supported by mercurial per se. Nevertheless, it fits my needs.
One solution to this is to use nested tree support (submodule in git), where the "junk" would be put in a different repository (to avoid cluttering the main repo), while enabling checking out the whole thing out in a consistent manner (right version of both repos in sync).
https://www.mercurial-scm.org/wiki/Subrepository?action=show&redirect=subrepos
In git, submodules are one solution to this issue - but they are not that great UI-wise. What I do instead is to keep two completely independent repositories, and using the subtree merge strategy when I need to update the main repo with the junk repo: http://progit.org/book/ch6-7.html
This is not quite a specifc question, and more me like for a criticism of my current approach.
I would like to include the program version number in the program I am developing. This is not a commercial product, but a research application so it is important to know which version generated the results.
My method works as follows:
There is a "pre-commit" hook in my .hg/hgrc file link to version_gen.sh
version_gen.sh consists solely of:
hg parent --template "r{rev}_{date|shortdate}" > version.num
In the makefile, the line version="%__VERSION__% in the main script is replaced with the content of the version.num file.
Are there better ways of doing this? The only real short coming I can see is that if you only commit a specfic file, version.num will be updated, but it won't be commited, and if I tried to add always committing that file, that would result in an infite loop (unless I created some temp file to indicate I was already in a commit, but that seems ugly...).
The problem
As you've identified, you've really created a Catch-22 situation here.
You can't really put meaningful information in the version.num file until the changes are committed and because you are storing version.num in the repository, you can't commit changes to the repository until you have populated the version.num file.
My solution
What I would suggest is:
Get rid of the "pre-commit" hook and hg forget the version.num file.
Add version.num to your .hgignore file.
Adjust version_gen.sh to consist of:
hg parent --template "r{node|short}_{date|shortdate}" > version.num
In the makefile, make sure version_gen.sh is run before version.num is used to set the version parameter.
My reasons
As #Ry4an suggests, getting the build system to insert revision information into the software at build time, using information from the Version Control System is a much better option. The only problem with this is if you try to compile the code from an hg archive of the repository, where the build system cannot extract the relevant information.
I would be inclined to discourage this however - in my own build system, the build failed if revision information couldn't be extracted.
Also, as #Kai Inkinen suggests, using the revision number is not portable. Rev 21 on one machine might be rev 22 on another. While this may not be a problem right now, it could be in the future, if you start colaborating with other people.
Finally, I explain my reasons for not liking the Keyword extension in a question of mine, which touches on similar issues to your own question:
I looked at Mercurials Keyword extension, since it seemed like the obvious solution. However the more I looked at it and read peoples opinions, the more that I came to the conclusion that it wasn't the right thing to do.
I also remember the problems that keyword substitution has caused me in projects at previous companies. ...
Also, I don't particularly want to have to enable Mercurial extensions to get the build to complete. I want the solution to be self contained, so that it isn't easy for the application to be accidentally compiled without the embedded version information just because an extension isn't enabled or the right helper software hasn't been installed.
Then in comments to an answer which suggested using the keyword extension anyway:
... I rejected using the keyword extension as it would be too easy to end up with the string "$Id$" being compiled into the executable. If keyword expansion was built into mercurial rather than an extension, and on by default, I might consider it, but as it stands it just wouldn't be reliable. – Mark Booth
A don't think that there can be a more reliable solution. What if someone accidentally damages .hg or builds not from a clone but from an archive? – Mr.Cat
#Mr.Cat - I don't think there can be a less reliable solution than the keywords extension. Anywhere you haven't explicitly enabled the extension (or someone has disabled it) then you get the literal string "$ID$" compiled into the object file without complaint. If mercurial or the repo is damaged (not sure which you meant) you need to fix that first anyway. As for hg archive, my original solution fails to compile if you try to build it from an archive! That is precisely what I want. I don't want any source to be compiled into our apps without it source being under revision control! – Mark Booth
What you are trying to do is called Keyword Expansion, which is not supported in Mercurial core.
You can integrate that expansion in make file, or (simpler) with the Keyword extension.
This extension allows the expansion of RCS/CVS-like and user defined keys in text files tracked by Mercurial.
Expansion takes place in the working directory or/and when creating a distribution using "hg archive"
That you use a pre-commit hook is what's concerning. You shouldn't be putting the rest of version_gen.sh into the source files thesemves, just into the build/release artifacts which you can do more accurately with an 'update' hook.
You don't want the Makefile to actually change in the repo with each commit, that just makes merges hell. You want to insert the version after checking out the files in advance of a build, which is is what an update hook does.
In distributed systems like Mercurial, the actual "version number" does not necessarily mean the same thing in every environment. Even if this is a single person project, and you are really careful with having only your central repo, you would still probably want to use the sha1-sum instead, since that is truly unique for the given repository state. The sha1 can be fetched through the template {node}
As a suggestion, I think that a better workflow would be to use tags instead, which btw are also local to your repository until you push them upstream. Don't write your number into a file, but instead tag your release code with a meaningful tag like
RELEASE_2
or
RELEASE_2010-04-01
or maybe script this and use the template to create the tag?
You can then add the tag to your non-versioned (in .hgignore) version.num file to be added into the build. This way you can give meaningful names to the releases and you tie the release to the unique identifier.
I'd like to evaluate Mercurial for my working projects. But most of my projects very heavily rely on the presence of svn:externals-like support. I've searched over StackOverflow and googled for corresponding support in Mercurial. All I found is subrepo feature added in Mercurial 1.3, but the page for this feature said:
subrepos are an experimental feature for Mercurial 1.3. So don't do this on mission critical repositories!
I don't want to use something unstable.
Can anybody shed some light on the real status of this feature, and the plans of polishing/finishing it and when it will be called "stable" and ready for mission critical repositories?
The word in the #mercurial IRC channel is that subrepos will continue to work as they do, and support will grow. For example currently the 'hg status' command isn't subrepo aware -- it works, it just doesn't recurse, but that in the future it will be. However, the current behaviors, fileformats (.hgsub and .hgsubstate) will only be changed in backward compatible ways.
So, go ahead and count on it now, and look forward to it getting better.
P.S. As of mercurial 1.4.2 the subrepos can now be subversion repos, so you can use a mercurial parent and a svn kid.
I've had good luck with the feature in my (light) usage of it so far. It's come in handy in two places:
Backing up a tree of unrelated repositories with a single hg pull command.
Tying a project together with specific versions of its dependencies, so that a single hg clone gets buildable source code. This is closer to the typical svn:externals usage.
Here are a couple of the limitations I've seen with it so far:
In case #1 above, you have to commit all subrepos at once. This is only occasionally annoying, as Mercurial (like any DVCS) encourages frequent commits—so most repos aren't left sitting around in an incomplete state to begin with.
Only the most basic Mercurial commands are subrepo-aware: clone, push / pull, update / commit, and perhaps a couple of others.
Extension authors are going to need time to test their extensions against repositories with subrepos.
When the Mercurial team describes the feature as "experimental," they don't mean that it's suddenly going to decide to erase all your data. They just mean that they haven't coded around all the edge cases like name conflicts (e.g., one developer adds a subrepo called README, while another developer adds a text file called README).
I have a project that is combining multiple hg repositories (different components) to build a single application. I'm looking for a cross-platform tool to support performing an operation on multiple repos at the same time (e.g. tag, pull, push, commit etc...) Essentially, I'm looking for the 'repo' script that Google wrote for Android, but for hg instead of git:
http://source.android.com/download/using-repo
I searched on stack overflow and found this:
mercurial windows batch file for pulling changes to multiple repositories
But it's still a bit manual and windows only. I know it's not that hard to write the script to either pass a command to the repos or try to encapsulate everything, but thought that it might be a common thing so maybe others already have a solution. I suppose one approach would be to port the repo script to hg (find and replace git with hg would probably get pretty far for simple operations).
What do other people do in this situation?
Definitely look at the new (in version 1.3) subrepositories feature in Mercurial. It lets you have an overarching repository that contains other repositories. The state of the top level includes a file that specifies the hash of the tip of the sub-repos, so you can effectively specify a single hash node id that encompasses the state of all the subordinate repos.
https://www.mercurial-scm.org/wiki/subrepos
I have been in a similar scenario for the past year, working daily with a project spread over 6 repositories, and being the one responsible for most branching/merging stuff etc.
At some point I found a simple bash-script for checking multiple SVN repo's. I adoptet it to Mercurial and have extended it over time. Right now I can't recall or google the original source of the script, so I unfortunately can't credit the original author (will credit in my script if I find the info later).
My "hgall" script is published on GitHub at https://github.com/JKrag/hgall
It is made for my own use, and thus directly references a few extensions that I have been using, but this can been easily edited out. (or just don't use the commands that call extensions you haven't installed).
The script is documented in the README, including my personal "tweak" of having it (by default) ignore repo's that start with underscore, as I use these as temporary clones to test messy operations (e.g. large merges).
I hope this script is useful to some of you - it has been of great help to me over the last year. Our company has just moved to Git, so I will probably not be making many updates in the near future, but might possibly be porting it to Git some day....
Additionally, Iain Lowe has an extension https://bitbucket.org/ilowe/multirepo/src which can manage multiple repositories.