So I'm pretty new to version control but I'm trying to use Mercurial on my Mac to keep a large Python data analysis program organized. I typically clone my main repository, tweak the clone's code a bit, and run the code on my data. If the changes were successful I commit and eventually push the changes back to my main repository. I guess that's a pretty typical workflow under version control.
My problem is that my code is run on the command-line, with several command-line arguments that refer to data files in the current working directory (and I have many such directories I need to test the code in, and they're outside of version control). So before using Mercurial I just kept my code in one ~/bin directory which was part of my PATH environment variable. Now, with version control, I need to either (1) after each edit, copy my current clone's executables to the ~/bin directory before running the code on the command line, or (2) each time I clone my code, add my current clone's path to the PATH, or (3) specify the entire/path/to/my/programs on the command line each time I run the code. None of these are very convenient, and I'm left feeling like there must be an elegant solution that I just don't know. Maybe something involving Mercurial's hooks? I want my under-revision code to be runnable on the command line between commits, so this seemed to rule out hooks, but I don't know... Many thanks for any suggestions!
Ry4an's answer is good if you want to continue with the multiple-clones workflow. But it's also worth being aware that Mercurial's powerful enough to allow you most of the benefits of that workflow without ever leaving your single "main" repo. I.e. you can create branches (named or anonymous) for experimental features, easily "hg update" to whatever version of the code you want to test, even use the mq extension to prune branches that didn't work out.
What I do in such a case is set up a two deep chain of symlinks to my binary in my current clone. For example I'll have:
/usr/bin/myappname
which is a symlink to
/home/me/repos/CURRENT/bin/myappname
where /home/me/repos/CURRENT is a symlink to whatever my current working clone is, for example:
/home/me/repos/myproject-expirment
After setting up the initial /usr/bin/myappname symlink all I have to do is update the CURRENT symlink when I create a new clone on which I'm working.
Related
Despite the decentralized nature of Mercurial, we have a centralized server that we all push to and that does nightly builds, packaging, etc...
Here's what we want to achieve: One of the files that is source controlled contains the major+minor version numbers which ideally would have to be increased with every commit. Since centralized numbering is not possible on developer's machines, we were thinking of a precommit script on the main server that would write a new minor version number to that file for each commit that is pushed. The question is/are:
since it's precommit, can this file change be part of the same commit?
if not, can precommit cause another commit and how do you prevent it from cascading/recursing?
how would one do that?
is there a better solution?
A "precommit" script is triggered only at commit time. By the time users are pushing to the "central" server they have already committed, and it's too late for a precommit hook to do anything at all. You can have changegroup and incoming hooks that are triggered to run on the "central" server when the developers push, but those can't modify the commits -- the commits are already committed/baked/done at that point, they can only react to them.
As a suggestion don't actually put the version string in the file -- having a file that changes with every commit just makes merging a pain. Instead do one or more of these:
have a CI server (like Jenkins) do builds on every push and use the Jenkins build number, which can be passed into your build script
use the Mercurial nodeid (hash) as part of your version string so you can always knew exactly what revision is in a build -- and don't put it in a file, just query for it in your build (or deploy) script
use a changegroup hook to automatically tag-on-push, which applies a pretty (possibly sequential) name to the commits (note, this pretty much doubles your number of commits since every tag is a commit)
Personally, I use something like this in my build script:
build.sh --version_string=$(hg log -r . --template '{latesttag}.{latesttagdistance}-{node|short}')
That gets me version strings that look like "1.0.3-5fd8ed67272e" which can be roughly read as "built from the changeset three commits since version 1.0 was tagged with nodeid 5fd8ed67272e", which is pretty darn good -- and it's never saved into a file it's either baked into the compile (for compiled languages) or written into a VERSION file when my deploy script uploads it to the server.
See this page in the Mercurial documentation for some comments and ideas about this issue. See also How to expand some version keywords in Mercurial? and the other SO answers referenced there.
My question is essentially the same as here but applies to mercurial. I have a set of files that are under version control, and one save operation changes quite a lot of files. Some of the resulting changes are important for revision control, and some of the changes are just junk. I can "partition" off the junk into separate files. These junk files need to be part of a basic checkout in order for it to work, but their contents (and changes over time) aren't that important for revision control. Right now I just tell all our developers not to commit these files, but we all forget and it creates a lot of extra baggage in the repository. I don't really like the svn solution proposed because there are quite a lot of files and I want a simple clone to just work without all this extra manual work, so I was wondering if mercurial has a better alternative. It's kind of like hg shelve but not quite, and kind of like ignore, but not quite. Is there some hg extension that allows for this? Can git do it?
Mercurial doesn't support this. The correct way to do it is to commit thefile.sample and then have your developers (or better you deploy script) do a copy from thefile.sample to thefile if thefile doesn't exist. That way anyone can update the example file, but there's no risk of them committing their local changes (say their personal database connect string).
Aha! So TortoiseHG's repository and global settings have an Auto Exclude List where you can define a list of files that will be unchecked by default when the status, commit, and shelve dialogs open. So they still show up, but the user has to check them in order to actually do a commit. The setting is stored in hgrc, but it's under the [tortoisehg] heading so it's not supported by mercurial per se. Nevertheless, it fits my needs.
One solution to this is to use nested tree support (submodule in git), where the "junk" would be put in a different repository (to avoid cluttering the main repo), while enabling checking out the whole thing out in a consistent manner (right version of both repos in sync).
https://www.mercurial-scm.org/wiki/Subrepository?action=show&redirect=subrepos
In git, submodules are one solution to this issue - but they are not that great UI-wise. What I do instead is to keep two completely independent repositories, and using the subtree merge strategy when I need to update the main repo with the junk repo: http://progit.org/book/ch6-7.html
We recently moved from Perforce to Mercurial and love it!
One little problem: after much research we can't figure out how to map a special directory in the repository to some special place on the client. Here is an example of our hg repo:
/foo/source files
/bar/source files
/build
/macosx/mac make files
/win/windows make files
With Perforce, we were using client spec mappings to map //depot/build/macosx/... to just /build/... on the Mac client, and //depot/build/win/... to /build/... on the Windows dev box. Directories foo and bar are synced as is. Makefiles in /foo and /bar assume that our build makefiles are located in /build and we would like to keep them as is. The final client set of files should look like this:
/foo/source files
/bar/source files
/build/client specific make files
I've read about subrepos, but this solution does not seem to be client specific.
Any idea how to solve this problem will be very much appreciated!
You can't check out only portions of a repository with Mercurial.
You always get a clone containing everything, and the working directory will also contain everything.
With Mercurial you should strive to have 1 repository for 1 project, so that everything you get logically belongs together, and then you shouldn't have much need for just a portion of it.
This also means that whatever directory structure you have in your Mercurial repository will always match exactly the structure you have on disk.
You can't do this with Mercurial as it doesn't have the concept of a client separate from a depot.
However, you can use a symlink on Mac OS X (ln -s) and a junction on windows (mklink on Vista and up using the junction tool on XP http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx) to solve this problem on the file system level.
Alternatively you can use a variable in the Makefiles to refer to the build directory (eg $(BUILD)/something.ext instead of build/something.ext).
This sort of mapping cannot be done in Mercurial. There is an outstanding TODO item for 'narrow' clones so you can check out just a subdirectory. And I could see an implementation of that supporting that sort of functionality. But then again, I know that something like this would be considered a little too 'clever' (read complex) and there would be a lot of push-back on the idea.
In the meantime, I would suggest one of these two solutions.
Symbolic links. Put the symbolic link to your build directory in your .hgignore file. Then each person can make their own symbolic link to the appropriate directory of build files. This has the disadvantage of not working on a platform without symbolic links.
An environment variable that's used in a top level makefile to construct the path to the platform specific makefile it should be calling.
I'm trying to sell our group on using Mercurial as a source repository rather than VSS. In the process of updating our build scripts, I'm running into an issue trying to retrieve files from the Hg repository.
Our builds are automated with NAnt and currently work for local builds or builds from VSS (ie, pull the source as needed from VSS). I'm trying to update them to work with Mercurial as well.
Basically, when I'm working with single files, I don't have any issues since I can just use NAnt's 'get' task (after getting the appropriate revision hash) to retrieve the individual file.
The problem that I'm having is when I need to work with a directory (and subdirectories) of files that aren't at the root of the repository. I can't seem to figure out the proper commands to retrieve/copy a subdirectory from the repository to my 'working' directory for the builds. I've spent basically the whole afternoon trying to figure out how to do this with the mercurial executables (so I can use a NAnt 'exec' task), and have basically hit a wall so I figured I'd try posting here.
Can someone confirm whether this is possible, and provide some suggestions as to how I might be able to do this? I realize that Mercurial tracks changes by files and not directories, but it seems odd to me that this isn't available out of the box (from what I can tell).
If it's just not possible, the only workarounds I see are either maintaining NAnt fileset lists of expected files to work with (ugh!), or cloning the entire repository to a temporary directory and then just copying the files from that source as needed (this feels like a cludge to me).
I realize that I could simply create another repository for the directory that I want to work with, but I'd prefer to not go that route since I think that would increase the complexity of what I'm trying to do by a significant amount (I would have to apply this a large number of times for all of the different libraries that we build..).
Mercurial doesn't let you get only part of a repository. You have to get the whole tree. It's much more whole-repo focused than svn is.
You could try and segment your repository into multiple repos and manage them using the subrepos feature. Then you can pull the subdirectories independently.
This is not quite a specifc question, and more me like for a criticism of my current approach.
I would like to include the program version number in the program I am developing. This is not a commercial product, but a research application so it is important to know which version generated the results.
My method works as follows:
There is a "pre-commit" hook in my .hg/hgrc file link to version_gen.sh
version_gen.sh consists solely of:
hg parent --template "r{rev}_{date|shortdate}" > version.num
In the makefile, the line version="%__VERSION__% in the main script is replaced with the content of the version.num file.
Are there better ways of doing this? The only real short coming I can see is that if you only commit a specfic file, version.num will be updated, but it won't be commited, and if I tried to add always committing that file, that would result in an infite loop (unless I created some temp file to indicate I was already in a commit, but that seems ugly...).
The problem
As you've identified, you've really created a Catch-22 situation here.
You can't really put meaningful information in the version.num file until the changes are committed and because you are storing version.num in the repository, you can't commit changes to the repository until you have populated the version.num file.
My solution
What I would suggest is:
Get rid of the "pre-commit" hook and hg forget the version.num file.
Add version.num to your .hgignore file.
Adjust version_gen.sh to consist of:
hg parent --template "r{node|short}_{date|shortdate}" > version.num
In the makefile, make sure version_gen.sh is run before version.num is used to set the version parameter.
My reasons
As #Ry4an suggests, getting the build system to insert revision information into the software at build time, using information from the Version Control System is a much better option. The only problem with this is if you try to compile the code from an hg archive of the repository, where the build system cannot extract the relevant information.
I would be inclined to discourage this however - in my own build system, the build failed if revision information couldn't be extracted.
Also, as #Kai Inkinen suggests, using the revision number is not portable. Rev 21 on one machine might be rev 22 on another. While this may not be a problem right now, it could be in the future, if you start colaborating with other people.
Finally, I explain my reasons for not liking the Keyword extension in a question of mine, which touches on similar issues to your own question:
I looked at Mercurials Keyword extension, since it seemed like the obvious solution. However the more I looked at it and read peoples opinions, the more that I came to the conclusion that it wasn't the right thing to do.
I also remember the problems that keyword substitution has caused me in projects at previous companies. ...
Also, I don't particularly want to have to enable Mercurial extensions to get the build to complete. I want the solution to be self contained, so that it isn't easy for the application to be accidentally compiled without the embedded version information just because an extension isn't enabled or the right helper software hasn't been installed.
Then in comments to an answer which suggested using the keyword extension anyway:
... I rejected using the keyword extension as it would be too easy to end up with the string "$Id$" being compiled into the executable. If keyword expansion was built into mercurial rather than an extension, and on by default, I might consider it, but as it stands it just wouldn't be reliable. – Mark Booth
A don't think that there can be a more reliable solution. What if someone accidentally damages .hg or builds not from a clone but from an archive? – Mr.Cat
#Mr.Cat - I don't think there can be a less reliable solution than the keywords extension. Anywhere you haven't explicitly enabled the extension (or someone has disabled it) then you get the literal string "$ID$" compiled into the object file without complaint. If mercurial or the repo is damaged (not sure which you meant) you need to fix that first anyway. As for hg archive, my original solution fails to compile if you try to build it from an archive! That is precisely what I want. I don't want any source to be compiled into our apps without it source being under revision control! – Mark Booth
What you are trying to do is called Keyword Expansion, which is not supported in Mercurial core.
You can integrate that expansion in make file, or (simpler) with the Keyword extension.
This extension allows the expansion of RCS/CVS-like and user defined keys in text files tracked by Mercurial.
Expansion takes place in the working directory or/and when creating a distribution using "hg archive"
That you use a pre-commit hook is what's concerning. You shouldn't be putting the rest of version_gen.sh into the source files thesemves, just into the build/release artifacts which you can do more accurately with an 'update' hook.
You don't want the Makefile to actually change in the repo with each commit, that just makes merges hell. You want to insert the version after checking out the files in advance of a build, which is is what an update hook does.
In distributed systems like Mercurial, the actual "version number" does not necessarily mean the same thing in every environment. Even if this is a single person project, and you are really careful with having only your central repo, you would still probably want to use the sha1-sum instead, since that is truly unique for the given repository state. The sha1 can be fetched through the template {node}
As a suggestion, I think that a better workflow would be to use tags instead, which btw are also local to your repository until you push them upstream. Don't write your number into a file, but instead tag your release code with a meaningful tag like
RELEASE_2
or
RELEASE_2010-04-01
or maybe script this and use the template to create the tag?
You can then add the tag to your non-versioned (in .hgignore) version.num file to be added into the build. This way you can give meaningful names to the releases and you tie the release to the unique identifier.