I've got a Mercurial repo set up like this, with a subrepo inside another subrepo:
Root
.hg
.hgsub
.hgsubstate
Nested
.hg
.hgsub
.hgsubstate
foo.txt
FurtherNested
.hg
bar.txt
If I change foo.txt and bar.txt and commit from inside Root then all is well and all the nested subrepos commit. However if I only change bar.txt and commit from Root then Hg thinks nothing is changed. I have to commit from inside Nested to get the FurtherNested changes to commit when there are only changes in FurtherNested. It seems that in order for nested subrepos to work, each nested level has to contain changes in order for the recursion to work.
Nothing I read in the Mercurial docs on subrepos seemed to imply that subrepo commits would only propagate if there were changes. In fact it says the opposite:
When we commit, Mercurial will attempt
to recursively commit in all defined
subrepos...
So my question is, is this to be expected or is something broken or just not done yet (Mercurial 1.5.4 on Windows)?
Looks to me like this is just something that's broken. The relevant source code (in subrepo.py) doesn't seem to recurse into subsubrepos, viz:
def dirty(self):
r = self._state[1]
if r == '':
return True
w = self._repo[None]
if w.p1() != self._repo[r]: # version checked out change
return True
return w.dirty() # working directory changed
Probably wouldn't be too hard to fix, though I don't know the code intimately enough yet. It would probably be worthwhile posting on the Mercurial mailing list and/or filing a bug. I doubt Matt and Benoit are reading this, but they definitely do read everything posted there.
Related
Is it possible to create a subrepository using a sibling path?
Subversion is our "chosen" VCS here, but I've already had quite a few issues with out of date commits. It's much more convenient for me to dual version my files under Hg and SVN, and I have had great success with it. However, I've got a few other co-workers using Hg, and we've had no problems there except for one they probably haven't noticed.
Our SVN layout looks like this
Area/
trunk/
Program1/
Program2/
...
Services/
Program1ServiceA/
Program1ServiceB/
branches/
Program/
Program/
...
Services/
Program1ServiceA/
Program1ServiceB/
tags/
Program1/
Program2/
...
Services/
Program1ServiceA/
Program1ServiceB/
Which makes it kind of stupid when you're working on a project, because if your main project is comprised of Program1 and Program2, and a few more services... I can't get all of the changesets in one go, because we've got a repository that matches the directories. So I have to make sure 4 or 5 repos are in sync, especially with some service references.
I've had some good luck using subrepos beneath a single directory:
MainRepo/
Subrepo1/
Subrepo2/
But what I would like to do is specify a relative path so I can use sibling directories to the subrepository, so I could have something like this:
Area/
Project1/
Program1/ (points to ../trunk/Program1)
Program2/ (points to ../trunk/Program2)
Service1/ (... You get the idea)
Service2/
trunk/
Program1/
Program2/
Services/
Service1/
Service2/
But so far it hasn't worked like I expected it to. trunk/Program1 is an Hg repo, and my Project1/.hgsub file contains
Program1 = ../trunk/Program1
I've also tried ../../trunk/Program1
But the result of either of those is that a new directory is created: Area/Program1/Project1 that is empty.
So far, the only search results I've been able to find use http based repositories for subrepos, so I'm not sure where to go from here. Our dev env is Windows 7, so the "easy" answer is to create junctions, but my primary concern is to make things like this easy to do, so the barrier to entry is as low as possible, and even something as easy as mklink /J Program1 ..\trunk\Program1 from an administrator cmd window is one more thing that would prevent people from migrating to a better workflow.
Is it possible to add a subrepository like I want, or is there a better way to do what we're doing?
Subrepositories are always inside another repository. In other words, subrepositories lets you version a collection of repositories where some repositories are nested inside other repositories. Subrepositories can thus not be siblings without creating an outer repository.
The relative paths you're talking about are used when Mercurial needs to figure out where to get a new subrepository from. That is, when you run hg update (or when it's run for you as part of hg clone) and Mercurial notices a .hgsub file, then it needs to create the subrepositories mentioned there. To create the subrepo, Mercurial uses the path on right-hand side:
sub-A = relative/path
sub-B = C:/absolute/path
Here sub-A will be checked out in the root of your working copy using the command
hg clone <default path for main repo>/relative/path sub-A
and sub-B is checked out using the command
hg clone C:/absolute/path sub-B
That's all — it's a very simple mechanism. I've tried to describe this in my subrepository guide and it's also explained in the wiki.
For your case, you can make a thin shell repository for the parts that belong together. This repo will be like Project1 above and have Program1, Program2, Service1, etc as subrepos. The .hgsub will look like this:
Program1 = Program1
Program2 = Program2
Service1 = Service1
Service2 = Service2
By using "trivial subrepo paths" you make things easy: a clone looks just like the clone source and everything is kept together.
A final note: unless you use Program1 or Service1 in other projects, then you should just put everything into a single repository.
There is public repo on mercurial (not mine). It contains two subrepos. I've made a fork for both main repo nad two subrepos. I want to create pull requests from time to time, but I can't make subrepos work properly.
When I clone my forked repo, it will be downloaded, but two subrepos will remain original, not my forked one. If I'll replace them with forked, I won't be able to make a pull request cos I don't want to switch original repo to my subrepos. But I want to change both repo and subrepo.
Once I've found a solution. I've modified hgrc file in .hg folder of subrepo.
It was:
[paths]
default = ssh://hg#bitbucket.org/originator/subrepo
It become:
[paths]
Me = ssh://hg#bitbucket.org/Me/subrepo
Original = ssh://hg#bitbucket.org/originator/subrepo
It was working exactly as it should. I've changed code, pushed to my subrepo, created pull request for subrepo. Then I've pushed changes in my main repo and it simply worked. .hgsubstate is now reffering to commit that exist on forked repo, but not on original one. When orignator will megre both pull requests, everything would work.
Is I said, it worked before, but not now. For some weird reason I can't just push .hgsubstate that point to new subrepo revision that exists on forked subrepo. I simply can't include it in commit. I click "commit", but in result commit it's not included. And I don't know how to force is being included - HG too "smart" about it. Also TortoiseHG marks my subrepo as dirty. But if I'll try to add both .hgsubstate and my dirty repo, it for some reason tries to push changes to originator's repo. It seems like my HG just can't commit .hgsubstate without pushing changes to subrepo on itself.
Any way to solve this? Everything I want is to commit and push .hgsubstate without doing anything else.
Did you adjust your .hgsub accordingly so that it works for both, your local paths as well as the remote paths to which you push to the server?
It's advisable to add to your .hgsub sections which describe the paths in an absolute sense. I used to struggle with that when I had the subrepos as the same level in the dir structure on the server but wanted them as sub-dirs on my own computer. It helped me to add to .hgsub a subpaths section:
[subpaths]
ssh://user#host.org/path/subrepo1 = ssh://user#host.org/subrepo1
ssh://user#host.org/path/subrepo2 = ssh://user#host.org/subrepo2
/home/user/project/subrepo1 = /home/user/subrepo1
/home/user/project/subrepo2 = /home/user/subrepo2
project/subrepo1 = subrepo1
project/subrepo2 = subrepo2
I'm offline a lot.
So normally, I use one local clone as a "hub" for features, bugs, etc.
hg clone local-hub bug-123
Works offline. Cool.
Can I use a similar workflow if that project contains remote subrepositories?
Because, if .hgsub says
sub/shared = http://server/hg/shared
hg clone says
abort: error: getaddrinfo failed
Note that once the clone is created (while connected), push and pull will use the path in the subrepo's hgrc (instead of the location in .hgsub). So I can point this to a local clone and everything is cool.
But clone looks at .hgsub (as it's supposed to). So if the "blessed" subrepo is on a server, I can't create new clones offline, even though the files I need are right there.
This is a problem, right?
Ideally whomever set up the project uses relative URLs in their .hgsub file like this:
sub/shared = ../shared
and then, of course, actually makes shared a sibling of the main repo. Then as long as you have cloned down the main repo and the subs (as siblings) then everything will work out.
If they've used absolute URLs in their .hgsub file you can work around it using the subpaths section in your .hgrc like this:
[subpaths]
http://server/hg/shared = ../shared
which provides a translation layer in your client.
The canonical way to use subrepositories is to have X = X paths in the .hgsub file:
sub/shared = sub/shared
That way a clone will structurally look just like the original -- and so you can use the clone to make further (local!) clones.
However, this is not always possible, for example, Bitbucket wont let you create the nested repositories on their server. In that case, the ../X style paths in the .hgsub file is better, and you can use the subpaths configuration section to translate these paths into paths you can use locally.
I have a project, which I have a bitbucket repository for, and it is dependent on another project that I incorporate as a subrepo. Now, I don't have push access to the subrepository, nor do I want or need to--it's a pull-only relationship.
I realize that when you push the main repository, it will try to push the subrepositories, as well. Since I cannot do that, I pulled a local copy of the dependent project, at the same level as the main repository's directory. In essence, I have the following layout:
Main/ ; pushes to https://mine.org/Main
.hg/
.hgsub
Lib/
SubRepo/ ; clone of Main/../SubRepo/
.hg/
SubRepo/ ; local copy of https://forbidden.org/SubRepo
.hg/
The content of .hgsub is something like,
Lib/SubRepo = ../SubRepo
Then I cloned,
~/path/to/Main $ hg clone ../SubRepo/ Lib/SubRepo
So far, so good. The problem is, after I set this all up and committed the changes, when I try to push Main Mercurial will try to push SubRepo to https://mine.org/SubRepo, which does not exist, thereby failing the whole push operation.
Is there something I'm missing?
Why not just create a https://mine.org/SubRepo -- if you don't want to advertise it you can always turn on hide for it in the [web] section in its .hg/hgrc file. This is the pattern I'm used to, where you clone down the main repo and all the subrepos in the same layout at each place you'll use them: both your development box and your web-facing hgweb install.
Alternately, you could use a [subpaths] section in Main/.hg/hgrc with something like this in it:
[subpaths]
https://mine.org/SubRepo = https://forbidden.org/SubRepo
which should let you intercept the derrived target for the push and point it at a place that which it won't let you push, will let you see nothing has changed so push can continue.
It seems like what Mercurial is doing is legitimate: using the paths listed in your .hgsub it's attempting to push to a directory called 'SubRepo' that exists one level up from Main. This is obviously not what you want, so you'll probably have to work some magic here. I can think of two options:
If you can support this, place the local copy of forbidden.org's repository at C:/Forbidden/Subrepo or something like that, and use this absolute path in your .hgsub. Mercurial will be able to push to this and it should work.
There's no problem including the actual forbidden.org url as your subrepo address if you don't make any modifications to this repo. If there are no changes to the subrepo, your push should succeed. Of course, this is a fairly manual option and on a larger team it would be impossible to enforce. If you did accidentally commit some modification to the subrepo, you'd have to go through and use histedit or MQueues to pull it out, and that can be tricky with subrepos.
As the definite guide aptly points out (search for "Tags and cloning"):
When you run hg clone -r foo to clone a repository as of tag foo, the new
clone will not contain any revision newer than the one the tag refers to,
including the revision where the tag was created. The result is that you'll
get exactly the right subset of the project's history in the new
repository, but not the tag you might have expected.
It means hg tags in your new clone does NOT show the foo tag. Same thing happens if you had cloned before foo tag was added, and you do hg pull -r foo.
(Digression: tag is about the only thing I don't quite get in hg. I understand there are advantages (e.g. merge) in putting it in a changeset, but it always feels weird to have meta data mixed with source code.)
It should be obvious that I'm asking for an automated way, instead of pulling the tag changeset as a separate manual step.
I know I could check for this scenario in an incoming hook (so it works for both clone and pull), or wrap clone and pull.
But is there a better/easier way?
UPDATE hg bug tracker already has this issue.
You want a giant hack with bash and an embedded Perl script? Well, here it is...
#!/bin/bash
if [[ "$1" == "" || "$2" == "" || "$3" == "" ]]; then
echo 'hgclonetag <src> <tgt> <tag>'
exit 1;
fi
REV=`hg log -R $1 --rev $3: --limit=2 | perl -F: -lane 'if (/:([\dA-Fa-f]+)$/) {print $F[2] if defined($flag);$flag=1;}'`
hg clone --rev $REV $1 $2
This invokes the hg log command to extract the revision number after the first tag-related revision and then clones to this revision.
Currently this does not work on remote repos: -R switch only works on local repos unfortunately.
The more I think about it the more I'm convinced the right answer is to just clone everything and update to the tag, which can be done in a single step:
hg clone http://host/path#tagname
That gets you everything and then does hg update to tagname which sets your working directory to the correct revision. Given delta compression that's not necessarily much larger, and if it is you can automate cloning the bulk of it from a previous local clone.
There is a postclone hook. It's called post-clone (the hgrc manpage shows a post-ANYCOMMAND and pre-ANYCOMMAND exist) though as you pointed out you could also use *changegroup or update hooks too, since clone uses both of those functions (unless you suppress update with -U).
What about just adding a --localtag so you have the name but not the extra changeset if you need it for reference only. Something like
hg clone -r tagname URL
hg tag --local tagname
which you could easily build into a shell alias.
Other than that there's not necessarily guaranteed to be a way to have revision X and the revision where revision X is tagged without also having other revisions you don't want since the tag could have been applied after other work was done. You can, of course, always update to 'X' and to have subsequent changesets in you working dir, but they'll still be in your repo.
Honestly, once I figured out that the tag name doesn't come a long when you clone up to a tag, which I admit confused the heck out of me at first, I didn't find any need to bring along the changeset with the tag in it.
Yes it can be done by post-clone/pull hooks, but there are a couple of crooks.
First, it only works for local repo, since you can't get the list of tags in a remote repo.
Second, dealing with clone/pull arguments and options is not trivial. (For clone I need to get the target repo, -r, -u, -U. For pull I need -r and -u.) I tried to use fancyopts, but it can't deal with global options, which are processed away in dispatch. I managed to hack dispatch to give me only the args and opts of a command, but it feels and looks ugly.
Using command wrapper would eliminate the second problem.
I hope one day hg will add an option to clone and pull to do it cleanly.