I'm facing problems fetch HG/Mercurial repo in Bitbake recipe.
can't fetch the source URL reported in logs
tested with env hg clone repo_url works fine
testet with hg:// and http:// headers
The right format of SRC_URI is:
SRCREV = "tip"
SRCMODULE = "myapp"
SRC_URI = "hg://hg_server_url;rev=${SRCREV};protocol=http;branch=${SRCBRANCH};module=${SRCMODULE}"
The key is that hg_server_url shouldn't include target repo subname, but include it as SRCMODULE !
Also you could pull latest commit not with AUTOREV as per git repo but SRCREV = "tip"
Edit: Working 100% tip
Due to logic of bitbake classes and nature of Mercurial repo above statement should work but just once. It's that way cause when bitbake checks for cached files it finds "tip" already in cache instead git's HEAD wich is translatet to uniq hash.
To avoid that behaviour we should avoid completly (Mercurial repo) cache when fetch hg repo. This is simple as put in recipe:
do_install[nostamp] = "1"
That way bitbake always think the recipe is executed for the first time. We not useing cache but always get latest repo commit.
Related
We are looking for a way to add / update a custom tag at the beginning of each file being committed during a commit. Its some kind of local timestamp we need.
I was thinking of hooks.
Unfortunately I cannot find a useful hook for that:
precommit: unsuitable as it fires before hg knows any metadata of the commit
pretxncommit: unsuitable, as the documentation clearly states that we should not change the working dir at this point
commit: unsuitable, as it fires when the commit has already happened.
EDIT:
I can not use hg's inline changeset-hash and / or datetime. For the following reason:
Our files get later imported into an external system (we do not have control over) which does not support any kind of versioning.
To simplify stuff: let's say tag is an ever-incrementing no. (everytime we commit). This tag is then used to help getting an idea of the version / status of the file on the system in respect to the file in the repo - like "no. of changes we're missing" and such.
Any ideas?
I would suggest a two-stage solution. First, create an alias along the following lines:
[alias]
tcommit = !tag-changed-files && $HG commit "$#"
Here, tag-changed-files would retrieve a list of modified and added/moved files via $HG status -ma -n or $HG status -ma -n -0 and tag them. I am assuming that re-tagging files that have been modified but aren't being committed yet is a harmless operation; more on that below. Note that you can even redefine commit if you absolutely want to via:
[alias]
commit = !tag-changed-files && $HG --config alias.commit=commit commit "$#"
However, this is potentially problematic, because it may confuse scripts.
You could also integrate the commit step in the program if you wanted to, and even try and parse the command line arguments to only tag those files that you are committing. For this approach, using hglib might be appropriate to avoid the overhead of invoking Mercurial multiple times. (Note that hglib and other tools that use the command server ignore aliases and command defaults, so this works even if you alias commit).
Second, you'd install a pretxncommit hook that verifies that files that are being committed have indeed been tagged appropriately (to ensure that the tag-changed-files program hasn't been bypassed by accident).
This should work without a problem on full commits; for partial commits, any files that were changed but have not been committed would also have been retagged, but since they will be either committed later (and get tagged properly at that point) or reverted, that should be harmless.
an idea of the version / status of the file on the system in respect to the file in the repo
Just one idea
Stop reinvent the wheel
Incremental counter is just shit, if you task is "to know, which version is on LIVE and which - in Mercurial's tip" (and this is your real business-task, yes?!)
Keyword Extension give you last changes per file.
If you want to inject changeset of repository into files (it's reasonable good way), re-read this part of wiki-page
If you just want to version your entire repo, do not use this
extension but let your build system take care of it. Something along
the lines of
hg -q id > version
before distribution might be well enough if file-wise keyword
expansion in the source is not absolutely required
You can insert hg id output into files at export stage (in planetmaker's sed-style), bu you can also have this additional metadata in files permanently in VCS with special encode|decode filters
There is - to my knowledge - no intrinsic system in mercurial which supports what you describe. However I can recommend an approach which somewhat is adopted from building a software release package from the repository: Make a small export script which replaces a certain KEYWORD in your files with the version information you need. A Makefile target could look like which creates a zip export for revision XXX with version information in all files which support it (thus contain the verbatim KEYWORD - use something truely unique here):
VERSION=$(hg log -rXXX --template="Version: {node|short} from {date|isodate}")
export:
hg archive -rXXX -t files export
for i in $(hg ma -rXXX); do sed -i "s/KEYWORD/$VERSION/g" $i; done
zip -9rq export.zip export
I use a similar approach in my Makefiles where I create versioned export for the source of a particular revision of my software.
EDIT: if your purpose (as stated by the comment) is only to implant the number of commits made to that file: then you can use indeed a pre-commit hook and modify the file. You can count the number of modifications made to a file with hg log FILENAME --template="{node}\n" | wc -l. Do that for every file and do the sed replacement in the header of each file in the pre-commit hook.
I have used mercurial's bfiles extension for some time and it works fine. The only problems are installation and the special "hg bfadd" command.
Now that Mercurial 2.0 include the largefile extension I would like to switch.
Can't find any tools or guides on how to do it? Anyone tried it yet.
I have several repositories that all use the same store and have the following mercurial.ini.
[bfiles]
store=\\Someserver\Mercurial\bFilesStore
autostatus = true
autoupdate = true
autorefresh = true
autoput = *
You can find the documentation here : https://www.mercurial-scm.org/wiki/LargefilesExtension
To enable the extension add the following to your hgrc :
[extensions]
largefiles =
You can add a new large file with :
hg add --large thisfileislarge
About the migration, the readme.txt of the bfiles extension says something about a migrate.txt file ( https://bitbucket.org/gward/hg-bfiles/overview section "The future"). But I can't find the file anywhere on the repository, maybe he forgot to upload it.
There's also a mail on mercurial-devel about this : https://www.mercurial-scm.org/pipermail/mercurial-devel/2011-October/035161.html but nothing since then.
Maybe the better solution is to contact the author of bfiles about his status on the migration process and keep using the old extension until you have an answer ?
Either way, there's a lot of bug report about largefiles since the release of 2.0, so it's maybe a good idea to wait anyway :)
In the latest hg-bfiles version (from 2011-12-05), if you update to the migrate branch, you get this help file:
bfiles: migrating to largefiles
If you want to migrate from bfiles to the new largefiles extension in
Mercurial 2.0, you first have to decide: convert your repo or keep it?
Converting your repo
This is appropriate if:
you have a small repository
you know exactly where every clone of it is
you can replace every clone
It involves creating an new repo with .hgbfiles/ replaced by .hglf/.
This means that your changeset IDs will differ, so you cannot
pull/push between the old and new repos. You must replace every
existing clone with a clone of the converted repo.
The advantage of converting your repo is that you can say goodbye
forever to bfiles, and move into the future using largefiles alone.
The process is mostly automated by two shell scripts: convert-repo and
convert-store.
Use the convert-repo shell script to convert the repository itself. This is just a wrapper around "hg convert" that takes care
of all the fussy details needed to turn .hgbfiles/ into .hglf/.
It's easy to run:
./convert-repo SRC-REPO DST-REPO
The resulting DST-REPO is not yet ready to use: you still have
to convert the bfiles store to a largefiles store.
Use convert-store to turn the bfiles store into a largefiles store. You must have a local copy of the bfiles store -- so you
probably want to run this on the server where your bfiles store
lives. Again, it's easy:
./convert-store SRC-STORE DST-REPO/.hg/largefiles
Putting the store inside your DST-REPO is the easiest way to
make largefiles just work.
Keeping your repo
(Yes, that's the end of the file, there's no help on how to keep your repository)
I have a need for a hook to run after update (this will build the solution they have updated) and I don't want to have to add that hook manually for each person that clones my central repository.
When someone first clones my central repository, is it possible to include hooks into that clone? It seems that the .hgrc file doesn't get cloned automatically.
I did read about site-wide hooks, but as far as I understand it, they work on each created repository, where I only want to have the hooks on some repos.
As Rudi already said, this is (thankfully) not possible for security reasons.
However, you can reduce the per-clone workload to set up hooks manually: Ship the hook scripts as part of your repository, e.g. in a directory .hghooks, and additionally include a script in your repo which sets up these hooks in a clone's hgrc. Each coworker now only needs to call the setup script once per clone.
This is not possible, since that hooks do not propagate to clones is a security measure. If this were possible, one could set up a rouge repository, which runs arbitrary commands on any machine where the repo is cloned.
See http://hgbook.red-bean.com/read/handling-repository-events-with-hooks.html#id402330 for more details.
This will allow for centralised per-repo hooks, with a single setup step per user. It will however cause problems for users who are disconnected from the network. An alternative if you tend to have disconnected developers (or ones over high-latency/low bandwidth links) would be to have a repo containing the hooks, and set up each user's global hgrc to point into that repo (and require regular pulls from a central hook repo).
Note that I treat the ID of the first commit as the "repo ID" - this assumes that the first commit in each repository is unique in some way - contents or commit message. If this is not the case you could do the same thing but applying it over the first N commits - but you would then have to account for repos that have fewer than N commits - can't just take repo[:5] for example as newer commits would then change the repo ID. I'd personally suggest that the first commit should probably be a standard .ignore file with a commit message unique to that repo.
Have a central shared_hgrc file, accessible from a network share (or in a hook repo).
Each user's global hgrc has:
%include /path/to/shared_hgrc
Create a shared repository of python hook modules. The hooks must be written in python.
Create your hook functions. In each function, check which repo the hook has been called on by checking the ID of the first commit:
# hooktest.py
import mercurial.util
FOOBAR_REPO = 'b88c69276866d73310be679b6a4b40d875e26d84'
ALLOW_PRECOMMIT_REPOS = set((
FOOBAR_REPO,
))
def precommit_deny_if_wrong_repo(ui, repo, **kwargs):
"""Aborts if the repo is not allowed to do this.
The repo ID is the ID of the first commit to the repo."""
repo_id = repo[0].hex().lower()
if repo_id not in ALLOW_PRECOMMIT_REPOS:
raise mercurial.util.Abort('Repository denied: %s' % (repo_id,))
ui.status('Repository allowed: %s\n' % (repo_id,))
def precommit_skip_if_wrong_repo(ui, repo, **kwargs):
"""Skips the hook if the repo is not allowed to do this.
The repo ID is the ID of the first commit to the repo."""
repo_id = repo[0].hex().lower()
if repo_id not in ALLOW_PRECOMMIT_REPOS:
ui.debug('Repository hook skipped: %s\n' % (repo_id,))
return
ui.status('Repository hook allowed: %s\n' % (repo_id,))
In the shared_hgrc file, set up the hooks you need (make sure you qualify the hook names to prevent conflicts):
[hooks]
pre-commit.00_skip = python:/path/to/hooktest.py:precommit_skip_if_wrong_repo
pre-commit.01_deny = python:/path/to/hooktest.py:precommit_deny_if_wrong_repo
As #Rudi said first, it can't be done for security reasons.
With some prior setup you can make it so that hooks are run on clone, but putting a hook with a repo-relative path in /etc/mercurial or in each user's ~/.hgrc, which in a corporate setting can be done via your system management tools or by building a custom Mercurial installer. In a non-corporate setting follow #Oben's advice and provide the scripts and a readme.
As the definite guide aptly points out (search for "Tags and cloning"):
When you run hg clone -r foo to clone a repository as of tag foo, the new
clone will not contain any revision newer than the one the tag refers to,
including the revision where the tag was created. The result is that you'll
get exactly the right subset of the project's history in the new
repository, but not the tag you might have expected.
It means hg tags in your new clone does NOT show the foo tag. Same thing happens if you had cloned before foo tag was added, and you do hg pull -r foo.
(Digression: tag is about the only thing I don't quite get in hg. I understand there are advantages (e.g. merge) in putting it in a changeset, but it always feels weird to have meta data mixed with source code.)
It should be obvious that I'm asking for an automated way, instead of pulling the tag changeset as a separate manual step.
I know I could check for this scenario in an incoming hook (so it works for both clone and pull), or wrap clone and pull.
But is there a better/easier way?
UPDATE hg bug tracker already has this issue.
You want a giant hack with bash and an embedded Perl script? Well, here it is...
#!/bin/bash
if [[ "$1" == "" || "$2" == "" || "$3" == "" ]]; then
echo 'hgclonetag <src> <tgt> <tag>'
exit 1;
fi
REV=`hg log -R $1 --rev $3: --limit=2 | perl -F: -lane 'if (/:([\dA-Fa-f]+)$/) {print $F[2] if defined($flag);$flag=1;}'`
hg clone --rev $REV $1 $2
This invokes the hg log command to extract the revision number after the first tag-related revision and then clones to this revision.
Currently this does not work on remote repos: -R switch only works on local repos unfortunately.
The more I think about it the more I'm convinced the right answer is to just clone everything and update to the tag, which can be done in a single step:
hg clone http://host/path#tagname
That gets you everything and then does hg update to tagname which sets your working directory to the correct revision. Given delta compression that's not necessarily much larger, and if it is you can automate cloning the bulk of it from a previous local clone.
There is a postclone hook. It's called post-clone (the hgrc manpage shows a post-ANYCOMMAND and pre-ANYCOMMAND exist) though as you pointed out you could also use *changegroup or update hooks too, since clone uses both of those functions (unless you suppress update with -U).
What about just adding a --localtag so you have the name but not the extra changeset if you need it for reference only. Something like
hg clone -r tagname URL
hg tag --local tagname
which you could easily build into a shell alias.
Other than that there's not necessarily guaranteed to be a way to have revision X and the revision where revision X is tagged without also having other revisions you don't want since the tag could have been applied after other work was done. You can, of course, always update to 'X' and to have subsequent changesets in you working dir, but they'll still be in your repo.
Honestly, once I figured out that the tag name doesn't come a long when you clone up to a tag, which I admit confused the heck out of me at first, I didn't find any need to bring along the changeset with the tag in it.
Yes it can be done by post-clone/pull hooks, but there are a couple of crooks.
First, it only works for local repo, since you can't get the list of tags in a remote repo.
Second, dealing with clone/pull arguments and options is not trivial. (For clone I need to get the target repo, -r, -u, -U. For pull I need -r and -u.) I tried to use fancyopts, but it can't deal with global options, which are processed away in dispatch. I managed to hack dispatch to give me only the args and opts of a command, but it feels and looks ugly.
Using command wrapper would eliminate the second problem.
I hope one day hg will add an option to clone and pull to do it cleanly.
I've got a Mercurial repo set up like this, with a subrepo inside another subrepo:
Root
.hg
.hgsub
.hgsubstate
Nested
.hg
.hgsub
.hgsubstate
foo.txt
FurtherNested
.hg
bar.txt
If I change foo.txt and bar.txt and commit from inside Root then all is well and all the nested subrepos commit. However if I only change bar.txt and commit from Root then Hg thinks nothing is changed. I have to commit from inside Nested to get the FurtherNested changes to commit when there are only changes in FurtherNested. It seems that in order for nested subrepos to work, each nested level has to contain changes in order for the recursion to work.
Nothing I read in the Mercurial docs on subrepos seemed to imply that subrepo commits would only propagate if there were changes. In fact it says the opposite:
When we commit, Mercurial will attempt
to recursively commit in all defined
subrepos...
So my question is, is this to be expected or is something broken or just not done yet (Mercurial 1.5.4 on Windows)?
Looks to me like this is just something that's broken. The relevant source code (in subrepo.py) doesn't seem to recurse into subsubrepos, viz:
def dirty(self):
r = self._state[1]
if r == '':
return True
w = self._repo[None]
if w.p1() != self._repo[r]: # version checked out change
return True
return w.dirty() # working directory changed
Probably wouldn't be too hard to fix, though I don't know the code intimately enough yet. It would probably be worthwhile posting on the Mercurial mailing list and/or filing a bug. I doubt Matt and Benoit are reading this, but they definitely do read everything posted there.