Regarding Mercurial Security + recursive checkins of subrepositories - mercurial

this is going to be a long post...sorry upfront.
I'm trying to wrap my head around how to hold together "Repositories for each project branch", and what the impact of that would be on a team.
Right now, it appears that
Can recursively checkin code of
nested checkins although hg status
doesn't give much info on file
changes within nested repos
It
appears that I -- and every team
member who wants to work on the same
project -- has to hand edit their
subrepositoies' .hgrc files in order
to make the checkin as painless and
automated as possible.
Can
recursively checkin, but recursively
checkout is not supported.
Is that a correct analysis of Hg's capabilities?
I'm really hoping not, as that's a lot more stick-shift coding (ie command prompt fiddling all over the place), than the average dev team I've seen could handle, while remaining productive. As I've understood it, refactoring a single assembly would probably grind the team to a halt as they stop to edit the .hgrc files to add location, user and password. No?
And I really want to double check that Hg can't recursively pull? Sounds like such an omission, that I feel I must have missed something.
Thanks!
PS:
For the brave or foolish, (and in case it helps), the notes I've been keeping as I work around the problem of projects that reference library modules that reference other library modules, is as follows (note the ???? QUESTIONS??? interspersed in them...
MERCURIAL
# requires an .hgsub with a ref to either
# an Hg Repo for only one Bin...?
# a website download...is that possible?
# an svn repo that allow referencing just one folder in it
# eg: "BIN/A3rdParty = svn:^/BinCache/A3rdParty/bin"
LibA\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/liba"
# "default-push = https://user:pwd#bitbucket.org/xact/liba"
.hgsub
# Map of nested repos as follows:
# "BIN/A3rdParty = svn:^/BinCache/A3rdParty/bin"
# "EXT/LibA = https://bitbucket.org/xact/liba"
# "EXT/LibB = https://bitbucket.org/xact/libb"
LibA.sln
BIN\
[A3rdParty\SomeLib.dll]
EXT\
SRC\
LibA\LibA.csproj
# ...which References "..\..\BIN\A3rdParty\SomeLib.dll"
LibA.Tests\LibA.Tests.csproj
# ...which References "..\LibA\LibA.csproj"
LibB\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/libb"
# "default-push = https://user:pwd#bitbucket.org/xact/libb"
.hgsub
# that contains:
# "BIN/A3rdParty = svn:^/BinCache/A3rdParty/bin"
# "EXT/LibA = https://bitbucket.org/xact/liba"
# ??? QUESTION ???
# do end users add user/pwd info here? or in the
# nested repos .hgrc file?
LibB.sln
BIN\
[A3rdParty\SomeLib.dll]
EXT\
LibA\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/liba"
# "default-push = https://user:pwd#bitbucket.org/xact/liba"
LibA.csproj
# ...which References "..\..\BIN\A3rdParty\SomeLib.dll"
LibA.Tests\LibA.Tests.csproj
# ...which References "..\LibA\LibA.csproj"
SRC\
LibB\LibB.csproj
# ...which References "..\..\EXT\LibA\LibA.csproj"
LibB.Tests\LibB.Tests.csproj
# ...which References "..\LibB\LibB.csproj"
ProjA\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/proja"
# "default-push = https://user:pwd#bitbucket.org/xact/proja"
.hgsub
# that contains:
# "BIN/A3rdParty = svn:^/BinCache/A3rdParty/bin"
# "EXT/LibA = https://bitbucket.org/xact/liba"
# "EXT/LibB = https://bitbucket.org/xact/libb"
# ??? QUESTION ???
# do end users add user/pwd info here? or in the
# nested repos .hgrc file?
BIN\
[A3rdParty\SomeLib.dll]
EXT\
LibA\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/liba"
# "default-push = https://user:pwd#bitbucket.org/xact/liba"
LibA.csproj
# ...which References "..\..\BIN\A3rdParty\SomeLib.dll"
LibA.Tests\LibA.Tests.csproj
# ...which References "..\LibA\LibA.csproj"
LibB\
hg\
.hgrc
# ??? QUESTION ???
# does each user have to edit their own files by hand
# to allow automatic push/pull?
# "default = https://user:pwd#bitbucket.org/xact/libb"
# "default-push = https://user:pwd#bitbucket.org/xact/libb"
LibB\LibB.csproj
# ...which References "..\..\EXT\LibA\LibA.csproj"
# Important: note that it is same path offset
# as when within context of LibB.sln
LibB.Tests\LibB.Tests.csproj
# ...which References "..\LibB\LibB.csproj"
SRC\
ProjA\ProjA.csproj
ProjA.Tests\ProjA.Tests.csproj

I will try to answer some of your questions though I really think you should discuss this with us instead of doing a Q&A here.
Right now, it appears that
Can recursively checkin code of nested checkins although hg status doesn't give much info on file changes within nested repos
See hg status --subrepos or hg status -S for short.
It appears that I -- and every team member who wants to work on the same project -- has to hand edit their subrepositoies' .hgrc files in order to make the checkin as painless and automated as possible.
No need to put usernamed and passwords into the .hg/hgrc files -- you should instead configure caching of HTTP credentials in Mercurial.
Can recursively checkin, but recursively checkout is not supported.
Checkout, i.e., update, is recursive. When you do hg clone to get a local repository, then Mercurial will notice the .hgsub and .hgsubstate files and it will recursively clone the subrepositories referenced there.
And I really want to double check that Hg can't recursively pull? Sounds like such an ommission, that I feel I must have missed something.
Yes, you're missing how Mercurial knows which subrepositories you want. Please see the documentation on the wiki or the Kick Start guide.

Related

Repository specific special / dot files in Mercurial

What are the most important special / dot files when using a hg repo ?
Like configuration files and similar.
There are a lot of files that a Mercurial repo will use for configuration or keep track of state, but here are the ones that have the best chance to come in handy:
.hg/last-message.txt -- used by hg commit to store backup of the commit message in case the commit fails.
Example:
My commit message!
.hg/localtags -- define local tags which are not shared among repositories.
Example:
8a7b128ab80b58fc2e63258c9e2bf1f58a5be7c2 myfirsttag
08ff3a0b2e5af9a74becbfdf3e92d6e9a2d0c960 secondtag
6535d105ea795a38808481b160314f9857736c53 thirdtag
.hgignore -- regular expressions that describe file names that should be ignored by hg.
Example:
syntax: glob
*.elc
*.orig
*.rej
*~
*.mergebackup
*.o
*.so
*.dll
*.exe
*.pyd
*.pyc
.hg/hgrc -- defaults and configuration values for mercurial.
Example:
[ui]
verbose = True
username = Joe User <j.user#example.com>
[extensions]
hgext.churn = /home/user/hg/hg/contrib/churn.py
[hgk]
path = /home/user/hg/hg/contrib/hgk
.hgsub -- locations of all subrepositories and where subrepository checkouts came from.
Example:
subrepo1 = https://user#example.org/user/repo
subrepo2 = https://user#example.org/user2/repo2
.hgtags -- contains changeset hash values and text tag names
Example(same format as localtags):
8a7b128ab80b58fc2e63258c9e2bf1f58a5be7c2 myfirsttag
08ff3a0b2e5af9a74becbfdf3e92d6e9a2d0c960 secondtag
6535d105ea795a38808481b160314f9857736c53 thirdtag

HG command line input receives unwanted default input automatically in THG

I've written an Hg hook (in Python) to check the validity of the committed files according to out team rules. One of these rules prohibits files larger than XX kB, unless agreed with the ream. In case a large file is committed, I would like the hook to ask the user to allow the file.
I implemented it like this:
import re, os, sys, mercurial
MAX_SIZE_KB = 500
def check_committed_files(ui, repo, **kwargs):
changelog = repo.changelog._cache
lines = changelog[2].splitlines()
ui.status("Checking files...\n")
for line in lines[3:-2]:
fn = line
ui.status(" " + fn)
# check file size
file_size_kb = float(os.stat(line).st_size) / 1024
if file_size_kb > MAX_SIZE_KB:
if ui.prompt(" Allow file [%s] of %g kB?" % (fn, file_size_kb)).lower() not in ['y', 'yes']:
ui.warn(" Not allowed by user\n")
return 1
ui.flush()
return 0
It all works well if I use Hg CLI. But when I use TortoiseHg, the prompt is automatically yes-ed, so I get this in console:
Allow file [test.txt] of 2573.49 kB? y
and the hook goes on. I would like TortoiseHg to show a dialogue with Yes/No buttons. Is it possible? I'd like to have the solution as portable as possible, so e.g. no external Python modules that users need to install.
Since this is my first attempt with Hg hooks, any other comments on my implementation are also much appreciated.

Mercurial: TortoiseHg configuration to always prompt for manual resolve for all files when merging

We use Mercurial via TortoiseHg with the 'workbench'.
When merging branches, we often find the default (auto?) choice of updates is not what we want. We'd like to configure the tool to always prompt for any merge operation.
Each branch has some core files that we need to keep up to date between the two, but also some branch-specific files which need to be preserved.
If we update the core files in one branch we'd like to do a merge to get the latest core files in the other branch. However the default TortoiseHg makes automatic updates that we don't want.
Examples:
1) Branch '1' contains a file '1.xls' and branch '2' has an equivalent '2.xls' which are stored as binary. Merging '1' into a working directory based off '2' causes a delete of '2.xls' and an add of '1.xls'.
2) Branch '1' contains an updated file 'not_core.c' which has an equivalent 'not_core.c' in branch '2'. Merging '1' into a working directory based off '2' causes an unwanted automatic update of 'not-core.c' in '2'.
3) Branch '1' contains updated files 'core1.c' & 'core2.c'. Merging '1' into a working directory based off '2' causes an automatic update of both files, when we'd prefer a prompt to allow us to chose if these core files are updated. Maybe we want just 'core1.c' for now.
I have seen reference to setting 'mergetool.premerge = false' in mercurial.ini, and have tried some variants of this without success.
The ideal setup would be that the graphical manual resolve is triggered for every file that is different.
Can anyone help with a configuration that supports the functionality described?
Thanks, Ian
Three described cases have different nature in you case, you have to use different tricks for different cases
Cases 2 and 3 (non-conflicting changes in files of ranches) can be covered (TBT!) by disabling auto-merges in TortoseHG (Main menu - File - Settings - Global setting - TortoiseHG - "Auto-resolving merges": False /default - True/)
For case 1 (and other non-content changes - renames, changing attributes) situation is worse and automerge doesn't play: merge mirror history of changes from one branch to another, i.e if in common parent you have only 2.xls, which was deleted|renamed to 1.xls in source branch this change will be reflected in target.
You can try to use different "internal" merge-tool in such merges (when your prefer changes only from one branch - "dummy merge") - internal:local and internal:other. But - they work only for changed content (again!)
Maybe Keep "My" or "Their" files when doing a merge from TipsandTricks section will help (you can join mentioned merge-revert-resolve into new command in [aliases] and use it instead of ordinary merge, when needed)
You can configure this using the merge-patterns and merge-tools sections of hgrc. This set up allowed me to force kdiff3 to be used in my test:
[merge-patterns]
** = noautomerge
[merge-tools]
noautomerge.executable = kdiff3
noautomerge.args = $base $local $other -o $output --qall
noautomerge.priority = 1
noautomerge.premerge = False
This will turn off the automatic pre-merge that Mercurial does and start up kdiff3 instead.
It will do it for all files but it you might be better to amend the merge-patterns section to be a little more specific if that is possible.
Also, be aware of the possibility that your merge tool might automatically resolve the merge itself and if that happens look for command line options to turn that feature off.

Mercurial ignore all files except specific file names

I have a large file system in which almost every folder has a file called content.txt
I want to track every file named content.txt and automatically ignore everything else. I want the repo to automatically track new files named content.txt so I don't want to ignore everything in the .hgignore and then manually add.
Anyone know how to do this?
It has to be regexp mode, not glob
You must debug path-part of regexp, but "all except content.txt" draft is re:.*\.(?!content.txt) as hope
Alternative solution can be
* ignore all
* add content.txt files pattern to commit command (-I option), see hg help commit and hg help patterns
hg commit -I '**content.txt'
Edit
re:.*/(?!content.txt)
Try this:
syntax: regexp
\.(?!txt$)[^.]+$ # "*." is followed by "txt" and we're at the end
(?<!\.)txt$ # "txt" follows a "."
(?<!/)content\. # "content." follows path separator
(?<!content)\. # "." follows "content"
I left in the comments I made while experimenting, to make sense of it all. (That's glob syntax in the first one.)

Mercurial: allow merge from a release branch to the default one, but not vice versa

I'm using default branch for ongoing development, and now going to create a new named branch to mark a release. All further development will be on the default branch, all production bugfixes will be done on the new one (with subsequent merge to default), like this:
#>hg branches
aristotle 42:dbd...
default 41:da5...
#>hg branch
default
#>echo "Feature #1 for the next release" >> feature1.txt
#>hg add
#>hg commit -m "Implement feature #1 for the next release"
...... eek, need to make an urgent fix on Production .....
#>hg update aristotle
#>echo "Fixed urgent bug #123 on Production" >> fix123.txt
#>hg add
#>hg commit -m "Fixed bug #123 on Production"
created new head
#>hg update default
#>hg merge aristotle
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, dont forget to commit)
#>hg commit -m "Merge in the fix for bug #123"
#>hg push
The above seems the way to go, however it seems easy to mess things up and merge the other way around (from default to aristotle which means all the new features will appear in the production branch).
Maybe my fears are groundless because one will notice the mess before pushing the commit to the central repo, but I'd like to see if it's possible to make the approach more foolproof.
So I started looking into hooks:
[hooks]
pretxnchangegroup.branch = hg heads --template "{branches} " | find "aristotle" && exit 1 || exit 0
..but then realized it's not what I need, because this will not allow me to push aristotle changes at all.
So I'm not sure what to do. Ideally, I want developers to see the "wrong way merge" message when they attempt to commit a merge from default to aristotle locally (obviously, there should be a double-check on the central repo), while merging from production branch to the default one should be possible.
Thanks!
This should do it. It uses a revset query to find any merges into aristotle from default.
hg log -r 'children(p2(::aristotle and ::default and merge()) and branch(default)) and branch(aristotle)'
::aristotle and ::default and merge() finds all merges that are ancestors of both aristotle and default branches
p2(...) and branch(default) grabs the second parent (incoming changeset) that are on the default branch.
children(...) and branch(aristotle) then grabs the actual merge changeset that is on the aristotle branch.
I recently needed to figure this out myself but also needed to ensure that default wasn't indirectly merged into my release branch, i.e. default -> feature -> release.
This is pretty much an exact duplicate of a question from a few days ago Ensuring a merge between branches happens in one direction
but I didn't like my answer to that one, so how about this one:
Keep separate clones for past releases. When, in your example above, you decide you need to do an emergency fix on aristotle do this instead:
cd ..
hg clone -r aristotle myrepo myrepo-aristotle
then you have a clone with only aristotle in it and you can't accidentally merge it to default in that clone.
That said, it's still not a great answer. The only real consolation is that if you merge in a direction you don't like you can always re-clone the two heads and merge in the other direction.
I found this question whilst looking for a solution to a related problem, I know its an old question, but thought I would share our solution.
We have release branches called release-x.y.z and ongoing development on default. We use a hook based on the example precommit_badbranch_badmerge found at https://www.mercurial-scm.org/wiki/HookExamples.
We basically extract the version number for each branch involved in any merge and make sure it is going the right way (default is treated as 9999.9999.9999 anything that isn't a release or default branch gets -1,-1,-1 (can be merged into anything).
Note that this only does a simple 'immediate' check, it doesn't catch issues where someone merges from default to the blah branch and then onto a release branch.
As a side note, we also have a policy that the merge to default must be pushed at the same time as the original change on the release branch, this avoids someone else having to do the merge in order to merge their own changes - it is enforcing that that I was looking for a solution for.
Note, you invoke this using the hooks section of the server/central repo
[hooks]
pretxnchangegroup.prevent_bad_merges = python:<path_to_hook>\prevent_bad_merges.py:prevent_bad_merges
python hook below:
# based on precommit_badbranch_badmerge
# at https://www.mercurial-scm.org/wiki/HookExamples
import re
# this isnt a proper compare, it will just return numbers > 0 if source is after target
def compare_versions(source_version, target_version):
# if either side is -1, ignore it
if (source_version[0] == -1) or (target_version[0] == -1):
return 0;
if source_version[0] > target_version[0]:
return 1
elif source_version[0] == target_version[0]:
if source_version[1] > target_version[1]:
return 2
elif source_version[1] == target_version[1]:
if source_version[2] > target_version[2]:
return 3
return 0
def get_version(branch):
if branch == 'default':
major=9999
minor=9999
revision=9999
else:
# note python uses ?P for named matches
match = re.match('(release-(?P<major>\d)\.(?P<minor>\d)\.(?P<revision>\d))', branch)
if match:
major = int(match.group('major'))
minor = int(match.group('minor'))
revision = int(match.group('revision'))
else:
major = -1
minor = -1
revision = -1
return [major,minor,revision]
def prevent_bad_merges(ui, repo, node, **kwargs):
ui.debug("in hook\n")
for rev in xrange(repo[node].rev(), len(repo)):
ui.debug("in loop\n")
# get context (change)
ctx = repo[rev]
ui.debug("got ctx\n")
if len(ctx.parents()) > 1:
ui.debug("got a merge\n")
branch = ctx.branch()
ui.debug(branch +"\n")
parent1 = ctx.parents()[0]
ui.debug("got parent1\n")
parent2 = ctx.parents()[1]
ui.debug("got parent2\n")
target_branch = repo[parent1.node()].branch()
ui.debug("got parent1 branch\n")
target_version = get_version(target_branch)
ui.debug("got parent1 version\n")
source_branch = repo[parent2.node()].branch()
ui.debug("got parent2 branch\n")
source_version = get_version(source_branch)
ui.debug("got parent2 version\n")
# This could happen if someone does
# hg update 1.1-branch
# hg branch 1.2-branch
# hg merge 1.0-branch
# which is a strange thing to do. So disallow it.
if target_branch != branch:
ui.warn('PREVENT BAD MERGE HOOK FAILED : \n'
'merging to a different branch from first parent '
'is just weird: please don\'t do that\n')
return True
ui.debug(source_branch, "\n")
ui.debug(str(source_version[0]), "\n")
#ui.debug("major:", source_version[0], "\n")
#ui.debug("minor:", source_version[1], "\n")
#ui.debug("revn :", source_version[2], "\n")
ui.debug(target_branch, "\n")
ui.debug(str(target_version[0]), "\n")
#ui.debug("major:", target_version[0], "\n")
#ui.debug("minor:", target_version[1], "\n")
#ui.debug("revn :", target_version[2], "\n")
# Check for backwards merge.
if compare_versions(source_version, target_version) > 0:
ui.warn('PREVENT BAD MERGE HOOK FAILED : \n'
'invalid backwards merge from %r to %r\n'
% (source_branch, target_branch))
return True
else:
ui.debug("Not a merge\n")
# Not merging: nothing more to check.
return False