I have a very large repo with thousands of files that can regularly get updated by automatic processes that are out of my control (this is for Unity 3D, for what it's worth).
For example, if I upgrade Unity to a new version, it will reimport all textures and maybe add a line in thousands of .meta files that correspond to a new serialized data that didn't exist previously.
Obviously reviewing thousands of files is terrible. Most of the time though, I can quickly identify a particular diff, and would just like to automatically check all the files that have the same diff, commit to get them out of the way, and see what's left: other diffs that I might not know about.
For example I just commited 4000+ files that all contained this diff:
So the pattern would be easy to find:
- textureFormat: -5
+ textureFormat: -1
I suppose I could write a script, or a TortoiseHg tool to do that, I just have no idea where to begin. I'd need to iterate over all changed files/chunks, match a pattern, commit the chunks...
I know of no tool to do exactly what you want. However I believe it's relatively easy to write a small bash script for such or use the command line:
hg diff --nodates --noprefix -U 0 | grep '^+' | grep -v '+++' | sort | uniq -c
will list you the inserted lines of the current diff in descending order of the number of occurences, thus the most frequently occurring diff first.
With that list you get a list of files which match the newly inserted pattern, for instance
hg files "set:grep('^ textureFormat: -1')"
should give you all files with that pattern (whether it's new or not, though). You probably want to check those files, whether their diff contains anything else:
hg diff "set:grep('^ textureFormat: -1')"
Now you can make use of the results and even exclude single files, if the diff output didn't suit you:
hg commit "set:grep('^ textureFormat: -1') and not 'unwantedFilename.cpp'"
In the above commands I made use of the fileset capability and of hg grep which accepts regular expressions. Check hg help grep, hg help fileset and hg help patterns for a more in-depth explanation.
Related
I'm looking for a way to make Mercurial output a table like this:
File Most recent revision changing the file Date of that revision
==== ====================================== =====================
foo.py 44159adb0312 2018-09-16 12:24
... ... ...
This is just like github does it on the "Code" overview page. (screenshot from torvalds/linux):
"Most recent" could refer the date or to the DAG hierarchy relative to the current changeset, or maybe to the current branch. Perhaps the latter is more useful, but in my particular use case, it doesn't make a difference.
I'd also like to be able to provide a list of files or a subdirectory for which I want the table. (I don't necessarily want it for everything)
I am aware that I could do it using a small script, looping over hg log -l 1 <file>, but I was wondering if there is a more efficient / more natural solution.
You won't get around looping over all files. Yet with hg manifest you get that list of files. Then template the output as needed:
for f in $(hg ma); do hg log -l1 $f -T"$f\t\t{rev}:{node|short}\t\t{date|isodate}"; done
This gives output like
.hgignore 38289:f9c426385853 2018-06-09 13:34 +0900
.hgsigs 38289:f9c426385853 2018-06-09 13:34 +0900
.hgtags 38289:f9c426385853 2018-06-09 13:34 +0900
You might want to twiddle more with the output formatting. See the mercurial wiki for a complete overview of output templating.
Git will follow the commit DAG, because that's all it has. In Mercurial, you have (many) more options because you have more data.
Probably the ideal option here is follow(file, .) (combined with first or last as appropriate). But as hg help revset will tell you, you have the following options (I've shrunk the list to the obvious applicable ones):
ancestors(set[, depth])
Use this with the set being . to get ancestors of the current commit, for instance, if you want to do DAG-following a la Git. Or, use ::., which is basically the same.
branch(string or set)
Use this with . to get all commits in the current branch. Combine with other restrictors (e.g., parents) to avoid looking at later commits in the current branch if you're not at the tip of the current branch.
file(pattern)
Use this with a glob pattern to find changesets that affect a given file.
filelog(pattern)
Like file but faster, trading off some accuracy for speed (see documentation for further details).
follow([file[, startrev]])
To quote the documentation:
An alias for "::." (ancestors of the working directory's first parent).
If file pattern is specified, the histories of files matching given
pattern in the revision given by startrev are followed, including
copies.
modifies(pattern)
Use this (with any pattern, not just glob) to find changesets that modify some file or directory. I think this is limited to M type modifications, not addition or removal of files, as there is also adds(pattern) and removes(pattern). Use all three, or-ed together, to find any add/modify/remove operations.
first(set, [n])
last(set, [n])
limit(set[, n[, offset]])
Use this to extract a particular entry out of the revset.
When searching forwards (the default), last(follow(file, .)) seems to work nicely to locate the correct revision. As you noted, you have to do this once per file—it will definitely go faster if you write your own Mercurial plug-in to do this without reloading the rest of the system all the time.
Somehow more efficient / more natural solution can be:
create template|style for desired log output (I can't predict, which way will be better for you)
create alias for hg log -l 1 --template ... or hg log -l 1 --style ...
EDIT
A lot later, more correct solution (from recent discoveries) with hg grep
hg grep "." "set:**.py" --files-with-matches -d -q -T"{files % '{file} {date|age}\n'}"
Part of output in test-repo
hggit/__init__.py 7 weeks ago
hggit/git_handler.py 7 weeks ago
hggit/gitdirstate.py 7 weeks ago
…
You have to modify fileset in order to get results only for part of your tree (for all branches) and, maybe, template in order to fulfill your needs.
I didn't have fileset for selecting "files in branch X" just now, I think, it will be something using revs() predicate
"revs(revs, pattern)"
Evaluate set in the specified revisions. If the
revset match multiple revs, this will return file matching pattern in
any of the revision.
because some not published predicates (according to examples, see # "set:revs('wdir()'..." for referencing working directory) can be used for defining revset and I can't discover/predict the correct form for branch predicate
i'm trying to find out who did the last change on a certain line in a large xml file ( ~100.00 lines ).
I tried to use the log file viewer of thg but it does not give me any results.
Using hg annote file.xml -l 95000 .. took forever and eventually died with an error message.
Is there a way to annote a single line in a large file that does not take forever ?
You can use hg grep to dig into a file if you have interest in very specific text:
hg grep "text-to-search" file.txt
You will likely need to add the --all switch to get every change that matches and then limit your results to a specific changeset range with -r firstchage:lastchange syntax.
I don't have a file on hand of the size you are working with, so this may also have trouble past a certain point, particularly if the search string matches many many lines in the file. But if you can get as specific as possible with your search string, you should be able to track it down.
how do I open from terminal window only files with conflicts during git/mercurial merge in textmate/sublime text2 editors
You can use the following to open all files with git merge conflicts in sublime text:
git diff --name-only | uniq | xargs subl
I wanted to add another answer. git diff --name-only will give you all files that have diffs. This is why sometimes it will yield duplicate entries because it marks the file as "modified" as well as in a merge conflict state. Piping it into uniq is a good solution for this but git diff --name-only will also include files you might have purposely changed so it doesn't actually filter only files with merge conflicts. When you are in the middle of rebasing, this is probably not going to happen often though I would say in most cases #StephanRodemeier's answer works.
However, what you can do though is leverage the --diff-filter option which assigns a states to files. See more in the docs
--diff-filter=[(A|C|D|M|R|T|U|X|B)…[*]]
Select only files that are Added (A), Copied (C), Deleted (D), Modified (M), Renamed (R), have their type (i.e. regular file, symlink, submodule, …) changed (T), are Unmerged (U), are Unknown (X), or have had their pairing Broken (B). Any combination of the filter characters (including none) can be used. When * (All-or-none) is added to the combination, all paths are selected if there is any file that matches other criteria in the comparison; if there is no file that matches other criteria, nothing is selected.
It seems when files are in the both modified state, the diff status gets set to U (Unmerged) and M (Modified) so you can filter for only Unmerged files.
git diff --diff-filter=U --name-only | xargs subl
Should work without needing to pipe into uniq
Another thing you can consider is simply setting your editor as the difftool i.e. for VSCode documentation specifies how to do this by adding this to your .gitconfig
[diff]
tool = default-difftool
[difftool "default-difftool"]
cmd = code --wait --diff $LOCAL $REMOTE
I have recently encountered the need to generate a Mercurial diff of all changes up to a particular changeset which includes the first changeset of the repo. I realize this kind of stretches the definition of a diff, but this is for uploading a new project to a code review tool.
Let's assume the following changesets:
p83jdps99shjhwop8 - second feature 12:00 PM
hs7783909dnns9097 - first feature - 11:00 AM
a299sdnnas78s9923 - original app setup - 10:00 AM
If I need a "diff" of all changes that have been committed, the only way that I can seem to achieve this is with the following diff command...
diff -r 00:p83jdps99shjhwop8
In this case the first changeset in the argument param (here - 00) takes the regexp form of 0[0]+
This seems to be exactly what we need based on a few tests, but I have had trouble tracking down documentation on this scenario (maybe I just can't devise the right Google query). As a result, I am unsure if this will work universally, or if it happens to be specific to my setup or the repos I have tested by chance.
Is there a suggested way to achieve what I am trying to accomplish? If not, is what I described above documented anywhere?
It appears this actually is documented, but you need to do some digging...
https://www.mercurial-scm.org/wiki/ChangeSetID
https://www.mercurial-scm.org/wiki/Nodeid
So the special nodeid you're referring to is the 'nullid'.
2 digits may not be adequate to identify the nullid as such (as it may be ambiguous if other hashes start with 2 zeros), so you may be better off specifying 4 0's or more.
Eg: hg diff -r 00:<hash of initial add changeset> has resulted in the abort: 00changelog.i#00: ambiguous identifier! error.
I'm a little confused about what you need. The diff between an empty repository and the revision tip is just the content of every file at tip-- in other words, it's the state of your project at tip. In diff format, that'll consist exclusively of + lines.
Anyway, if you want a way to refer to the initial state of the repository, the documented notation for it is null (see hg help revisions). So, to get a diff between the initial (empty) state and the state of your repository at tip, you'd just say
hg diff -r null -r tip
But hg diff gives you a diff between two points in your revision graph. So this will only give you the ancestors of tip: If there are branches (named or unnamed) that have not been merged to an ancestor of tip, you will not see them.
3--6
/
0--1--2--5--7 (tip)
\ /
4
In the above example, the range from null to 7 does not include revisions 3 and 6.
I wonder whether there is a mercurial command/extension that just tests whether a given changeset is in a branch. The command would be something like:
hg contains [-r branch] changeset_id
and should check whether the given changeset is in the current/given branch, returning just "Yes" or "No".
I know about the "debugancestor" command, but a "Yes/No" answer is way easier to read.
And if there is, is it possible to check for transplanted changesets as well?
EDIT: The scenario is located in a repo where named branches have multiple heads. Lets say a branch is named "dev-X", having more than 1 head and a longer history, too long at least to track it with various graph visualizations.
I want to figure out whether a changeset X in branch "dev-X" was merged into another head of "dev-X". Therefore I cannot use branch names but only changeset numbers/hashes to specify a branch.
And to top it all, I'm trying to find out whether changeset X was transplanted there, possibly taking more than 1 transplantation step. I know that the necessary info is stored in mercurial (I've seen it when tampering with the mercurial internals), it's just not accessible via the command line interface.
How about this:
hg log -r changeset_id -b branchname
That will give some output if changeid_id includes changes on branch branchname, otherwise no output is returned.
You could wrap it in a bash function if you want:
function contains() {
if [ "$(hg log -r $1 -b $2)" == "" ]
then
echo no
else
echo yes
fi
}
which does this:
$ contains 0 default
yes
$ contains 0 other
no
using 1.6 and later with the power of revision sets all you need is
hg log --rev "ancestors(.) and <revNum>"
eg
hg log --rev "ancestors(.) and 1234"
blank means no, output means yes, its in your history. Some of the other solutions posted here wont work if the changeset was created in a named branch, even if it was merged at some point later.
As mentioned in the comment above I gave it a shot, this is what came out:
http://bitbucket.org/resi/hg-contains/
It should be pretty easy to transform the results from debugancestor into a yes or a no (but there's definitely no built-in way to do that; write a script already!). Be aware that the answer might be wrong if the branch has more than one branch head, though.
(Writing an extension to add a command to do this should also be nigh-trivial, BTW.)
You could always just print out the name of the branch for that revision (it'll be empty if it's default) and then test that against whatever you want (in bash or in a scripting language of some sort):
hg log --template '{branches}' -r <revision name/number>
I've tested most of approaches above, did not work. The extension 'contains' somehow takes wrong revision (I think its a bug), the hg log --rev "ancestors(.) and 1234" work, but I found even more simple approach to do this:
hg merge -P <changeset>
Will show you if anything unmerged remains (it will also include changesets which are not merged parents of the changeset in question)