Why does mercurial's log --follow option miss changesets? - mercurial

If I use log -k it pulls up two changesets, but when I add --follow or -f it misses one of them. Why would that be?
(I'd thought that --follow could only add changesets to the set returned, i.e., the ones that had modified selected files before their names were changed to the current names.)

The -f option only follows the history through ancestors/descendents of the starting revision.
My guess is that -k (the keyword search) is returning results from two branches in your repo. Therefore, by adding -f / --follow, you are restricting the results to the changesets that are directly related to the starting revision.

I'm back here because I finally get (mostly) what --follow keeps and what it drops. Although Edward was basically right, I was confused after testing out his answer because I'd misunderstood what was meant by "starting revision," and also there are some nuances about the way it follows ancestors/descendants that led to results that didn't fit what I expected.
Here's a repository with a simple branch:
% hg glog --template {rev}
# 2
|
| o 1
|/
o 0
% hg glog --template {rev} -r '0+1+2'
# 2
|
| o 1
|/
o 0
% hg glog --template {rev} -r '1+2+0'
# 2
|
| o 1
|/
o 0
The next examples using --follow illustrate that the "starting revision" that's relevant to --follow is the first one in the specification (not the earliest in the repository or anything like that), and that even the subsequent ordering matters.
Everything is included if we start with the shared ancestor.
% hg glog --template {rev} -r '0+1+2' --follow
# 2
|
| o 1
|/
o 0
If we don't, then the results even depend on the order of later ones. It looks like it can traverse across the graph, but only in one direction (to ancestors or to descendants, but not both in the same call.)
% hg glog --template {rev} -r '1+2+0' --follow
o 1
|
% hg glog --template {rev} -r '1+0+2' --follow
o 1
|
o 0
And it's not just an issue if you're ordering manually. Using the basic primitives, you can wind up with the revisions in different orders, leading to confusing behaviors.
% hg log --template {rev} -r 'all()'
012
% hg log --template {rev} -r 'all()' --follow
012
% hg log --template {rev} -r 'reverse(all())'
210
% hg log --template {rev} -r 'reverse(all())' --follow
20
I still don't have a clear and concise specification in mind for what it does, and I think it's tricky to generate revsets that are "safe" for follow. (Could hg give a warning message when changesets are dropped from the results for this reason?) So when I'm doing tricky processing that depends on --follow, I run without --follow too, comparing to make sure I'm not losing anything.

I think that your expectation is valid, and I have found --follow to be a bit buggy, when combined with other options.
See these confirmed (and somewhat old) bugs:
https://bz.mercurial-scm.org/show_bug.cgi?id=4959 [Marked FIXED as of Jan 2018]
https://bz.mercurial-scm.org/show_bug.cgi?id=5053
For me, a workaround was to use the --rev "follow('my/filename')" revset syntax, rather than the problematic --follow flag.
Maybe give that a shot?

Related

Flatten Mercurial Revision Chain

I have a long sequence of mercurial changes (each with only a single parent), some of which conflict with each other and others which do not. I'm trying to "flatten" that sequence of changes such that the resulting tree has the minimum depth, without triggering manual merges.
I have a simple bash script which seems to do this. It works by continually trying to rebase revisions on top of their grandparent revision. However, this approach is incredibly slow for long revision chains.
#!/bin/bash
for rev in $(hg l --template "{node}" | egrep --only-matching "[0-9a-f]+")
do
while :
do
# Attempt to rebase the revision on top of its grandparent.
hg rebase -s $rev -d "first(parents(parents($rev)))"
if [ $? -eq 0 ]
then
# Find the updated revision after the rebase.
rev=$(hg log --hidden --rev "$rev" | egrep rewritten | egrep --only-matching "[0-9]{4}[:][0-9a-z]+")
else
hg rebase --abort
break
fi
done
done
Is there a faster way to do something similar?
Linear history can't contain "conflicting" changes (in case of overlapping changes latest win all)
You have at least two ways (if we'll forget about MQ) for squashing commits for any size of commit-history
histedit
Fold command from Evolve extension

Some hg changesets not merging after graft

I have two hg branches (dev and stable) that aren't merging like I'd expect.
On stable: I grafted in a one-line commit from dev.
On dev: Changed that one line that was grafted, committed change.
On stable: merged dev into stable (no conflicts).
However after this merge stable still has the grafted version of the line (step 1). Not the latest changes to that same line from dev (step 2). Why is this?
The file looks like:
This
file
to
be
merged
Changesets:
Changes "to" to "might" on dev
Grafts changeset 1 to stable
Changes "might" back to "to" on dev
Merges dev into stable. Result is "might" (not "to" like I'd expect to see from changeset 3).
Sorry about the delay here: As soon as you shrank the reproducer to the five commits, I knew what was going on, but I wanted to write my own reproducer before answering, and the priority of this dropped a lot. 😀 The script I used, mktest.hg, to create the commits, the graft, and the merge, appears at the end of this answer.
The key issue here is the way merge actually works in Mercurial. It uses the same algorithm as Git does: that is, it completely ignores any of the branch information, and completely ignores any timing information. It looks only at three specific commits, as found by examining the commit graph, as shown in your image. Here's a text variant via my own reproducer:
$ cd test-hg-graft/
$ cat file.txt
This
file
might
be
merged
$ hg lga
# 4:b027441200d2:draft stable tip Chris Torek
|\ merge dev into stable (9 minutes ago)
| |
| o 3:01c6cc386a08:draft stable Chris Torek
| | back to "to" on stable (9 minutes ago)
| |
| o 2:ad954507e465:draft stable Chris Torek
| | s/to/might/ (9 minutes ago)
| |
o | 1:f7521e4f0941:draft dev Chris Torek
|/ s/to/might/ (9 minutes ago)
|
o 0:a163d2c4874b:draft stable Chris Torek
initial (9 minutes ago)
The lga alias is one I stole borrowed copied from someone else:
lga = log -G --style ~/.hgstuff/map-cmdline.lg
where map-cmdline.lg is in the link above. It's just log -G (aka glog) with a more-compact format.
What's going on
When we run:
hg merge dev
Mercurial locates three specific commits:
The current commit on stable, -r3 in this case (the SHA ID will vary), is one of the two endpoint commits.
The target commit on dev is the result of resolving dev to a revision. We can do this ourselves with hg id -r dev for instance:
$ hg id -r dev
f7521e4f0941 (dev)
$ hg id -n -r dev
1
Note that we can do the same thing with # to identify our current revision, although hg summary spills everything out more conveniently.
Last (or in some sense first, though we need the other two to get here), Mercurial locates a merge base commit from these two commits. The merge base is the first commit in the graph that is reachable from both of the other inputs to the merge. In our particular case, that's rev zero, since we split the branches apart right after -r0.
Technically, the merge base is the output of a Lowest Common Ancestor algorithm as run on the Directed Acyclic Graph. See Wikipedia for some examples. There can be more than one LCA; Mercurial picks one at (apparent) random for this case. In our case there is only one LCA though.
Having found the merge base, Mercurial now runs the equivalent of two diff operations:
hg diff -r 0 -r 3
to see what we changed, and:
hg diff -r 0 -r 1
to see what they changed, since the merge base snapshot.1 If we do this ourselves, we see what Mercurial sees:
$ hg diff -r 0 -r 3
$ hg diff -r 0 -r 1
diff --git a/file.txt b/file.txt
--- a/file.txt
+++ b/file.txt
## -1,5 +1,5 ##
This
file
-to
+might
be
merged
(I have my hg diff configured with git = true so that I get diffs that I can feed to Git—long ago I was doing a lot of conversion work here.)
As far as Mercurial is concerned, then, we did nothing on our branch. So it combines do nothing with make this change to file.txt and comes up with this one change to file.txt. That one change is applied to the files from the merge base commit. The resulting files—well, file, singular, in this case—are the ones that are ready to go into the final merge commit, even though they're not the ones you wanted.
Because Mercurial has more information than Git—in particular, which branch something happened on—it would be possible for Mercurial to behave differently from Git here. But in fact, both do the same thing with this kind of operation. They both find a merge base snapshot, compare the snapshot to the two input commit snapshots, and apply the resulting combined changeset to the files from the merge base. Mercurial can do a better job of catching file renames (since it knows them, vs Git, which just has to guess) and could do a different job of merging here, but doesn't.
1Some might object that Mercurial stores changesets, not snapshots. This is true—or rather, sort of true: every once in a while, Mercurial stores a new copy of a file, instead of a change for it. But as long as we have all the commits needed, storing changes vs storing snapshots is pretty much irrelevant. Given two adjacent snapshots, we can find a changeset, and given one snapshot and a changeset to move forward or backward, we can compute a new snapshot. That's how we can extract a snapshot in Mercurial (which stores changesets), or show a changeset in Git (which stores snapshots).
Script: mktest.hg
#! /bin/sh
d=test-hg-graft
test "$1" = replay && rm -rf $d
if test -e $d; then
echo "fatal: $d already exists" 1>&2
exit 1
fi
set -e
mkdir $d
cd $d
hg init
hg branch stable
cat << END > file.txt
This
file
to
be
merged
END
hg add file.txt
hg commit -m initial
hg branch dev
ed file.txt << END
3s/to/might/
w
q
END
hg commit -m 's/to/might/'
hg checkout stable
hg graft -r 1 # pick up s/to/might/; graft makes its own commit
ed file.txt << END
3s/might/to/
w
q
END
hg commit -m 'back to "to" on stable'
hg merge dev
hg commit -m "merge dev into stable"

Mercurial commands equivalency

I am trying to find out if the following Mercurial commands are equivalent (beside the additional + in id -i output if the directory is has modified files): hg id -i and hg log -l1 --template "{node|short}". Are they or are they not?
The purpose is to potentially speed up some automation not needing to execute the id -i and use {node|short} in the log template instead. They seem equivalent but I'd rather if somebody else took a look as well before I proceed.
They're equivalent under certain conditions. log -l 1 gives you the first revision when all revisions are ordered from tip ("newest") to 0. hg id tells you what you currently have checked out. So if you've not checked out tip (say you've checked out a branch org a specific revision) you'll get entirely different output:
ry4an#four:~/projects/mercurial-crew$ hg log -l1 --template "{node|short}\n"
7eda5bb9ec8f
ry4an#four:~/projects/mercurial-crew$ hg id -i
824f7b3545c1
If, however, you've checked out tip (and you have no modified files) then they're identical (except for log not providing a newline without \n):
ry4an#four:~/projects/mercurial-crew$ hg checkout tip
4 files updated, 0 files merged, 0 files removed, 0 files unresolved
ry4an#four:~/projects/mercurial-crew$ hg id -i
7eda5bb9ec8f
ry4an#four:~/projects/mercurial-crew$ hg log -l1 --template "{node|short}\n"
7eda5bb9ec8f
They're not the same, because hg id defaults to examining the work-tree, and:
$ hg merge side
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg id -i
eb51a3480d0c+62e318575165+
if you're in the middle of merging, the work-tree has two parents and hg id prints both of them.
Meanwhile, hg log defaults to looking at the current (oops, thanks to Ry4an Brase) tip commit (rather than the work-tree); {node | short} prints its ID:
$ hg log -l1 --template '{node|short}'
eb51a3480d0c
On the other hand, if you're not in the middle of a merge, the work-tree has only one parent. You can also specify the revision ., which is the current commit ... so in that case, they are effectively the same.

How to get a summary of commits in Mercurial

I would like to get a summary of commits with the following information
Number of days worked, start day & end day.
Activity summary by day - just number of commits & number of lines changed.
Is there an extension which does this ?
hg help log + hg help diff + hg help revsets + hg help templating hg help dates + bash
Date of first commit|last commit
Initial commit always has rev 0, latest is always tip
hg log -r 0 --template "{date|date}\n"
hg log -r tip --template "{date|date}\n"
Number of days worked: number of days with non-zero number of commits
hg log --template "{date(date,'%d%m%y')}\n" | sort -u | wc -l
Activity summary by day - just number of commits
hg log -r "date('YYYY-MM-DD')" --template "{.}\n" | wc -l
number of lines changed (first ugly draft iteration: "feci quod potui, faciant meliora potentes")
hg diff --stat -r "first(date('YYYY-MM-DD'))" -r "last(date('YYYY-MM-DD'))"
Sample output of such diff
404.php | 4 +-
comments.php | 14 +-----
footer.php | 2 +-
functions.php | 24 +++++++++-
header.php | 2 +-
readme.txt | 38 +++++++++++++++++
screenshot.png | Bin
search.php | 12 +++-
sidebar.php | 45 ++------------------
style.css | 121 +++++++++++++++++++++++++++----------------------------
10 files changed, 139 insertions(+), 123 deletions(-)
Note: YYYY-MM-DD is placeholder, you have to write real datein this format into command
Note 2: Less than hour for preparing and testing results!!!
LazyBadger's answer gives you a way to find out the dates of first and last commits. For a breakdown of number of daily changesets committed and lines of code changed, enable the churn extension that ships with Mercurial. In either global or repository config:
[extensions]
churn=
Then for a breakdown of number of changesets by day, in chronological order:
$ hg churn --template "{date|shortdate}" --sort --changesets
or for lines of code:
$ hg churn --template "{date|shortdate}" --sort
with optional diffstat to show added/removed lines separately:
$ hg churn --template "{date|shortdate}" --sort --diffstat
Check hg help churn for more options, like limiting range of dates or changesets considered.

Create a new branch, made a lot of changes, how to view list of files changed?

So there was a new branch created where we made some breaking changes to the codebase.
Now we are going to merge, but before that I want to get a list of all the files that were changed in the branch.
How can I get a list of files? I tried:
hg status --change REV
But i'm not sure if that is what I want, since I want all files changed in this branch and not a specific revision in the branch.
BTW, how can I view the revision numbers?
Try with
$ hg status --rev "branch('your-branch')"
to get the changes between the first and the last changeset on the branch (hg status will implicitly use min(branch('your-branch')) and max(branch('your-branch')) when you give it a range of revisions like this).
Since you'll be merging, you should really look at
$ hg status --rev default:your-branch
to see what is changed between the default branch and your-branch. This shows you the modifications done, and filters out any modifications done on the branch due to merges with default.
This is necessary in case your history looks like this:
your-branch: x --- o --- o --- o --- o --- o --- y
/ / /
default: o --- a --- o --- b --- o --- c --- o --- o --- d
where you've already merged default into your branch a couple of times. Merging default into your branch is normal since you want to regularly integrate the latest stuff from that branch to avoid the branches drifting too far away from each other.
But if a new file was introduced on default and later merged up into B, then you don't really want to see that in the hg status output. You will see it if you do
$ hg status --rev a:y
since the file was not present in a, but is present in y. If you do
$ hg status --rev d:y
then you wont see the file in the output, assuming that it's present in both heads.
You write in a comment that you're working Kiln repository. They mean "clone" when they say "branch", but the above can still be adapted for your case. All changesets will be on the default named branch, but that's okay.
Run the following command in your local clone of the "branch" repository:
$ hg bookmark -r tip mybranch
This marks the current tip as the head of mybranch. Then pull all the changesets from the main repository:
$ hg pull https://you.kilnhg.com/Repo/public/Group/Main
You then mark the new tip as the tip of the main repository:
$ hg bookmark -r tip main
You can now run
$ hg status --rev main:mybranch
to see the changes between main and my-branch. If you want to see what you did on the branch itself, the use
$ hg status --rev "::mybranch - ::main"
The ::mybranch part will select changesets that are ancestors of mybranch — this is all your new work, plus old history from before you branched. We remove the old history with - ::main. In older versions of Mercurial, you would use hg log -r -r mybranch:0 -P main.
In cases like this, I prefer to do a test merge from a newly checked-out copy of the repo. This has the advantage that I can see how many conflicts the merge will produce, and I can keep the merge result because I did it in its own copy.
To view the revision numbers, enable the graphlog extension and run:
$ hg log -b your-branch -G
This gives you a nice ASCII graph. This can be handy to quickly look at the graph, but I recommend using TortoiseHg for a cross-platform log viewer:
I had to merge the default branch into my branch to get some fixes, now the commands above shows also files changed because of merges (this files changed after the merge again in the default branch).
Therefore, to get only the correct files I use something like this:
hg log --rev "branch('my-branch') and not merge()" --template '{files}\n' | sed -e 's/ /\n/g' | sort -u
if you have spaces in file names, you can do it this way:
hg log --rev "branch('my-branch') and not merge()" --template '{rev}\0' | xargs -0 -I # hg status -n --change # | sort -u
And to answer your last question, revisions can be shown this way:
hg log --rev "branch('my-branch') and not merge()" --template '{rev}\n'
TIP: I use a hg-alias for this:
[alias]
_lf = ! $HG log --rev "branch(\"$1\") and not merge()" --template '{rev}\0' | xargs -0 -I # hg status -n --change # | sort -u
With mercurial, if you want to get the list of all the files changed in your current branch (changes done of your changeset) you can use these commands: :
hg log --branch $(hg branch) --stat | grep '|' | awk -F\ '{printf ("%s\n", $1)}' | sort -u
Example result:
api/tests/test_my_app.py
docker/run.sh
api/my_app.py
Explanation of the commands:
hg log --branch $(hg branch) --stat
Show revision history of entire repository or files and output diffstat-style summary of changes
hg branch
Show the current branch name
grep '|'
Search for a text pattern, in this case, it is "|"
awk -F\ '{printf ("%s\n", $1)}'
Space separator denotes each field in a record and prints each one in a new line
sort -u
Sort all the printed lines and delete duplicates