Convert largefiles to normal files - mercurial

I have a repository where the trunk does not contain any largefiles. In my working branch, I previously added and committed some binaries as largefiles. Now, I don't want them to be largefiles anymore. I haven't merged into trunk yet.
I've tried removing the files entirely, then committing, then adding the files using hg add --normal, but this results in my diff containing both the standins in .hglf as well as the binary files. Is there any way to get rid of the standins without making a new branch and grafting changes?

I ended up creating a new branch off of the ancestor I had originally branched off from, then merging in my messed up branch. Before committing this merge, I excluded all of the largefiles in question. Then, I created a new commit, committing the largefiles as normal files using the --normal flag.
When I creating a pull request to the ancestor with this new branch, no standin files appeared in the diff and thus no largefiles were merged in, while preserving the commit history from my messed up branch.

Related

How to strip and rebuild an existing branch and push it to the main repository? (aka replace a branch with a new one)

THE CONTEXT:
There are three branches: "mine" and "yours" and "main". I started with a clean clone and the only requirement is to make the "mine" branch like the "main" branch with only a few added changes. I do not want to alter the other branches. The current "mine" branch has only one changeset that I would prefer to delete and replace with a new changeset.
MY ATTEMPT:
"mine" contains only one changeset, which I want to discard. So, I stripped this changeset(effectively deleting "mine"), updated directory to "main" branch, recreated "mine" ("hg branch mine"), added my changes, committed, and now I want to push. The output of "hg push -b mine" is "abort: push creates new remote head..." and references my new changeset. However, I just want to replace my new changeset with my old (locally deleted) changeset. "hg outgoing" lists my new changeset, "hg incoming" lists my old changeset.
I think I am very close, any suggestions? Would a forced push work here? Alternative solutions and helpful references are welcome.
After my attempt, which involved stripping the changesets that I did not want from the "mine" branch and then committing my new changesets on the "mine" branch, here is my solution:
I resolved this issue with a forced push "hg push -f" which thus created two heads on the "mine" branch (in the main repository), and then stripped the extra head using the "hg strip" extension. This was done from the main repository after I had pushed, which may not be perfect, but it worked for these purposes.
The forced push was needed to add my new changesets (the new "mine" branch) and the strip (which had to be done after the push, on the main repository) deleted the old "mine" branch. It turns out that stripping the "mine" branch in my local repository doesn't make a difference because the stripped changesets are still present on the main repository (thus the need to strip them after the push).
If there is a better solution (or a better way to phrase the question), I am curious to find out.

How to remove largefiles from Mercurial repo

See also this question.
Without knowing what I was doing, I enabled the largefiles extension, committed a file and pushed it to kiln. Now I know the error of my ways, and I need to permanently revert this change.
I followed the guidance from SO on the subject; and I can remove largefiles locally, but this doesn't affect the remote repos in kiln. I have tried opening the repo in KilnRepositories on the Kiln server and nuking the largefiles folder (as well as deleting 'largefiles' from the requires file), but after a few pushes/pulls the folder and the require's line come back.
Is there a way to make this permanent? (Setting requires to readonly doesn't work either).
Note: This is true at least of (for Windows) TortoiseHg 2.4 (Mercurial 2.2.1) -> 2.7.1 (Mercurial 2.5.2). I will not speak for future or older versions.
After looking through the various mercurial extensions available, I have concluded that it is generally not possible to convert a repository 'back' once a file has been committed using the largefiles extension.
First, the two rationales for why you do not want largefiles in play on your repos: one and two.
Once a file has has been committed as a largefile, to remove it, all references to the '.hglf' path must be removed from the repo. A backout isn't sufficient, as its commit contents will reference the path of the file, including the '.hglf' folder. Once mercurial sees this, it will write 'largefiles' back to the /.hg/requires file and the repo is once more largefile locked. Likewise with hg forget and remove.
Option 1: If your repo is in isolation (you have end-to-end control of the repo in all of its local and remote locations AND no one else has branched from this repo), it may be possible to use the mq extension and strip the changeset. This is probably only a viable option if you have caught the mistake in time.
Option 2: If the offending changeset (the largefile commit) exists on a commit phase that is draft (or that can be forced back into draft), then it may be possible to import the commit to mq and unapply the changeset using hg qpop. This is superior to stripping, because it preserves the commit history forward from the extracted changeset. In real life, this is frequently not possible, because you've likely already performed merges and pushed/pulled from public phase branches. However, if caught soon enough, mq can offer a way to salvage the repo.
Option 3: If the offending changeset is referenced in one and only one place (the original commit), and no one has attempted to backout/remove/forget the changset (thus creating multiple references), it may be possible to use hg rebase, to fold the first child changeset after the offense with the parent changeset of the offense. In doing so, the offensive changeset becomes a new head which can then be stripped off with mq strip. This can work where attempts to import to mq have failed.
Option 4: If none of the above work, you can use transplant or graft, to export all of the non-offending changesets as patches (careful to export them in the correct sequence), then hg update to the first sane changeset before the offense, mq strip the repo of all forward changesets and then reapply your exported patches in sequence.
Option 5: (What I ultimately did). Clone the repo locally, so that you have two copies: clone_to_keep, clone_to_destroy. In clone_to_keep, update to the first sane changeset before the offense. Mq strip all forward changesets. Merge back down if left with multiple heads. In clone_to_destroy, update to the tip. In windows explorer, copy everything in /clone_to_destroy except the .hg and .hglf folders to the /clone_to_keep folder. Inside Tortoise, commit all changes in clone_to_keep as a single changeset. Preserve one remote instance of clone_to_destroy in a readonly state for historical purposes, and destroy all others.
Option 6: The nuclear option. If all else fails, and if you don't care about the integration of your repo with external systems (bug tracking, CI, etc), you can follow the aforementioned SO post and use the hg convert extension. This will create a new copy of the infected repo, removing all references to the offending changesets; however, it does so by iterating each changeset in the entire repo and committing it to the new repo as a NEW changeset. This creates a repo which is incompatible with any existing branch repos--none of the changeset ids will line up. Unless you have no branch repos, this option will probably never work.
In all cases, you then have to take your fix and manually reapply to each distinct repository instance (copy the repo folder, clone, whatever your preferred method).
In the end, it turns out that enabling largefiles is an extremely expensive mistake to make. It's time consuming and ultimately destructive to fix. I don't recommend ever allowing largefiles to make it into your repos.

Mercurial: How can I create an unplanned branch

My use case is this:
I am working on a new feature and I have several commits on that feature.
Since it was a minor feature, I didn't even consider doing the feature in a feature branch.
However. Now my boss comes along and tells me to fix a bug on the same branch that I am working on (default).
To fix that I'd like to create a feature branch for my feature, push all my existing (unpushed) commits into that branch.
So I'd like to create a branch just before my first commit and then somehow move all my commits to that branch.
How can I do this?
There’s two ways to approach this, depending on your preference:
In a new repository.
Make a new clone of your repository, and do the bug fix you need to make there. Then push it to the main repository when you’re done, and continue where you left off in the original repository. Pull and merge to get the new changes as usual.
In the existing repository.
Update to the changeset before your local changes, and just start fixing and committing there. This creates a new anonymous branch. When you’re done, push using push -r ., this will only push the changes that are included in the working copy. After this, merge with your original branch (hg merge) and continue where you left off.
Note that you can bookmark the feature branch with hg bookmark if you do not feel comfortable with leaving your changes unlabeled. Also you can easily find back any heads you left behind using hg heads.
Personally I prefer to work in a new clean clone, as you don’t need to worry about branching and where to leave uncommitted changes. However if your project setup is complicated it may be more convenient to reuse the existing repo.
For this situation you can fix it by rebasing (which may need enabling in your configuration).
On your branch, update to the revision before the change-sets you want to move:
hg up -r<revison>
This assumes contiguous revisions need moving.
Create a new branch:
hg branch "TempWork"
Put a dummy commit onto it in order to get a new revision:
hg commit -m"New Branch"
Then perform the rebase from the first of the change-sets you want to move (it moves descendants automatically) and specify the new branch revision as the destination:
hg rebase -s<base revision> -d<new branch revision>
Then update back onto your main-line branch.
Fourth method: Using mq-patches
You have to have mq extension enabled and initiated for repo
On hotfix moment you
convert feature-commits into set of mq-patches (hg qimport)
Unapply all patches in set (hg qpop -a)
Code hotfix, commit
...
Finish and test hotfix on clean codebase
Apply all patches in set (hg qpush -a), fix possible conflicts
Convert patches back to changeset (hg qfinish)

Continuously Fixing Same Mercurial Merge Issues

My co-worker and I are working on a project and we are using mercurial + bitbucket.
We are experiencing problems when one of us pulls from bitbucket and merge. For instance, after I pull from bitbucket with hg pull --rebase, I have to
remove files that were removed in previous commits
modify files that were modified in previous commits
move files/folders that were moved in previous commits
He does hg pull followed with hg merge and gets the same results as me.
What are we doing wrong?
Besides rebase/merge, we work the same
work, work, work
pull rebase/merge
fix merge issues
push
The most likely cause is that you're rebasing changesets that have already been pushed. Rebase modifies history, and unless you really know what you're doing you should only rebase changesets that have never been shared to another repository.
If you are using Mercurial 2.1 or later it has support for phases. When phases are working rebase will only allow rebasing of draft and secret changesets, not public ones.
In general, I'd advise you to use merging rather than rebasing.

Can I commit changes to multiple mercurial subrepos to a new named branch?

I have a mercurial repository with multiple sub repositories inside it. The repository has a visual studio solution containing projects in the repository and the subrepository.
Suppose I want to implement a new feature that will require changes to the main project in the repository and changes to one of it's dependency projects in the solution (let's say, adding a new shared interface in the dependency, and an implementation of the interface in the main project).
I then want to commit the changes, but to a new named branch as it is unfinished and will be merged in later. Using tortoiseHg I commit the changes in the repository, specifying a new branch to create. The commit in turn commits the changes of the sub repo, but in my tests it doesn't create a new branch in it's repository, but just adds the changeset to the current branch.
I can perform the commit to the sub repo explicitly and specify the branch name at that time but I was hoping for a way to commit the whole change set across repositories to new branches in each repository on one go so my workflow is cleaner. Is this possible?
First commit the sub-repo and create a named branch there.
Afterwards the main commit should continue to commit to that sub-repo in the same branch as before.
First you should think of the two repositories as separate, completely separate. In other words, you do something in one repository and commit that. Then you do something in the other repository, and commit that.
The relationship the two repositories have is that the main repository stores a file, the .hgsubstate file, which contains the hash of the parent of the working folder of each sub-repository.
In other words, the sub-repository has no knowledge in itself that it is being part of a bigger picture. However, the main repository knows the current checked out revision in that sub-repository. This knowledge is committed into the repository as part of normal commits.
This means that if you do something in the sub-repository, commit that, the sub-repository is now at a new revision. When you later commit in the main repository, the hash of that new revision in the sub-repository is updated into the .hgsubstate file, and then committed.
The purpose behind this is of course that if you update to an older revision in the main repository, an old copy of the .hgsubstate file is brought into the working folder as well, and then the sub-repository is then updated to that revision, which has the effect of turning back the clock to how the sub-repository looked like when those revisions where in play.
Additionally, commands on the main repository sometimes work on the sub-repositories as well. If you push in the main repository, the sub-repositories are pushed as well, to ensure that others who clone your main repository can also safely rely on being able to clone the appropriate content for the sub-repository.
So to answer your question in the comments.
If you create a branch in the sub-repository, named or otherwise, you will keep committing new changesets down that branch. At some point you need to merge, and you need to merge in the sub-repository as well as in the main repository.
First you would merge in the sub-repository. This ends with a commit, which means the sub-repository is now at the changeset that committed the merge. Then you merge in the main repository and commit, which stores that knowledge, which changeset in the sub-repository that is in use right now.
So yes, you need to merge in both as well, at a later point.