I used the BFG-Repo-Cleaner to remove a big unused folder from history and want to push it to a remote non-bare repository.
I closely followed the Instructions on the website (https://rtyley.github.io/bfg-repo-cleaner/) and until the point I try to push it everything works fine. The files are deleted from history and the size of the pack file in my mirrored clone is reduzed to the expected amount.
As soon as I try to git push I get an error saying that branch is currently checked out.
! [remote rejected] dev -> dev (branch is currently checked out)
error: failed to push some refs to 'C:\projects\repositoryname'
This results to two identical branches in the original repository with every commit being duplicate on both branches. There are also multiple Pack files afterwards and they are not reduzed to the size they had in the mirrored bare clone.
Is there any way to push the results of the BFG Repo Cleaning to a non-bare repository successfully without causing any issues?
Related
I am working with Hg and TortoiseHg on a project and pushing every couple of days to a remote repo on Bitbucket. When I tried to push changes today, I got an error saying that I was trying to create a new head. I thought this was odd since I am definitely the only person working on the project and I work from one PC.
I pulled to see what was going on on the remote repo and after pulling the local repo tree looks like so:
At the bitbucket end the repo looks like this:
Can someone help me understand why I got two heads if I'm the only one working on the project and why Hg is not recognising that Rev.40 and Rev.36 are the same revision?
How do I fix this now? If I strip 40 locally, what will happen when I try to push changes to the remote repo? Will it strip the revision at the remote repo too?
you can try this (on a clone repo if you prefer to be sure)
having a clean workgin directory
hg co 40
hg backout -r 40
hg merge 39
hg push
revision 40 would be the one that exists in the remote repo, before the amendment
so, you check it out, you back it out (put the inverse on top of it) then you merge your ongoing work (left at 39) and there should be no merge conflicts at all, since all changes are incoming
then, when satisfied, you push
===
why I got two heads
this part was already addressed in the comments, you realized you amended the commit after pushing it, thus the apparent duplicate of it
How do I fix this now?
you do a merge on your local repo, to get rid of the two heads, so having only one the remote repo won't complain about it
if you like, you can backout the before-amendment commit to be safe (or not, if you really know how to merge the conflicts making your local prevail)
but in both these cases, the non-amended commit survives, therefore it'll show up in your history, beware
If I strip 40 locally, what will happen when I try to push changes to the remote repo?
is gonna be there unless you strip it there (directly on the remote repo)
Will it strip the revision at the remote repo too?
no, it won't
As commit r40 is only present locally, there is no drawback in stripping it from your local repository - IF you are sure that it doesn't contain changes you want to keep. It won't propagate to your bitbucket repo then.
However, the two commits are not the same in the sense of the repo - they probably differ at least by a small time difference (if not some content, e.g. white space) - thus they are unique; only you can tell how they came into existence.
We have a SOLUTION folder (Mercurial repository) in whitch we have a PROJECT folder, that is also a Mercurial repository.
So two repositories: one - the root(solution) folder and other - a subfolder of the root folder(the project) (yes strange but it is like this)...
Everything worked, but one day someone somehow included the SOLUTION branch into the PROJECT repository... So all the history from the Solution branch was included in parralel with the Project branch into the PROJECT repository....
Now is a little mess in the PROJECT repository... There is need to clean that repository...
Locally it worked by applying the hg strip rev XXS (where XXS was the revision number of the very first node from the freshly added Solution branch in the Project repository).
But it seems there is no strip equivalent on the server?!
Every time we'll pull incoming changes in the Project repository, the "Solution" branch will be re-imported....
Is there a way to manage it on the server side?
Of course the same solution would also work on the server. Thus you need login access to the server itself to execute the same local history operation on it. But for the default setup (publishing server) a push will never remove changesets which are present on a remote location; when you history edit your local repository, the changes will not all propagate: only additions to the graph will, but no deletions.
If such changes to the remote server are expected to be pushed, and this is a regular thing, you might want to look into use of phases and how to setup a non-publishing server, e.g. a server with mutable history: Phases#Publishing_Repository.
Mind that such a workflow also means that every single one of the people with push privilige has to change their default phase to 'draft' instead of 'public' - at least for that project.
kill the server repo. start a fresh one, then from local:
hg push -rev XXR
where XXR is the last rev you want to keep.
I have a mercurial repository at c:\Dropbox\code. I've created a clone of this repo locally using:
hg clone -U c:\Dropbox\code c:\GoogleDrive\codeBackup
This bare repo serves the purpose of backup only. I regularly push changes to codeBackup. Furthermore, both the directories are backed-up in the cloud (Dropbox & Google Drive respectively).
If my repo in code becomes corrupt would the codeBackup repo automatically be corrupt since the clone operation used hard links to the original repo? Thus my double-cloud-backup strategy would be useless?
P.S. : I understand that the fall back option is to use the cloud service to restore a previous known good state.
UPDATE : After digging around, I'll add these for reference
Discussion on repo corruption in mercurial
The problem is, if a 'hg clone' was done (without --pull option), then
the destination and the source repo share files inside .hg/store by
using hardlinks 1, if the filesystem provides the hardlinking
feature (NTFS does).
Mercurial is designed to break such hardlinks inside .hg if a commit
or push is done to one of the clones. The prerequisite for this is,
that the Windows API mercurial is using should give a correct answer,
if mercurial is asking "how many hardlinks are on this file?".
We found out that this answer is almost always wrong (always reporting
1, even if it is in fact >1) iff the hg process is running on one
Windows computer and the repository files are on a network share on a
different Windows computer.
To avoid hardlinks (use --pull):
hg clone -U --pull c:\Dropbox\code c:\GoogleDrive\codeBackup
To check for hardlinks:
fsutil hardlink list <file> : Shows all hardlinks for <file>
find . -links +1 : Shows all files with hardlinks > 1
ls -l : shows hardlinks count next to each file
The biggest problem here, regarding repository corruption, is that you're using Dropbox and Google Drive to synchronize repositories across machines.
Don't do that!
This will surely lead to repository corruption unless you can guarantee that:
Your machines will never lose internet connection
You will never have new changes unsynchronized on more than one machine at a time (including times where you have had internet problems)
That Dropbox will always run (variant of never lose internet connection)
You're not just plain unlucky regarding timing
To verify that Dropbox can easily lead to repository corruption, do the following:
Navigate to a folder inside your Dropbox or Google Drive folder and create a Mercurial repository here. Do this on one machine, let's call this machine A.
Add 3 text files to it, with some content (not empty), and commit those 3 text files.
Wait for Dropbox/Google Drive to synchronize all those files onto your second computer, let's call this machine B
Either disconnect the internet on one of the machines, or stop Dropbox/Google Drive on it (doesn't matter which one)
On Machine A, change file 1 and 2, by adding or modifying content in them. On Machine B, change file 2 and 3, making sure to add/modify in some different content from what you did on machine A. Commit all the changes on both machines.
Reconnect to the internet or restart Dropbox/Google Drive, depending on what you did in step 4
Wait for synchronization to complete (Dropbox will show a green checkmark in its tray icon, unsure what Google Drive will display)
Run hg verify in the repositories on both machine A and B
Notice that they are now both corrupt:
D:\Dropbox\Temp\repotest>hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3.txt#?: rev 1 points to unexpected changeset 1
(expected 0)
3.txt#?: 89ab3388d4d1 not in manifests
3 files, 2 changesets, 6 total revisions
1 warnings encountered!
2 integrity errors encountered!
Instead get a free bitbucket or kiln account and use that to push and pull between to synchronize across multiple computers.
The only way you code repository can become corrupt (assuming it was not corrupt when you initially cloned it over to codeBackup) is when you write something to it, be it committing, rewriting history, etc. Whenever something gets written to a hard-linked file, Mercurial first breaks the hard link, creates an independent copy of the file and then only modifies that newly created copy.
So to answer your questions: under normal usage scenarios repository corruption will not propagate to your codeBackup repository.
I am using Tortoise HG two handle source control for a couple of projects on my local machine. I am pushing to a local directory elsewhere on the HD, just as a way to handle revision history.
My directories are set up like so:
Projects are located here: like MyDocuments\Project1.
I push commits to here:
C:\Repository\Project1
C:\Repository\Project2
For one of these projects, this works. For the other, when I try and push, I get an error that it cannot find the directory. Security settings (Windows 7) are the same for both directories.
MyDocuments\Project2 was under this source control at one time, and then something got messed up. So, I deleted the .hg file and hg directory and started over, creating a new repository there and adding all the files for the initial commit. But the initial commit will not push. It says:
repository C:\Repository\Project2 not found
I'm at a loss. I've deleted the HG files and directory three times and started over, but I cannot push. I've tried pushing to a different directory - no luck. I am guessing something is glitched from trying to start over.
You shouldn't delete the .hg folder as that is what makes it a Mercurial repository.
I'd say that the solution would be to re-clone the MyDocuments\Project2 repository to C:\Repository\Project2 and start again from there.
In future, if you get to the position where you feel that you need to delete the .hg folder, don't. Come back here and see if we can do anything to help resolve the problem.
I have a website that I want to deploy to a clients DEV and UAT environments, the site is part of a mercurial repo - it is in the Website folder at the same level as the .hg folder. I know I can push the entire repository but would rather push only the website folder so the client does not have the other files and folders.
The repo looks like this:
Project root
.hg
Database (SQL Source Control uses this)
Documentation (All specs, pdfs, art work etc.)
Lib (pre-Nuget 3rd party dlls)
packages (Nuget stuff)
Website (this is the only area I want to deploy)
.hgignore
Project.sln
Edit:
The clients servers are not connected directly to the internet, my access to them is over a vpn and then RDP. Currently to deploy any changes I need to zip the site up, put it on a shared ftp server then wait up to 3 days for the files to be copied to the servers. Rules have been configured so I can use Mercurial over this connection.
Edit 2
I have managed to create a subrepo from the Website folder by forgetting the Website folder and all it's contents, committing the change then putting the files back, creating a repo then echoing out the .hgsub file. Locally this works for me, I can clone from the Website repo without getting any of the additional folders. However I have not been able to use this version of the repo, even if I repeat the process on our repo server. When I try to clone the hosted version down to my local working copy I get 404 errors, but I can clone the hosted version on the hosting server.
I would appreciate some step-by-step instructions (a guide for dummies if you like) on how to achive my goal; which is to be able to push only the Website folder to the clients servers. The master copy of the repo is on our repo server, I have a local clone and need to be able to push out versions from my copy.
Edit 3
Turns out that the problem I was having converting a folder to a subrepo as described in http://mercurial.aragost.com/kick-start/en/subrepositories/#converting-folder-into-a-subrepository was that the convert command, in versions after 2.1.0, is broken and is still broken in 2.3.1. After I figured that out and rolled back to that version of TortoiseHg I was able to convert the folder to a subrepo, in the root of the repo I have .hgsub which says Website = Website. I was able to work with that locally, commit to the whole repo, the subrepo, clone either the full repo or the subrepo (which is what I want), however I can't get this to work from our master repo server.
I zipped the whole thing up and ftp'd it to our remote master repo server, then set it up so I could clone from it. Directly on the server this works fine (hg clone --verbose -- C:\Repositories\EM .), however when I try to clone from the server to my local development machine with (hg clone --verbose -- https://myserver.com/hg/EM/ .) it fails with "HTTP Error: 404 (Not Found)".
requesting all changes
adding changesets
adding manifests
adding file changes
added 628 changesets with 6002 changes to 4326 files
updating to branch default
resolving manifests
calling hook preupdate.eol: <function preupdate at 0x00000000035204A8>
getting .hgignore
getting .hgsub
getting .hgsubstate
HTTP Error: 404 (Not Found)
[command returned code 255 Fri Apr 20 10:51:23 2012]
I don't know what the problem is, the files are there so why the 404?
In my opinion Mercurial shouldn't be used for this purpose. This is particularly true if that website is a web application because you shouldn't have the DLLs in Mercurial.
You should look at the web deployment tool built into Visual Studio. Have a look at this page to see if it suits your purpose.
If you can't install the required services on the destination server then it can be configured to use FTP instead.
You can not push part of repo tree
If DEV and UAT environments are unversioned targets, you can use any other way for distributing Mercurial content
You can separate Website into subrepo and will be able to push this repo
As others have pointed out you can't use push for this. Just do 'rsync' from your server to theirs. You could even automated that in a hook, where you push to a local repository and it auto-deploys to their site. Something like:
[hooks]
changegroup.deploy = $HG update ; rsync Website account#theirserver:/path/to/docroot
I have a working solution to this. I created a batch file that creates an outgoing repo and starts the built in server so I can pull from it on the client machines. First it clears out the previous folder, then clones from my local working copy (there's a parameter to determine which tag it should clone from). Next it creates a map file and converts the Website folder to a new Website2 folder in order to preserve the history then gets rid of the original folder and renames the new one. Finally it spins up the built in server.
cd c:\inetpub\wwwroot
rd /S /Q _ProjectName
hg clone -- C:\inetpub\wwwroot\ProjectName#%1 C:\inetpub\wwwroot\_ProjectName
cd c:\inetpub\wwwroot\_ProjectName
echo include Website > map.txt
echo rename Website . >> map.txt
hg --config extensions.hgext.convert= convert --filemap map.txt . Website2
cd Website2
hg update
cd ..
hg remove Website/*
hg commit -m "Removed Website"
rename Website2 Website
hg serve
So it isn't pretty, but now I just need to call the batch file and pass the tag I want to build the outgoing website from (uat, dev etc.) and give it a minute to create my Website folder, with history, that I can use to pull from or push from. I don't need to call hg serve because I know the names of the client servers so I can push the changeset out by creating aliased remote repositories. But I included that step so the client machines can pull. I haven't fully explored this option, so I'm not sure whether it's got any particular advantage. It's fine for the case when it's just me working on the project, but if any other developer needs to work on this then the Uri for their local project server will obviously be different (http://SIMON-PC:8000/ won't be the case for everyone), in which case pushing into the client might be best.
But by using this approach my local working repo doesn't need to change and so I don't get any issues communicating with our central repo, the 404 errors mentioned in edit3. I keep the entire history of the repo with the convert process, so the next time I need to send changes I'm not starting at revision 1 - in other words it isn't destructive of the Website and although I am deleting the entire outgoing repo (_ProjectName) each time I am retaining the history and yet in a position to pull / push ONLY the Website directory because it is created each time as a 'standalone' repo