I am trying to clone this project with mercurial:
https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver
I have downloaded TortoiseHg and selected clone repository.
It gives 404 error:
% hg clone --verbose https://hg.mozilla.org/mozilla-central/file/tip/testing/geckodriver "D:\Projects\gecko"
HTTP Error: 404 (Not Found)
[command returned code 255 Mon Oct 17 23:12:18 2022]
What is the correct path I should be trying to clone?
The URL used everywhere is just the URL of page in some Web-frontend for the some repository in some state, loosely connected to real URL of repository
I don't know the internal structure of Mozilla Central (they can use subrepos for separate projects or have one giant monorepo), but by inspecting changesets, which affected geckodriver dir in tree, I see modified files with path testing/geckodriver/ in names, based on which I assume that the project of interest is not an independent entity, but only a subtree in a large repository. Thus, because with HG you can't clone just part of tree (contrary to SVN, f.e.), you, maybe, have to clone the whole Mozilla Central repository in worst case
PS: hg clone -v https://hg.mozilla.org/mozilla-central/testing/geckodriver will not work, don't waste time on it
PPS: If I grok docs correctly (NB: it's not guaranteed!!!) you can enable (still experimental, but bundled for years) sparse extension, prepare sparse-config and try to clone needed subtree testing/geckodriver (it will be not dumb hg clone, but hg init, hack-hack hgrc, hg pull https://hg.mozilla.org/mozilla-central in order to have limiting sparse-profile in game on getting sources)
Related
I have a mercurial repository with a clone on several computers. In particular, I have a Ubuntu computer where the repository resides in ~/.vim and a Windows computer with a related repository in C:\Users\ben\vimfiles.
This repository contains subrepositories, some of them git subrepositories. For example, from .hgsub:
pack/thirdparty/start/signify = [git]pack/thirdparty/start/signify
I wanted to merge changes made in each repository. So I cloned the repository from my Ubuntu computer to a USB stick, plugged it into my Windows computer, and pulled from the repository on the Windows computer into the clone on the USB stick. So far, so good.
Now I go to merge, or even just update to any of the versions coming from the Windows computer. I got an error like this:
pulling subrepo pack/thirdparty/start/signify from /home/ben/.vim/pack
/thirdparty/start/signify
fatal: 'C:/Program Files/Git/home/ben/.vim/pack/thirdparty/start/signify' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
abort: git fetch error 128 in pack/thirdparty/start/signify (in subrepo pack/thirdparty/start/signify)
[command returned code 255 Thu Dec 13 11:24:15 2018]
After a bunch of digging, I eventually solved the issue by manually editing the .git/config file in every single git subrepo to change the origin path from /home/ben/.vim to C:/Users/ben/vimfiles.
What's the correct way to do this using only Mercurial commands or configurations? Using TortoiseHg, when I update (but not merge!) I can select any of the saved paths to pull subrepos from. This seems to manually override the paths.default configuration for the pull operation. But, that setting seems to be ignored by the git subrepos.
From the mercurial's documentation on subrepositories (accessible on command line through hg help subrepos):
Subrepositories
...
Remapping Subrepositories Sources
A subrepository source location may change during a project life,
invalidating references stored in the parent repository history. To
fix this, rewriting rules can be defined in parent repository hgrc
file or in Mercurial configuration. See the [subpaths] section in
hgrc(5) for more details.
Documentation on subpaths:
subpaths
Subrepository source URLs can go stale if a remote server changes name
or becomes temporarily unavailable. This section lets you define
rewrite rules of the form:
<pattern> = <replacement>
where pattern is a regular expression matching a subrepository source
URL and replacement is the replacement string used to rewrite it.
Groups can be matched in pattern and referenced in replacements. For
instance:
http://server/(.*)-hg/ = http://hg.server/\1/
rewrites http://server/foo-hg/ into http://hg.server/foo/.
Relative subrepository paths are first made absolute, and the rewrite
rules are then applied on the full (absolute) path. If pattern doesn't
match the full path, an attempt is made to apply it on the relative
path alone. The rules are applied in definition order.
I have a website that I want to deploy to a clients DEV and UAT environments, the site is part of a mercurial repo - it is in the Website folder at the same level as the .hg folder. I know I can push the entire repository but would rather push only the website folder so the client does not have the other files and folders.
The repo looks like this:
Project root
.hg
Database (SQL Source Control uses this)
Documentation (All specs, pdfs, art work etc.)
Lib (pre-Nuget 3rd party dlls)
packages (Nuget stuff)
Website (this is the only area I want to deploy)
.hgignore
Project.sln
Edit:
The clients servers are not connected directly to the internet, my access to them is over a vpn and then RDP. Currently to deploy any changes I need to zip the site up, put it on a shared ftp server then wait up to 3 days for the files to be copied to the servers. Rules have been configured so I can use Mercurial over this connection.
Edit 2
I have managed to create a subrepo from the Website folder by forgetting the Website folder and all it's contents, committing the change then putting the files back, creating a repo then echoing out the .hgsub file. Locally this works for me, I can clone from the Website repo without getting any of the additional folders. However I have not been able to use this version of the repo, even if I repeat the process on our repo server. When I try to clone the hosted version down to my local working copy I get 404 errors, but I can clone the hosted version on the hosting server.
I would appreciate some step-by-step instructions (a guide for dummies if you like) on how to achive my goal; which is to be able to push only the Website folder to the clients servers. The master copy of the repo is on our repo server, I have a local clone and need to be able to push out versions from my copy.
Edit 3
Turns out that the problem I was having converting a folder to a subrepo as described in http://mercurial.aragost.com/kick-start/en/subrepositories/#converting-folder-into-a-subrepository was that the convert command, in versions after 2.1.0, is broken and is still broken in 2.3.1. After I figured that out and rolled back to that version of TortoiseHg I was able to convert the folder to a subrepo, in the root of the repo I have .hgsub which says Website = Website. I was able to work with that locally, commit to the whole repo, the subrepo, clone either the full repo or the subrepo (which is what I want), however I can't get this to work from our master repo server.
I zipped the whole thing up and ftp'd it to our remote master repo server, then set it up so I could clone from it. Directly on the server this works fine (hg clone --verbose -- C:\Repositories\EM .), however when I try to clone from the server to my local development machine with (hg clone --verbose -- https://myserver.com/hg/EM/ .) it fails with "HTTP Error: 404 (Not Found)".
requesting all changes
adding changesets
adding manifests
adding file changes
added 628 changesets with 6002 changes to 4326 files
updating to branch default
resolving manifests
calling hook preupdate.eol: <function preupdate at 0x00000000035204A8>
getting .hgignore
getting .hgsub
getting .hgsubstate
HTTP Error: 404 (Not Found)
[command returned code 255 Fri Apr 20 10:51:23 2012]
I don't know what the problem is, the files are there so why the 404?
In my opinion Mercurial shouldn't be used for this purpose. This is particularly true if that website is a web application because you shouldn't have the DLLs in Mercurial.
You should look at the web deployment tool built into Visual Studio. Have a look at this page to see if it suits your purpose.
If you can't install the required services on the destination server then it can be configured to use FTP instead.
You can not push part of repo tree
If DEV and UAT environments are unversioned targets, you can use any other way for distributing Mercurial content
You can separate Website into subrepo and will be able to push this repo
As others have pointed out you can't use push for this. Just do 'rsync' from your server to theirs. You could even automated that in a hook, where you push to a local repository and it auto-deploys to their site. Something like:
[hooks]
changegroup.deploy = $HG update ; rsync Website account#theirserver:/path/to/docroot
I have a working solution to this. I created a batch file that creates an outgoing repo and starts the built in server so I can pull from it on the client machines. First it clears out the previous folder, then clones from my local working copy (there's a parameter to determine which tag it should clone from). Next it creates a map file and converts the Website folder to a new Website2 folder in order to preserve the history then gets rid of the original folder and renames the new one. Finally it spins up the built in server.
cd c:\inetpub\wwwroot
rd /S /Q _ProjectName
hg clone -- C:\inetpub\wwwroot\ProjectName#%1 C:\inetpub\wwwroot\_ProjectName
cd c:\inetpub\wwwroot\_ProjectName
echo include Website > map.txt
echo rename Website . >> map.txt
hg --config extensions.hgext.convert= convert --filemap map.txt . Website2
cd Website2
hg update
cd ..
hg remove Website/*
hg commit -m "Removed Website"
rename Website2 Website
hg serve
So it isn't pretty, but now I just need to call the batch file and pass the tag I want to build the outgoing website from (uat, dev etc.) and give it a minute to create my Website folder, with history, that I can use to pull from or push from. I don't need to call hg serve because I know the names of the client servers so I can push the changeset out by creating aliased remote repositories. But I included that step so the client machines can pull. I haven't fully explored this option, so I'm not sure whether it's got any particular advantage. It's fine for the case when it's just me working on the project, but if any other developer needs to work on this then the Uri for their local project server will obviously be different (http://SIMON-PC:8000/ won't be the case for everyone), in which case pushing into the client might be best.
But by using this approach my local working repo doesn't need to change and so I don't get any issues communicating with our central repo, the 404 errors mentioned in edit3. I keep the entire history of the repo with the convert process, so the next time I need to send changes I'm not starting at revision 1 - in other words it isn't destructive of the Website and although I am deleting the entire outgoing repo (_ProjectName) each time I am retaining the history and yet in a position to pull / push ONLY the Website directory because it is created each time as a 'standalone' repo
I have been trying to follow the instructions in the answer to this question, using kiln.
i'd like to be able to arrange things as follows:
/somepath/thirdparty maps to a kiln repository "thirdparty" and contains assorted code
/somepath/common maps to a kiln repository "common" and contains shared code i have written
and
/somepath/project1 maps to kiln repository "project1"
/somepath/project1/thirdparty maps to branch of thirdparty above
/somepath/project1/common maps to branch of common above
and
/somepath/project2 maps to kiln repository "project1"
/somepath/project2/thirdparty maps to another branch of thirdparty above
/somepath/project2/common maps to another branch of common above
I found that when I created the .hgsub file as instructed and added/pushed it to Kiln, I could no longer view the Kiln files in the Kiln web file viewer — it displayed an obscure message about the Kiln "overheating" :-) Additionally, whilst it did automatically create the subfolders in the correct place, they were not populated with files, (possibly because the pull failed).
Anybody tried anything like this before, using Kiln?
As I intend to develop a number of apps using the common code (and potentially eventually release the library as open source), I would like to have it managed in discrete repositories. As some of the projects are for end clients however, I need to be able to give them a single repository that includes things as described above.
Kiln does currently not support subrepos that use nested URLs on the server. This means that you cannot have both the following URLs working:
http://server/kiln/somepath/project1
http://server/kiln/somepath/project1/thirdparty
So you should setup Kiln so that you have four repositories on the server:
http://server/kiln/somepath/project1
http://server/kiln/somepath/project2
http://server/kiln/somepath/thirdparty
http://server/kiln/somepath/common
That's easy — just four normal repositories. Then clone "project" and create the .hgsub file with:
thirdparty = http://server/kiln/somepath/thirdparty
common = http://server/kiln/somepath/common
When you push that back to Kiln, it will notice and display links for the subrepositories. However, the subrepositories wont end up being nested on the server. So there wont be any project1/thirdparty path on the server.
It's also far from clear that you would want that. When you have several projects that collaborate and use some common code base, then you want "project1" and "project2" to get each other's changes to this common code base. So it very useful that the common subrepo in both projects push and pull from http://server/kiln/somepath/common.
In Mercurial, we normally recommend that you use paths of the form common = common in the .hgsub file. This means that the server must support nested repositories. When Kiln doesn't support nested repos, you can use full paths instead.
When you initially setup the subrepositories, then remember that you need to update them manually. So with the above URLs, you would setup "project1" by running:
$ hg clone http://server/kiln/somepath/project1
$ echo "common = http://server/kiln/somepath/common" > .hgsub
$ echo "thirdparty = http://server/kiln/somepath/thirdparty" > .hgsub
$ hg commit -m "Created subrepos"
This creates initial empty subrepositories. They are empty because you haven't told Mercurial which changeset you need in them. This is tracked in .hgsubstate where you'll find:
0000000000000000000000000000000000000000 common
0000000000000000000000000000000000000000 thirdparty
To populate the subrepositories you do
$ cd common
$ hg pull --update
$ cd ../thirdparty
$ hg pull --update
$ cd ..
$ hg commit -m "Updated subrepos"
This updates the 000... lines in .hgsubstate with the current tip changeset IDs for the two subrepos. Future clones of "project1" will notice the .hgsubstate file and make sure to update the subrepos to the revision mentioned there.
I just did hg pull on a repository and brought in some changesets. It said to run hg update, so I did. Unfortunately, when I did that, it failed with the following error message:
abort: integrity check failed on 00manifest.i:173!
When I run hg verify, it tells me there are a number of issues with things not in the manifest (with some slight path obscuring):
>hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
somewhere1/file1.aspx#172: in changeset but not in manifest
somewhere2/file1.pdf#170: in changeset but not in manifest checking files
file3.csproj#172: ee005cae8058 not in manifests
somewhere2/file1.pdf#171: 00371c8b9d95 not in manifests
somewhere3/file1.ascx#170: 5c921d9bf620 not in manifests
somewhere4/file1.ascx#172: 23acbd0efd3a not in manifests
somewhere5/file1.aspx#170: ce48ed795067 not in manifests
somewhere5/file2.aspx#171: 15d13df4206f not in manifests
1328 files, 174 changesets, 3182 total revisions
8 integrity errors encountered!
(first damaged changeset appears to be 170)
The source repository passes hg verify just fine.
Is there any way to recover from an integrity check failure or do I need to re-clone the repository completely from the source (not a huge issue in this case)? What could I have done to cause this, so I don't do it again?
Well, since the first damaged changeset is 170, you could clone your local repository to 169 and then pull from the source. That means only pulling 5 changesets.
hg clone -r 169 damagedrepo fixedrepo
cd fixedreop
hg verify
And then:
hg pull originalsource
As for manual recovery of repository corruption, this page expounds on that better than I can. See section 4:
I have found corruption once in a while before, and although the above
documentation says it is usually from user error, my instances were on
removable USB drives with empty working directories. Sometimes things
just don't get written correctly or are interfered with somehow: it's
not always user error. But I always have multiple copies I can reclone
from so I've been able to get away with basic fixing.
If the simple fix of a partial local clone and pulling from the server doesn't fix it, you're down to 2 options after backing up your changes (if any) to a bundle or patches:
Manually hacking at Mercurial's files.
Doing a new full clone from the server. Usually the easier and faster of the two.
Beware: This method will change all hashes.
Actually there is another way to recover the repository when it is corrupted like this -
You can do a complete rebuild of the repository by using the convert extension. See Section 4.5 on https://www.mercurial-scm.org/wiki/RepositoryCorruption#Recovery_using_convert_extension
First enable the convert extension by adding the following to your ~/.hgrc file
[extensions]
convert=
Then convert the bad repo to create a fixed repo:
$ hg convert --config convert.hg.ignoreerrors=True REPO REPOFIX
This worked for me when I had the experience of suddenly finding that there were missing files in the manifests - "error 255".
Try remove your file 00manifest.i from repo and next use hg remove 00manifest.i and hg commit commands. Worked for me.
What we ended up doing was making a new copy of our 'central' repository, deleting the .hg folder in this copy, creating a new repository there (hg init), and then working with this as the central repository.
Be aware however this is only an appropriate solution if you don't need your changeset history other than as a reference (which we don't). You can still use your old central repository for this purpose.
I created a repository on a remote machine using:
hg init
hg add
hg commit
The repository was created.
I cloned the repository on a local machine with no errors reported; The files seem to be there
Now I'm trying to make a clone of the clone (as a working copy) using:
hg clone "path to original clone"
It returns:
destination directory: "name of repository"
abort: No such file or directory: "path to original clone"/.hg/store/lock
What am I doing wrong?
Thanks
What filesystem is used on the partition where the main repository is ?
Actually, when Mercurial is doing some operations, it needs to lock the repository. For doing this it creates a symbolic link to an nonexistent file, when the filesystem supports it, in the .hg repository, telling every other processes that the repository can't be modified at this time. When symbolic links aren't supported by the filesystem, a normal file is created.
However, there's some problems with some FUSE filesystems, typically SSHFS with the follow_symlinks option activated. FUSE reports that he knows about symbolic links, but since SSHFS follows the symbolic link and the file doesn't exist, the "state" of the link is marked as unknown thus Mercurial thinks the repository isn't correctly locked and abort the operation.
I see you're using Cygwin, so maybe it's the same kind of problem with tools designed for UNIX on a windows filesystem. It's a strange, coworkers of mine are using Mercurial via Cygwin just fine.
I don't know if it's the case for you, but I lost nearly half a day on this problem. Maybe this answers can help some people in the future.
Please paste in the actual command that's failing and the output including the actual path to the clone that you're cloning. When you do the clone use --debug and --traceback too.
As a workaround you can can always try hg init newclone followed by hg pull -R newclone pathtooriginalclone, which is effectively equivalent except it doesn't use local hardlinks when possible.