I have a local repository, let's call it ONE. ONE is the actual program. It's an android program, in case it matters for some reason.
I have a remote repository, let's call it EXT. EXT is somewhat a library, used by ONE.
ONE has a complex directory structure, mandated by android. The main sources are in src/bla/bla/ONE. Since ONE uses EXT, to do it I had to create another directory next to that one, that is src/bla/bla/EXT.
I think would like to keep them separated in two repositories, but I need for them to actually be in this same directory structure to compile ONE.
At the moment I just created a symlink to do it, but I wonder if there is a better way of doing that, that uses some hg feature.
Subrepositories are great for this. Take a look at this related SO question: (how do I add a subrepo to an existing repo in mercurial.
I'm no expert on this, but I don't think sub-repositories work in this case.
You have 2 projects with the same deeply nested directory structure:
Project "ONE":
ONE
/src
/bla
/bla
/ONE
Project "EXT"
EXT
/src
/bla
/bla
/EXT
When you compile these projects you want the following structure:
Compile Project
/src
/bla
/bla
/ONE
/EXT
Or something similar - essentially both source trees combined under a single "src".
Since you can't checkout part of a repository, wherever you create a sub-repository you'll get the full "EXT" directory. So, if you make a subrepo next to "ONE" you'll end up with:
Combined Project
/src
/bla
/bla
/ONE
/src
/bla
/bla
/EXT
What you are after is a "Partial Clone", which doesn't exist yet.
https://www.mercurial-scm.org/wiki/PartialClone
I think OS links are the way to go.
Hope this helps.
Use hg subrepos. For example:
$ git init ONE-proj
$ cd ONE-proj
$ mkdir -p src/bla/bla/ONE
$ ... # commit your initial project files for ONE
$ echo src/bla/bla/EXT = /path/to/hg/repository/EXT > .hgsub
$ hg add .hgsub
$ hg clone /path/to/hg/repository/EXT src/bla/bla/EXT
$ hg commit
Related
Over time a number of the developers have committed files that were then added to the .hgignore. From what I hear there is no way to remove items from the history of mercurial, which is ok. But I also heard that there is a way to do a clone, I think using the convert plugin, to clone/export a repo while specifying which files to not include in the conversion.
I can't help but think that someone out there has a script that does this export/filter/convert using the patterns from the .hgignore file.
Has anyone created such a beast?
You could create a filemap from .hgignore doing something like this:
hg clone -U yourrepo temprepo # create a temp repo with no files in working dir
cd tmprepo
hg revert --all # put files in working dir
hg forget ** # un-add the files
hg status --ignored --no-status | sed 's/^/exclude /' > ../filemap
that will get you a filemap you can pass into hg convert that removes all the added files that would be ignored given your .hgignore.
Do understand though, that running convert creates a whole new repo that is unrelated to your previous repo. All existing clones will be unusable with the new one. It's not normally worth it.
hg convert is indeed the thing you want to use.
You will want to create a file map (just a text file) which will list all of the things you either want to include, exclude, or rename:
include subfolder
exclude subfolder/supersub
etc...
Read the following for a more concrete example:
https://www.mercurial-scm.org/wiki/ConvertExtension#A--filemap
Once you have created this file you will just use the following command:
$ hg convert --filemap my_file_map /path/to/source/repo /path/to/dest/repo
The source repo will not be modified and a dest repo will be created. I don't want to just copy verbatim what the documentation already says so here is the link:
How to keep just a subdirectory (or run on the mercurial repo):
https://www.mercurial-scm.org/wiki/ConvertExtension#Converting_from_Mercurial
For a while I had just my code under source control, but I decided it would be cool to put things like my stuff in my .vim folder among other things in a repo. (I'm obviously not going to store everything in my repo, just various config files, my src directory, and maybe a few other things as well)
I was able to set up a repo fine, then push it to my server where I can access it from my other computers, but I can't clone it to my other computers. When I try to clone it it fails because the home directory isn't empty. Is there a way to do what I want here?
Since the versioned files between my computers are the same, what I did was:
~$ hg clone ssh://myserver/hg/dotfiles mydotfiles
~$ mv mydotfiles/.hg .
~$ rm -rf mydotfiles
and that's it, now your home folder is under version control, but of course if your dot files are not identical between computers you will have to do something about it.
Since I only want to version some files and not everything under my home folder, I have this simple rule in ~/.hgignore
# This .hgignore is for the dotfiles repository only,
# the rest of my HG repositories use the file
# .hgignore_global as referenced by [ui]'s ignore setting.
syntax:glob
*
This way I don't get an ocean of files when I do hg status and only see those files that I have under version control that have been modified.
But since I want to see unversioned files when working within another hg repository, I have this in my ~/.hgrc file
[ui]
ignore=/home/gajon/.hgignore_global
And ~/.hgignore_global contains some filters for common transient files:
syntax: glob
*.pyc
*~
.*.swp
.svn*
*.svn*
*.fasl
syntax: regexp
^\.pc/
Suppose you have proj1 and proj2. proj1 is a mercurial repo you want to clone to proj2, but proj2 already has files in it.
Try this:
hg clone proj1 proj3
mv proj3/.hg proj2
rmdir proj3
cd proj2
hg update -C -r tip
In the other directories that already exist, you could hg init, hg add what you want, hg commit that, hg pull from the central repo, and deal with the resulting merge.
Update because of new insights: Upon seeing this question five years later, I realise that this stems from trying to use a version control system as a package manager. This of course leads to all sorts of unexpected issues, and we shouldn't be using it that way. If you're reading this question, I suggest searching for a package manager for your preferred language.
My original question: I'm currently in the process of moving from Subversion to Mercurial, and I have to say I don't regret that decision. However, when trying to convert my project, I ran into a problem of Mercurial, which I can't seem to get fixed. I have two distinct projects: one is a framework, and the other is an application that relies on that framework. Here's what the repositories look like:
The Framework repository:
docs/
deploy/
lib/
tests/
The Application repository:
application/
config/
lib/
tests/
www/
What I'd like is for the application's lib directory to contain a copy of the frameworks' lib/ directory. I used to do this using svn:externals. Now, I am aware that Mercurial supports the concept of subrepositories, but that doesn't seem like the "correct" solution, as it doesn't actually pull in the lib/ directory like I wanted, as you'll still have to pull and push changes manually. That, plus once you clone the framework repository, you'll get all of it, not just the lib/ directory. I only need the lib/ directory, not the tests, or the docs.
Now, I thought up two different solutions to this problem, but I wonder which is the best. The first solution would be to clone the framework in a different directory altogether and create symlink in the application's lib/ directory which points to the framework's lib/ directory. Putting the symlink in .hgignore should make sure all is well, I think? That means that you could edit the frameworks code, and commit that, and you could edit the application's code and commit that, too.
The other option is to have multiple repositories. The framework gets pulled as a whole, which means you'll get the docs/, deploy/, test/ etc. directories, which are not needed for usage of the framework. I thought maybe creating a repository purely for the library might be a solution, although I sincerely doubt it, as the Unit Tests are very dependant upon the library itself.
Does anyone know a decent solution for this problem?
You should put separate components in their own repositories. Then, when you create an application, you can use the convert extension to create a pullabale framework repository out of the normal one:
$ hg convert --filemap map.txt framework new-framework
with map.txt containing the renames/includes/excludes (the following one should only include the lib directory and move everything in there to the repository root):
include lib
rename lib .
From the application repo, you can now just pull the framework repo (use -f the first time, since the repositories will likely have nothing to do with each other).
$ cd project
$ hg pull -f ../new-framwork
$ hg merge
Now, when the development goes on, you just have to re-create the converted repo every time before you pull and you're good to go. We actually have a hook on our framework repository that re-creates the converted repo on every changegroup (= every push).
This way you have both work areas (app and framework) in their own repositories, while the app repo contains the complete framework history and is able to be updated by simply pulling from the converted repo.
You should separate out the libraries into a separate repository if you need to refer to only that.
Provided you actually do need to only refer to that. What is the problem with referring to the repository containing the lib directory + other stuff, other than "it doesn't feel right"?
As for "still have to pull and push manually", when you pull your main clone, it'll pull the subrepos as well, but it won't update them to a newer revision, which is a good thing, you need to do that manually, just as you should with Subversion.
Have just started using git. Building and installing it was easy. Then I went into the directory of one of my web projects and added a git repo to it.
$ cd ~/Sites/webapp
$ git init (and so on)
I also set up gitweb, and when I added ~/Sites/webapp to $projectroot setting in gitweb.cgi, that appeared in my browser when I went to http://localhost/gitweb/gitweb.cgi
My question is thus -- from what I understand, git doesn't have a central repo concept. Every project that I may be working on will have its own git repository. Since my projects are all over my hard disk, their respective repos are also all over the hard disk. How do I add multiple repositories to gitweb? Is there some kind of central registry of all my repos? Should I really rejig how I work, and move all my projects to a central directory? How is this done?
Better late then never I guess.
I solved it by creating a project root directory and linking the git repositories.
project_root="/opt/gitweb"
In /opt/gitweb
ln -s ~/Sites/webapp webapp.git
ln -s ~/someotherplace/whereitis/application appliation.git
I do this by creating an empty repository and linking to the repositories I want to browse. This workaround is necessary because of the complicated way instaweb runs gitweb and sets the project root.
git init instaweb
cd instaweb
ln -s ~/projects/gitproj1 gitproj1
ln -s ~/projects/gitproj2 gitproj2
git instaweb --httpd webrick
The server is up and running now and the homepage will list a .git project (which is the empty repository you just initialised) along with the two actual projects you linked to.
I have a Mercurial repository containing a handful of related projects. I want to branch just one of these projects to work on it elsewhere.
Is cloning just part of a repository possible, and is that the right way to achieve this?
What you want is a narrow or partial clone, but this is unfortunately not yet supported.
If you already have a big repository and you realize that it would make sense to split it into several smaller repositories, then you can use the convert extension to do a Mercurial to Mercurial conversion. Note that this creates a new repository foo and you cannot push/pull between your-big-repo and foo.
The convert extension is not enabled by default so add the following to your repo's hgrc file or your mercurial.ini file:
[extensions]
hgext.convert=
Then create a map.txt file with
include "libs/foo"
rename "libs/foo" .
(note you can use forward slashes even on Windows) and run
$ hg convert --filemap map.txt your-big-repo foo
That will make foo a repository with the full history of the libs/foo folder from your-big-repo.
If you want to delete all evidence of foo from your-big-repo you can make another conversion where you use exclude libs/foo to get rid of the directory.
When you have several repositories like that and you want to use them as a whole, then you should look at subrepositories. This feature lets you include other repositories in a checkout — similarly to how svn:externals work. Please follow the recommendations on that wiki page.
Instead of doing a partial clone, you can use the Convert Extension to split your repo into more than one repo by sub repository.
Specifically, see the section, Converting from Mercurial:
It's also useful to filter Mercurial repositories to get subsets of an existing one. For example to transform a subdirectory subfoo of a repository foo into a repository with its own life (while keeping its full history), do the following:
$ echo include subfoo > /tmp/myfilemap
$ echo rename subfoo . >> /tmp/myfilemap
$ hg convert --filemap /tmp/myfilemap /path/to/repo/foo /tmp/mysubfoo-repo
I've stumbled accross this issue and found one way to do it: Using symlinks (Linux only unfortunately)
For example, if you only need /project in the repository, on your computer clone the repo in another folder, then use ln -s /repo/location/ project. Mercurial will handle it
(Late 2016) Mainline Mercurial still doesn't package support for "narrow clones" but there are third party extensions that tackle the problem in different ways.
If you can cope with just a narrow checkout (aka "sparse checkout" or "partial checkout by file path") then Facebook's sparse.py extension from the hg-experimental repository (look inside the hgext3rd/ directory) may be workable. In this scenario, you still clone the full history (thus the .hg directory is no smaller) but your working directory only shows/acts on a subset of the full repository.
Alternatively Google have created a NarrowHG extension that does narrow cloning (aka "partial cloning by file path"). You will need to be in control of the server, the client and be willing to use experimental features but it really does restrict the clone's copied history in .hg to a subset of what was in the original repository.
(2019) The sparse extension was merged into Mercurial 4.3 as the experimental sparse extension. The NarrowHG extension was merged into Mercurial 4.6 as the hgext.narrow extension.
It is not possible, hg clone will clone the whole repository.
You can take a look a the sub-repository extension that allows you to have repositories inside a repository, which might match your needs.
This is straight forward with the Convert extension.