"factorizing" a mercurial repository on kiln - mercurial

Summarized questions:
What is the simplest (and best) way to shift a group of files from an existing repository to a new sub repository, so those files can be integrated with other parent repositories, some of which may not yet exist?
Do files in subrepositories need to be in discrete folders, or can they exist alongside other files?
Detailed Questions:
I have begun the process of creating multiple repositories representing several projects that have shared components, and that is going well, thanks to SO and some helpful answers to my question here
As I move on to adding a second project I notice there are a few files in my projects that are duplicated, and are essentially the same thing, with enough similarity to warrant taking them out of a main project repository and creating a new subrepository so they can be
used by any new projects I begin, and
removed from other existing repositories, since they are identical.
I am assuming the best way is to simply create a new repository, move the files across on the local file system, push both repositories, and then create a .hgsub file and proceeed as in the answer to my earlier question. This would obviously then shift the files concerned to a subfolder in the local file system under each main project, which i can live with, but it does raise the hypothetical question - is it possible to have a list of files in a repository that are effectively part of a sub repository but reside alongside other files (i.e. not in a sub folder).
If I wanted to (for example) have a "acme.h" file in each project that is part of another repository could I do this? as it happens, I don't need to do this at this point in time, and in my current situation it would be better from a design point of view to have the files I need to "refactor" into another repository in their own subfolder, however that might not always be the case. I use refactor in quotes here, as strictly speaking it's more about refactoring duplicated files that is refactoring code - however the same principle applies.
hopefully my questions are succinct enough to be answered without too much more explanation.

Thanks for summary, makes it much easier to answer!
What is the simplest (and best) way to shift a group of files from an existing repository to a new sub repository, so those files can be integrated with other parent repositories, some of which may not yet exist?
You can use the convert extension to extract a directory from an existing Mercurial repository. You'll want to use the --filemap flag and in the filemap you include the directory you want and rename it to the root. See hg help convert for more info.
After you get a smaller repository with the
Do files in subrepositories need to be in discrete folders, or can they exist alongside other files?
They must be in their own folders. This is simply because that's how a repository looks like in Mercurial, Git, Subversion, ... When you're dealing with subrepositories, then Mercurial is not tracking the files inside the subrepo: it's just asking some (other) system to make a checkout of repository foo at some location.
So when your .hgsub file has
foo = foo
bar = [git]bar
baz = [svn]baz
then Mercurial will notice this on hg update and run
hg clone default-path-of-this-repo/foo foo
git clone default-path-of-this-repo/bar bar
svn checkout default-path-of-this-repo/baz baz
for your. This explains why subrepostories are directories in the outer repository: that's simply what a clone/checkout looks like these days.
As you can see, subrepositories can be of different types. It's conceivable that someone could add a RCS subrepository type for tracking individual files. They would then not have to live in a directory.

Related

Is there any side effect if I clone a repo and rename it?

I have central mercurial repository server. I cloned repoA on my local system
initiated new repoB on central server. cloned repob to local. copied everything from repoA to repoB, commit and pushed to repoB (central server)
now i have all the changeset history from repoA on this new repoB
there was a need to do so as there were two application code on same repoA, to separate it i did the above experiment. and it is working.
my question is by doing so is there any side effects , or is there a better way to do it (recommended way ) please suggest, thank you !
When you clone a repository to your local PC, the repository lives in a folder. That name of that folder is typically how people refer to the "name of the clone" or the "name of the repository".
Other than that, the folder name itself has very little significance and is not even properly part of the Mercurial repository.
It sounds like you did several other steps, but basically if you renamed repository A to B it won't make much difference (but see notes below).
You do not need to use hg clone to clone a repository. You can literally just copy the entire repository folder and the copy will work just fine. The one difference that I am aware of when you use clone vs. operating system file copy is that the clone will point back to the repo you cloned from (for use in push/pull operations). The copy would point back to the original source. (See notes below about some related effects).
One situation where you might cause some problems by renaming the repository folder is if you have have cloned FROM it. Example: you have local repo A. You clone A to B. Now internal to the configuration data in B is a reference to the folder path including A. If you rename A to A1 then that path is obviously broken.
In such a situation you can easily edit the B/.hg/hgrc file and modify the line starting with default= to correct the path.
Based on your question it sounded like you copied a bunch of stuff from one repo to another. Presumably this also included the .hg folder. Generally speaking I recommend avoiding the contents of that folder, and always approach it with caution.
Although technically some of it is human-readable it is simpler & safer to treat it as a black box, or you risk corrupting your repository. There are occasional exceptions (like hgrc) but they are few & far between.
Of course if you are just trying to learn how it works then by all means try things & see what happens! One of the great Mercurial features is the ability to copy a repo, mess around with it, and throw it away when done.

Creating Mercurial subrepositories, while maintaining the history

I am about to make some major changes to my Mercurial repositories. As I am going to be using a Feature of Last Resort, I am looking for some advice and reassurance that I am not doing something stupid.
Where I Am:
I have a Mercurial repository with a complete history of all of these files:
/source
/secret_subsystem
/unclassified_subsystem
/common_files
Source is the Mercurial repository.
The secret subsystem folder contains code which is intellectual property we want to keep in-house.
The unclassified subsystem folder contains code which we want to outsource to a third-party to maintain.
The common files folder contains code that both subsystems depend on. We will be keeping ownership, but we want to share it with the third-party.
Obviously, I can't just push out my whole repository to the third-party company. The third-party would see too much.
Where I Want To Be:
Having read up on subrepositories, this is where I think I need to be:
Have THREE subrepositories: secret_subsystem, unclassified_subsystem, common_files.
Ensure there are no other files at the /source level, due to this recommendation.
Have the outsourcers create a brand new respository at the source level on their machines, and two corresponding subrepositories.
Push the unclassified_subsystem and common_files to the out-sourcer, pulling back unclassified_subsystem as required, pushing out new common_files repositories as required.
Maintaining History:
I would like to maintain the commit history, as much as practical, for all of the subsystems.
To do this, I will run the hg convert extension command three times, once for each subrepository. I will filter down to only the files that belong in each subrepository. I may also need to map filenames to move the files from ./common_files/foo.py to ./foo.py (for example).
My Questions:
1) Is dividing up a repository into subrepository a reasonable way of implementing security - viz. that a third-party can only see and edit some of our files?
2) Is using hg convert a reasonable way to create a subrepository from an existing repository, while still maintaining the history?
3) Will hg convert's filter strip out (a) all commits messages about files NOT in the filtered respository? Will it filter out all diffs for files NOT in the filtered repository?
There is another implied question: Am I heading into a world of hurt? If so, I will simply give up on retaining file histories, or even make them seperate repositories and forget about cross-repository commits.
I've not used subrepos so far, but I can answer 2) and 3):
2) Yes, sounds reasonable.
3) Yes.
There was a similar question just 2 days ago: Convert mercurial repository to subrepositories with full history (like hg log -f)

mercurial: "Cloning" just one file from one repository into another one?

I've got a perl project's mercurial/hg repository here with a lot of modules. Now, I've realized that some of the modules, a branch of modules, does not fit into this project anymore but would be better suited in another project.
So, is it possible to "clone" these modules from repository1 into my other repository2 while keeping these modules's revision history and at the same time flagging the modules in repository1 as "removed"?
I think it is possible with a number of steps. But I think cloning the whole repo1 and then removing all the unnecessary files would clutter my new repo2 with a lot of unrelated revision history. Still, as I understand the "clone" and "pull" commands neither of them is suited to clone just a slice of another repository, right?
Or is it possible to "move" a file from one repository into another? As I see it that's the analogy of what I want to do..
Absolutely! Take a look at the Convert extension, specifically the Mercurial-to-Mercurial section which should do exactly what you want.
After you've split off those modules, you can use hg forget, hg remove to remove them from your original repository, or even use hg convert again to create a new repository without them.

Can one Mercurial repository live inside another Mercurial repository?

Can one hg repo live inside another hg repo on my local file system?
I am pulling down the bitbucket wiki for 'sandbox', and I want to know if this should be placed in repos/sandbox/wiki or repos/sandbox-wiki.
Is the former okay to do?
Edit: See Subrepository.
The short answer is yes, but I can't imagine why you would want to.
In your example, I think you should go with:
repos/sandbox-wiki
[edit] Additionaly:
Yo Dowg, I herd you like repositories.
So we put a repo in your repo so you can version while you version
:-)
Yes and no. Depends on what you want to do. You can create repo 'sandbox/wiki' but files in this inner repos won't be commited in the outer 'sandbox' repo (#Jason is right). If you don't want to, no problem.
Try explicitly adding files from wiki repos in sandox and you'll get the message below. If you just add path to some directory containing an inner repo the files will just be ignored.
From sandox root directoy:
hg add wiki/myfile
abort: path 'wiki/myfile' is inside repo 'wiki'
Mercurial does not allow nested repositories, but there is at least one reason for them:
Imagine that you are working in a project: /MyProject. In this folder you put everything: code, documentation, tests, etc.
You want to backup your work because it is very important, so you create a repository for /MyProject. Then, overtime you use bundles to save the evolution of /MyProject and back up them in a USB flash memory so that you can recover everything just in case your hard drive breaks.
Remember that /MyProject contains everything. And among all those things, there are the main code and some auxiliary projects. You also want to track the progress of an auxiliary project that is in /MyProject/AuxiliaryProject, so you use Mercurial to track its evolution.
Also, you want to have a separate repository for the main code: /MyProject/Main
In this situation you want nested repositories: one big one for being able to back-up everything using bundles and child repositories for managing each subproject.
I think Mercurial should give the user several options when initializing a repository. For example:
- ignore nested repositories
- include nested repositories but ignoring .Hg folders (i.e. act as if there were no nested repositories but do not ignore the information contained in the nested respositories).
- include nested repositories and also include .Hg folders (makes sense for back-up purposes)
--------- Edit:
Subrepositories is a feature that is work in progress:
https://www.mercurial-scm.org/wiki/subrepos
Also, there is an extension named "forest" that might become obsolete in the future:
https://www.mercurial-scm.org/ForestExtension
You'd need to set up an .hgignore file in sandbox to exclude wiki because mercurial assumes that it is responsible for all descendants. This would probably generate more user confusion than it is worth.

Fetching a single file from another mercurial repository [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Mercurial: copying ONE file and its history to another repository
I have several repositories in my local machine.
One is my main code, another is an assortment of useful code/tools.
These are two fundamentally different repos. It might make sense to setup a new repo and pull these two in as sub-repos, but I want to wait until Mercurial devs mark sub-repos as non-experimental before I do that.
One of the useful code files has become so useful, I want to put it into my main code area...but I want to maintain its history. This will, of course, result in some variant of a fork, but that's acceptable. (best case would be being able to push-pull it back and forth and keep updating its history).
I'd just use the subrepo feature that came online in 1.3. It might change slightly, but you won't be left high and dry backwards compatibility wise.
If you can't bring yourself to so, then what you need to do is:
use hg convert with a filemap that deletes all files except the one you want and convert from the repo with the single useful file to a new repo containing only that file and all its history
then hg pull from the new single-file-full-history repo into the target repo
hg merge in the target repo and you'll have that file with all it's history
The other option would be to hg export the entire tools repo, use grepdiff (part of difftools) to limit to only one file, and then import into the target repo, but that's crazy.
The short answer is you can't copy a file and its history simply, as stated in this thread:
https://www.mercurial-scm.org/pipermail/mercurial/2009-April/025105.html
I'm relatively new to DVCS and you really have to think of each repo as a self contained package. Not like svn or p4, where you can hang different projects off the root and configure it how you like and do partial repo checkouts. (That said, I really like the flexibility of being able to clone repos quickly to try things out. And being able to commit on a local machine.)
I'm just looking at a similar problem. I've branched a repo to make changes and I only want one file out of one changeset. And it is nice to have the history.
You could look at:
hg cat
This would probably involve writing a script to transfer history, i.e. commit N changesets in the target repo with the hg cat results from the source. Wonder if there is an extension to do this?
You could get the log of the file you want to copy and paste that into a commit comment. It's not in the metadata, but you do have a record and all the hashes etc.
may be
hg export
also can help you.