There are one or two files, like .hgignore, which I generally want to be the same in each of a bunch of projects.
However, the nature of these files means that I can't simply move them to a common shared project and just make the other projects depend on that project. They have to be inside each project. Symbolic links are not an option either because some of our developers use Windows.
How can I share these files between repositories and have changes propagated across (on my local machine, at least)? I'm using Eclipse.
For your specific case of hgignore you can put an entry like this in each project's .hg/hgrc file:
[ui]
ignore.common = ~/hgignore-common
If you you know your common library will always the in the parent directory, as is often the case with a subrepo setup you could do:
[ui]
ignore.common = ../hgignore-common
or if you know it will always be in a sibling directory of project checkouts you could do:
[ui]
ignore.common = ../company-wide-defaults/hgignore-common
Unforunately there's no absolute way to reference a file that's valid everywhere, but you can at least to to a point where on your machine all your checkouts are referencing a common ignore.
Hardlinking instead of copying the relevant files sort of works with Eclipse - although you have to refresh each of the other projects to get it to pick up the change. However, you can configure Eclipse to watch the filesystem and automatically refresh whenever it needs to - I recommend this.
It does not work by default with Emacs, because Emacs breaks hard links (which is normally the right thing to do, but is not what you want in this case). However, you can disable this behaviour for multiply-linked files with (setq backup-by-copying-when-linked t).
Related
We have a Mercurial repository with many projects, each resides in its own directory:
- Main Repo
- Project A
- Project B
- ...
Each of the projects is self contained and must reside in it's own directory, but there are some common files that should be similar between projects.
For example, some projects are websites, and they share a common javascript library we develop. When changing the library in one project, we would like it to change in other projects too, but the file must reside in each of the projects.
I read about sub-repos but they don't seem a good solution for this.
Is there a way to accomplish this in Mercurial?
You are looking for a feature for keeping the same file version in multiple places, also known as file cloning or file sharing in other types of source control, like Sourcesafe or Vault. There is no mechanism like this in Mercurial. Every file is a single entity with a single location.
The first solution you have is to keep the common libraries in a separate place. You need a single copy that can be accessed by all your projects. It does not matter if you use sub-repos or not, they can all be in the same repo, as long as your folder structure includes everything, but sub-repos can be easier to manage if your projects are not related.
The other solutions you have could be to state an internal policy to always sync and commit the common libraries manually (which I do not suggest as it is error-prone and requires effort), or to create a script, either as hook or not, to sync your files, before a commit or after an update (which is more tedious to establish and maintain anyway)...
Conclusion, go for the separation of your common libraries. You'll be glad you spent the extra time to set everything up correctly from the start.
Under Unix you could use soft-link (ln -s) for shared files and Mercurial will detect / save / create them. Just don't use absolute or empty path.
With Windows symbolic links won't work:
Tracking hard or symbolic links with mercurial on Windows
Bug 1825 - junction/parse point for windows directory symlinks
In my experience (local Linux repository) using symlinks to handle shared files works but it's usually better to create a library that contains the common files.
Even if you have one repository for all your projects, it is advised to have a separate library/tool/etc. repository(ies) for the common code(s).
The way you can "use" this code inside your project will then heavily depend on your technology and infrastructure: java/maven/ant world, linux distrib, ruby gems etc. You will generally have some kind of "dependencies specification" language where you can specify that you need such and such library. In a Gemfile for rails, using autoconf for C/C++ etc. Most of the time you can also specify a specific version (or greater than etc..) which allows taking care of API changes.
Basically it is not advised to solve this issue at SCM level but instead to use the right framework for decoupling your common code from the projects repositories.
I'm looking for a solution to have one .hgrc for mercurial to work in all my working environments(win/lin/mac), just like one vimrc to rule all.
It comes to a problem when I have to specify the path for some extensions, for example, hg-git:
on windows:
[extensions]
hggit = E:\hg-git\hggit
on linux/mac:
[extensions]
hggit = ~/hg-git/hggit
For vimrc, we have solution to distinguish different platform, like
if has('win32')
...
endif
Knowing that mercurial's hgrc is barely a normal ini file, I wonder if there is similar solution for hgrc file.
Or maybe some other solution?
Update
Thanks for mentioning using "custom environment variable instead of hard coding", that works for extensions' path.
However, to make extensions work is just one goal. I believe there are other configurations' behavior that depends on platform, like in http://www.ogre3d.org/docs/OGREDeveloperGuide/index.html, specific options are needed for Windows and Mac. How to solve this kind of problem?
How about using the .hgrc include mechanism and three separate files:
~/.hgrc
~/.hgrc-windows
~/.hgrc-linux
where the main ~/.hgrc has:
%include .hgrc-$OS
and then set $OS in your windows profile and your linux profile's. Alternately you could probably find an env variable that already exists on both systems ($USERNAME is usally already set in both windows and unix and may differ for you).
This is just in case you don't find a way to have only one .hgrc (or one set of .hgrc* files, per Oben Sonne's comment).
I only face Windows environments, so my mercurial.ini files are the same except for 1 path. I keep a copy in version control along with my Powershell profile script.
However, if I were faced with environments more varied, I would abstract the configuration to a script in a cross-platform language (e.g., Python) that overwrites the target .hgrc or mercurial.ini. I'd be able to have the pieces that have to be different for each OS/machine right next each other with easy ways of distinguishing which environment is current. Then I would keep that script in version-control to help keep the same version on each machine.
Since I don't change it often, I would probably execute it manually on each machine when I visited it next. For a larger number of machines or more frequent changes, I would have another script executed on login that pulls changes to the generating script from a central repo, and if there were indeed new changes pulled, update to tip and run the script.
(Heck, I'm now tempted to rig this kind of setup anyway.)
Why don't you just use a custom environment variable instead of hard coding the paths?
"If it’s not in source control, it doesn’t exist."
This question was addressed for Git here: Techniques to handle a private and public repository?. What about for Mercurial?
I have several public Bitbucket repos (with multiple committers) where I'd like the source to be public, but which load API, SSH keys and other sensitive info from untracked files. However this results in someone emailing around the new config file if we add a new Mailchimp or Hunch or Twilio API key. Is there a way to shield these files from public view somehow and still track them? Everyone is syncing their repo through Bitbucket.
There are two good ways to handle this (besides zerkms's solution, which doesn't offer the easy of synchronization you want, but is what I'd do anyway):
Use Mercurial Queues. When you create a mercurial queue with hg qinit --create-repo it creates an overlay system that can be qpushed atop the existing repo. So you keep your secrets in queues and qpush them when you need them, and qpop them when you don't. With --create-repo the set of overlays (patches) is handled in a repository of its own. So people in the know can push/pull the secret overlay repo and people w/o access to it can use the base repo. The patch repo can be a private repo on bitbucket or hosted elsewhere.
or
Use a subrepo exactly as described in the git solution.
Create filename.ext.sample files with templates inside (probably filled with dummy data), which need to be copied and filled with actual data in the particular working directory.
That is what I usually do ;-)
Zerkms' solution is fast, easy, and clean, and likely your best bet for preventing secure content from being tracked / published; however as you say, "If it’s not in source control, it doesn’t exist." I find that far more often what I'm trying to keep out of source control is not a security concern, but simply a configuration setting. I believe these should be tracked, and my current employer has a rather clever setup for dealing with this, which I'll attempt to simplify / generalize / summarize here.
REPOSITORY
code/
...
scripts/
configparse.sh
...
config/
common.conf
env/
development.conf
testing.conf
production.conf
users/
dimo414.conf
mycoworker.conf
...
hosts/
dimo414-laptop.conf
dimo414-server.conf
mycoworker-laptop.conf
...
local.conf*
makefile
.conf*
* untracked file
Hopefully the idea here is pretty clear, we define settings at each appropriate level, enabling highly granular control of the codebase's behavior in a logical and consistent fashion.
The scripts/configparse.sh script reads all the necessary configuration files in turn and builds .conf out of all the settings it finds.
config/common.conf is the starting point, and contains logical default values for every setting. Many will likely get overwritten, but something is specified here. It's an error for a setting to be found in another file that isn't first set in common.conf.
config/env/ controls the behavior in different environments, doing things like pointing to the correct database servers.
config/users/ looks for a $USER.conf file, useful for setting things I care about, such as increasing the logging level for aspects my team works on, or customizing behavior I prefer to use across all my machines.
config/hosts does the same for machines, looking for $HOSTNAME.conf. Useful for machine-specific settings like application paths or data directories.
config/local.conf is an untracked file, and lets you set checkout-specific values and/or content you don't want in version control.
The aggregate of all these settings is output to .conf, which is what the rest of the codebase looks for when loading settings.
We recently moved from Perforce to Mercurial and love it!
One little problem: after much research we can't figure out how to map a special directory in the repository to some special place on the client. Here is an example of our hg repo:
/foo/source files
/bar/source files
/build
/macosx/mac make files
/win/windows make files
With Perforce, we were using client spec mappings to map //depot/build/macosx/... to just /build/... on the Mac client, and //depot/build/win/... to /build/... on the Windows dev box. Directories foo and bar are synced as is. Makefiles in /foo and /bar assume that our build makefiles are located in /build and we would like to keep them as is. The final client set of files should look like this:
/foo/source files
/bar/source files
/build/client specific make files
I've read about subrepos, but this solution does not seem to be client specific.
Any idea how to solve this problem will be very much appreciated!
You can't check out only portions of a repository with Mercurial.
You always get a clone containing everything, and the working directory will also contain everything.
With Mercurial you should strive to have 1 repository for 1 project, so that everything you get logically belongs together, and then you shouldn't have much need for just a portion of it.
This also means that whatever directory structure you have in your Mercurial repository will always match exactly the structure you have on disk.
You can't do this with Mercurial as it doesn't have the concept of a client separate from a depot.
However, you can use a symlink on Mac OS X (ln -s) and a junction on windows (mklink on Vista and up using the junction tool on XP http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx) to solve this problem on the file system level.
Alternatively you can use a variable in the Makefiles to refer to the build directory (eg $(BUILD)/something.ext instead of build/something.ext).
This sort of mapping cannot be done in Mercurial. There is an outstanding TODO item for 'narrow' clones so you can check out just a subdirectory. And I could see an implementation of that supporting that sort of functionality. But then again, I know that something like this would be considered a little too 'clever' (read complex) and there would be a lot of push-back on the idea.
In the meantime, I would suggest one of these two solutions.
Symbolic links. Put the symbolic link to your build directory in your .hgignore file. Then each person can make their own symbolic link to the appropriate directory of build files. This has the disadvantage of not working on a platform without symbolic links.
An environment variable that's used in a top level makefile to construct the path to the platform specific makefile it should be calling.
I am writing a set of django apps and would like to use Hg for version control. I would like each app to be independent of the others so in each app there may be a directory for static media that contains images that I would not want under version control. In other words, the binary files would not all be in one central location
I would like to find a way to clone the repository that would include copies of the image files. It also would be great if when I did a merge, if there were an image file in one repo and not another, that there would be some sort of warning.
Currently I use a python script to find images and other binary files that are in one repo, but not the other. But a lot of people must face this problem, so there must be a more robust and elegant solution.
One one other thing...for reasons I do not want to go into, usually one of my repos is on a windows machine, and the other is on Linux. So a crossplatform solution would be nice.
Since Mercurial 2.0 the extension largefiles is now included in the main distribution. That extension keeps and manages large files outside of the "normal" repository in a way that you get the benefit of DCVS but without the benefit of exponential size and processing time growth.
Other extension that work along similar lines are SnapExtension and BigFilesExtension. However, those two are not distributed with Mercurial (you have to get them manually).
Mercurial can track any kind of file, for binary files if something changes then the whole file gets replaced not just the changes.
On the getting a warning if one repo doesn't contain a file, that's kind of the point of a DVCS is that the repos are related but are autonomous. You could always check and see what files were added during a synch or merge operation.
The current Mercurial book (by Bryan O'Sullivan) says, that Mercurial stores diffs also for binary files. How efficient this is, obviously depends on the nature of changes to binary files.