I am designing an Eclipse feature that contains bundles that require dependencies that may be present in the installation target, but this is not necessarily the case. I would like to avoid the installation of unnecessary IUs. I would like to know how to advise the p2 installer that I would like a feature installed only if required IUs are not present in the installation and provide the repository location to download the IUs from.
The desired scenario for a main IU M that depends on an IU D would be:
p2 attempts install of M, finds that it requires D
p2 finds that D is missing in the current installation
Based on the installation advice, p2 obtains D from the repository URL associated with D
p2 installs D
p2 installs M
AFAIK, it is neither possible to
conditionally add a p2 repository only if a dependency is not satisfied, nor
add a p2 repository before an installation - you can only add p2 repositories e.g. during the installation of a feature, which then may be used in the next operation, e.g. an update.
However you can achieve your use case by including the p2 repository with the dependencies of your feature in your own p2 repository.
Technically, there are different solutions to this, but the simplest one is probably to create a so-called "composite repository" that includes both the repository with your own content, as well as the repositories with the external dependencies.
Related
We have Mercurial version control system and we are having difficulties managing the release for mandatory features, optional features and research. We have two separate repositories, one for release and one for development.
We have scheduled deployment every quarter. So we develop features (mandatory and optional) for three months, and testers test the features on Dev site (continuous build).
At the end of release cycle we have to take mandatory and optional features (QA passed only) from development repo and manually copy them into release repo. We cannot merge the development repository into release and have to copy files manually because, at the time of release there could be un-tested/un-developed features/code in development repository. We then build out Test site from release repository and have testers do full testing there. If there are any issues found these are fixed in release repo first and the the release repo is merged into development repo.
However, because of manual merge, there is a chance of copying unwanted changes/files into release repo and cause issues.
Can someone please advise how to get rid of this manual copy paste using Mercurial? I am sure this is a standard process for a development company and there should be a better process to handle this.
Thank you.
You must make feature development from release points. Never on tip or from random points:
v1 ----- v2 -- v3 --> v?
|\ // /
|\f1 ---/ /
|\f2 --- /
| f3 -------
f4 ------------------>
f... are features development. v... are points after successful merging/testing sets of features (strictly speaking it's a tag, but for maintaining release fixes you may use named branches).
In example I select features f1 and f2 for v2 release by merging/testing/fixing to v2 branch or default branch with tag v2 changes that in f1 and f2 branch. If I found bug in f1 I fix it in f1 and then merge to v2' and v3'.
f3 feature ready for shipment only in v3 release. While we still consider f4 isn't safe for shipping.
Essential part - properly choose point from which you start implement feature. If you select too early v point - some features may be missed in source code tree, if you select too late and will be asked to deliver feature in early release v0 - you must backport your features without support from DVCS tool.
You can't develop feature on top of another feature, only on top of features joint points.
This is similar to patch-management but we try to keep merge history so further merges would easy as we have clean directional feature-relation graph.
Also you ask to avoid cherry-picking of "features". That term usually associated with bugs picking which is possible to avoid by finding by bisect first changeset that introduce bug, fix bug on top of that changeset and merge to all necessary branches that fix.
Or just fight with cherry-picking like you do now as very unlikely it is easy adopt development/release model to such wired. SVN branch model (implementation specific) very close to that you are doing with Mercurial.
I am trying to take all our recent code from our development branch to release branch upto a certain changelist (say CL no. 1234). How can I achieve that?
I know that I can do p4 describe 1234 to see the timestamp and then do p4 integ //path/to/dev/branch/...#timestamp //path/to/release/branch/.... But is there a more elegant way of doing this?
Integrate your dev branch (source) into your release branch (target) at a changelist, rather than integrating using a timestamp. Since you want to integrate up through CL 1234, you should run the following command:
p4 integrate //path/to/dev/branch/...#1234 //path/to/release/branch/...
This integrates the dev branch (all files in the state they were in after CL 1234 was submitted) into the release branch.
There have been a couple of questions about Hg sub-repo dependencies in the past (here and here) but the accepted answers don't seem to address the problem for me.
A project of mine has 4 dependencies: A, B, C, D. D is dependent on A, B and C; and B and C are dependent on A:
I want to use Hg sub-repositories to store them so I can track what version of each they rely on. This is because, while I am using A,B,C and D in this project, other projects will require just A and B. Therefore B and C must track what version of A they need independently of D. At the same time, in my application the versions of B and C referenced by a given version of D must always use the same version of A as that referenced by the given version of D (otherwise it will just fall over at runtime). What I really want is to allow them to reference each other as siblings in the same directory - i.e. D's .hgsub would look like the following, and B and C's would look like the first line.
..\A = https:(central kiln repo)\A
..\B = https:(central kiln repo)\B
..\C = https:(central kiln repo)\C
However this doesn't seem to work: I can see why (it'd be easy to give people enough rope to hang themselves with) but its a shame as I think its the neatest solution to my dependencies. I've read a few suggested solutions which I'll quickly outline and why they don't work for me:
Include copies as nested sub-directories, reference these as Hg sub-repositories. This yields the following directory structure (I've removed the primary copies of A, B, C, B\A, C\A as I can accept referencing the copies inside \D instead):
project\ (all main project files)
project\D
project\D\A
project\D\B
project\D\B\A
project\D\C
project\D\C\A
Problems with this approach:
I now have 3 copies of A on disk, all of which could have independent modifications which must be synced and merged before pushing to a central repo.
I have to use other mechanisms to ensure that B, C and D are referencing the same version of A (e.g. D could use v1 while D\B could use v2)
A variation: use the above but specify the RHS of the .hgsub to point to a copy in the parent copy (i.e. B and C should have the .hgsub below):
A = ..\A
Problems with this approach:
I still have three copies on disk
The first time I clone B or C it will attempt to recursively pull the referenced version of A from "..\A", which may not exist, presumably causing an error. If it doesn't exist it gives no clue as to where the repo should be found.
When I do a recursive push of changes, the changes in D\B\A do not go into the shared central repo; they just get pushed to D\A instead. So if I push twice in a row I can guarantee that all changes will have propagated correctly, but this is quite a fudge.
Similarly if I do a (manual) recursive pull, I have to get the order right to get the latest changes (i.e. pull D\A before I pull D\B\A)
Use symlinks to point folder \D\B\A to D\A etc.
Problems with this approach:
symlinks cannot be encoded in the Hg repo itself so every time a team member clones the repo, they have to manually/with a script re-create the symlinks. This may be acceptable but I'd prefer a better solution. Also (personal preference) I find symlinks highly unintuitive.
Are these the best available solutions? Is there a good reason why my initial .hgsub (see top) is a pipe-dream, or is there a way I can request/implement this change?
UPDATED to better explain the wider usage of A,B,C,D
Instead of trying to manage your dependencies via Mercurial (or with any SCM for that matter), try using a dependency management tool instead, such as Apache Ivy.
Using an Ivy based approach, you don't have any sub-repos, you would just have projects A, B, C and D. A produces an artifact (e.g. a .jar, .so or .dll, etc), which is published into an artifact repository (basically a place where you keep your build artefacts) with a version. Projects B and C can then depend on a specific version of A (controlled via a ivy.xml file in each project) which Ivy will retrieve from the artifact repository. Projects B and C also produce artefacts that are published to your repository. Project D depends on B and C and Ivy can be told to retrieve the dependencies transitively, which means it will get the artifacts for B, C and A (because they depend on A).
A similar approach can be used with Apache Maven and Gradle (the later uses Ivy)
The main advantages are that:
it makes it very clear what versions of each component a project is using (sometimes people forget to check .hgsub, so they don't know they are working with subrepos),
it makes it impossible to change a dependant project (as you are working with artifacts, not code)
and it saves you from having to rebuild dependent projects and being unsure of what version you are using.
saves you from having multiple redundant copies of projects that are used by other projects.
EDIT: Similar answer with a slightly different spin at Best Practices for Project Feature Sub-Modules with Mercurial and Eclipse?
You say you want to track which version they each rely on but you'd also be happy with a single copy of A shared between B, C and D. These are mutually exclusive - with a single copy of A, any change to A will cause a change in the .hgsub of each of B, C and D, so there is no independence in the versioning (as all of B, C and D will commit after a change to A).
Having separate copies will be awkward too. If you make a change that affects both B's copy of A and C's copy then attempt to push the whole structure, the changes to (say) B will succeed but the changes to C will fail because they require merging with the changes you just pushed from B, to avoid creating new heads. And that will be a pain.
The way I would do this (and maybe there are better ways) would be to create a D repo with subrepos of A, B and C. Each of B and C would have some untracked A-location file (which you're prompted to enter via a post-clone hook), telling your build system where to look for its A repository. This has the advantage of working but you lose the convenience of a system which tracks concurrent versions of {B, C} and A. Again, you could do this manually with an A-version file in each of B or C updated by a hook, read from by a hook, and you could make that work, but I don't think it's possible using the subrepos implementation in hg. My suggestions really boil down to implementing a simplified subrepo system of your own.
My project is made up of code in the following locations
C:\Dev\ProjectA
C:\Lib\LibraryB
C:\Lib\LibraryC
Presently each of these folders is a completely independent Mercurial repository. Project A changes all the time, Library B and Library C change rarely.
I currently tag each version of Project A as it is released and (when I remember) put a corresponding tag in the Library B and C repositories.
Can I improve upon this by using subrepositories? Would that require me to make Library B and C a subdirectory of Project A?
If Library B and C must be subdirectories of Project A what do I do if I want to start a Project D that uses Library B but isn't otherwise affiliated with Project A at all?
If Library B and C must be
subdirectories of Project A what do I
do if I want to start a Project D that
uses Library B but isn't otherwise
affiliated with Project A at all?
Any project can exist both independently and as subrepository of another project at the same time. I'll explain by suggesting a workflow.
First of all, each of your projects (A, B, C) should have a blessed repository that is published somewhere:
You could run hgwebdir on your own server, or make use of a Mercurial hosting service like Bitbucket or Kiln. This way developers have a central authorative point to pull/push changes from, and you have something to make backups of.
Now you can make clones of these repositories to work on in two different ways:
directly clone your project. For example:
hg clone http://bitbucket.org/LachlanG/LibraryB C:\Lib\LibraryB
and/or create subrepository definitions by putting a .hgsub file in the root of ProjectA with the following content:
libraries/libraryB = http://bitbucket.org/LachlanG/LibraryB
libraries/libraryC = http://bitbucket.org/LachlanG/LibraryC
These subrepository definitions tell Mercurial that whenever Project A is cloned, it also has to put clones of Library B and Library C in the libraries folder.
If you are working in Project A and commit, then your changes in libraries/LibraryB and libraries/LibraryC will be committed as well. Mercurial will record which version of the libraries is being used by Project A in the .hgsubstate file. The result is that if you hg update to an old version of the project to see how things worked last week, you also get the corresponding version of your libraries. You don't even need to make tags :-)
When you hg push the Project A changes to the blessed repository, Mercurial will also make sure to push the subrepository changes first to their own origin. That way you never accidentally publish project changes which depend on unpublished library changes.
If you prefer to keep everything local, you can still use this workflow by using relative paths instead of URLs in the subrepository definitions.
You can indeed declare B and C subrepos of project A (they will appear as subdirectory, as described in Mercurial Subrepository).
That would improve your release mechanism as it would allow you to:
get all repos in one place (A and under)
reference an exact tag of B and C under A
tag each sub-repo s first if they had any modification
tag A with the information about B and C tags in it (any clone of A will be able to get the exact tags of B and C used by A)
You can also declare B as a subrepo of D, independently of A. What you make in A (regarding B) will have no consequences for B used in D.
I'm working in a application with many modules each having it's own mercurial repository.
I initially thought it's good to have the modules in individual repositories but after a couple of releases, I feel something is not right. It's really a pain to create the branches and tags in all the modules.
Most, if not all modules follow a similar release cycle.
Should I go ahead and use a single repository for all the modules? Or is there a better way?
A single repository for all modules means they are tightly coupled in their development lifecycle:
any branch applies to all modules (which is what you want)
any tag applies to all modules (which maynot be what you want)
If "v1.2" for your software has any meaning for each and everyone of your modules, then yes, having them all within one repo is useful.
If some modules are at v2.4 while another is at v3.6, and another at "v4.5", and..., then having independent modules declared as subrepos is best.
Lasse V. Karlsen comments:
if you're sharing things, like components and general framework libraries, they belong in their own repositories
Which is right, since the development lifecycle of said components and general framework libraries is completely unrelated to the one of the main program
But the OP adds:
We have two sets of modules:
a set of core modules that can be reused across many applications and
another set of modules for the respective application
So some of those modules (the "set of core modules") can be kept as subrepos (independent repos referenced by the parent repo and main project).
The others can be merged directly into the parent repo (kind of like the git subtree merge strategy) with the Hg tip you mention: "Combining Repositories"
If all these modules belong to a single project they should have a single repository. The module code can be grouped in directories within a single repo.
[Edit: based on comments]
The structure looks like :
You have core modules ~ Platform
Various other apps / modules that utilizes the core modules.
In such a case, the platform or core module can develop at a different speed than app modules. It is better to segregate them into separate repositories. Initially, it does look enticing that they both may follow similar release cycle but in any typical platform / application development, they do go out independently and out of sync. At least that has been my experience.
P1 -------P2 ------P3 ------p4
A1------A2--------------A3--------- (A1, A2, A3 utilize platform P1, P2, P3..)
B1--------B2----------B3--------- (B1, B2, B3 utilize platform P1, P2, P3..)
A3------------------B3--------------
It's really a pain to create the branches and tags in all the modules
Because this really not needed at all ("in all modules").
If you use Subrepo (or, better, GuestRepo - created exactly for your use-case and as compensation of some subrepo's drawbacks) extension and your Product is "SuperRepo", which contain only linked sub|guestrepos, then:
For every and each changeset in Superrepo state of all child-repos is known and predefined (each definition contain changeset-ID of foreign repo). Thus:
Then you tag, you can (have) only tag Superrepo - tagged changeset will have all (immutable) relations
Then you branch, you can don't branch submodules at all, or branch submodule when it needed for development, not for policy (final result in any case for SuperRepo - changed changeset ID in link to this subrepo): branch "Release N" doesn't require the same branch on submodules, only slightly more handwork in Superrepo
From POV of flexibility and manageability I still prefer separate repo for each low-level module (self-sufficient object, without external dependences) and GuestRepo for collecting modules in Product(s) and managing Product in it's lifecycle - I can't see "branching|tagging nightmare" here