Suppose I'm developing software for different clients with a common need.
I develop the system with a single line of development, and that is ok. Each client has the software installed on a server, and updates are made by pulling from a bitbucket repository.
Later, one of the clients asks for a custom module for his own needs only. Other clients may also have other custom needs as well.
My problem is: How can I manage the customization, pulling (updating) new versions of the main line of development, and the customs only when they are present?
One option: make the customisation part of the codebase rather than managed through Mercurial. There are lots of good reasons for doing this, including that there's no danger of you forking and ever maintaining separate version of code, or indeed maintaining separate branches of a repository tree. Both of those are pretty dull tasks that could lead to problems, so I'd favour that if I were you. It depends on your language but I would aim for keeping a load of modules in the MainCodeBase with some way for users to configure which are used.
Second option: sub repositories. Make a repo for Customer1 which includes that customer's modules and has a sub-repository of MainCodeBase. Customer1 just needs to recursively pull their repository to get the latest modules and the latest MainCodeBase. Problem here is it starts to get ugly when you want more than one customer to use the same module.
Related
I have my Banana Pi set up as my Mercurial server. It works well for me for my software as generally speaking I have firmware and that's about it in my repositories. I can access it via open VPN from anywhere in the world. However, I have started to use version control for my PCB files as well now, due to a new CAD system which complicates my old, crude but effective way of doing my PCB archiving and backup. (Also, everything in my new CAD system, all the PCBs and schamtics, are text files which makes version control work nicely.)
So, with Mercurial I started doing as I did with software and creating a new repo for my PCB for one of the boards I'm updating for a customer, and immediately came across an issue that svn seems to cope with easily and I was wondering whether Mercurial can do the same.
I have my BH0001 project repository which has all the embedded C in it and I have started creating a new issue of the PCB for which the C code is used. I had to create a new Mercurial repo called BH0001_pcb to differentiate between code and PCB. With svn you can have a project repo and then Hardware and Software directories within the project number, but still be able to check out the two different types of files to different places independently.
I could, of course, clone the BH0001 software repository to a local machine, add the PCB info in a new folder in the local Mercurial repo send it all back to the server and it would be perfectly happy. The problem then comes when checking out because I would be cloning both firmware and PCB on to a machine when I might only want one or the other.
Also, this goes against how I store stuff locally. In my /username/home directory I have a Software directory and a CAD directory and within those I have projects. So I would have:
home/CAD/CustomerName/BH0001
and
home/Software/CustomerName/BH0001.
If I'm to carry on using my current method do I have to:
Change my local directory structures to be something like:
home/Projects/CustomerName/BH0001/CAD
and
home/Projects/CustomerName/BH0001/Software
Suck it up and use things like ProjectName_pcb for separate repos.
Some other way I can't think of/can't find/am unaware of? e.g. There's a way of checking out part of a Mercurial repository to one directory and a different part of the repo to a different directory.
Or should I just use svn if I really want to carry on as I have?
With default mercurial you currently cannot do partial repository clones as you can do with SVN. So your approach to use separate repositories is a good choice.
However there ways to achieve a similar result: sub-repositories. In your case I'd create a parent repository which contains your two current repositories as sub-repositories. Mind though, sub-repositories have some rough edges, so read the linked page carefully - I'd like to especially stress that it's good practise to have a parent repo which basically only contains the 'real' repos but not much on its own.
There exist ideas like a thin or narrow clone (which is somewhat identical to what SVN does), but I haven't seen them in production.
Assuming I am expanding some project hosted on bitbucket with multiple features (managed by mercurial).
If I build up the features one on top of the other (linear local history) I have a local code base that has all the features I need, but the maintainer of the package cannot pick and choose the features he likes. (Because they build on each other.)
If I build up each feature in a separate branch based on the origin master, all the feature PRs are independent of each other (allowing the maintainer to pick and choose), but I no longer have a unified local code base with all my required features.
How does one solve this problem? With patch queues? If so, how?
I'd go for both actually: create a separate feature branch (anonymous heads, maybe named by means of a bookmark) and pull request for each feature.
Additionally, for your own benefit, and maybe also for others to check it all quickly, merge those into your main development branch, your main line.
In principle mercurial has the system of phases and allows for non-publishing repositories which allows to keep a draft history and allows you to easier make updates - but that's afaik something which bitbucket does not yet (fully?) support.
I am currently using Mercurial, along with the Guestrepo extension, to manage and version the different components of a project. I have come to a quite stable workflow to manage the different versions of the components.
However, I can't come up with an effective solution when it comes to versioning subtle variations of a component. This is, for example, a slightly different embedded device driver (different serial port speed for example), or a GUI which is written in English instead of German.
I don't think stacking them in the Release/Stable branch is a good workflow, as the proliferation of different configurations (English,Spanish,Chinese,...) could lead to a serious and nonsense bloat of the Release branch.
On the other hand, creating a separate Release branch for each, would lead me to many, many branches, which is not the best solution IMHO.
Creating separate repositories for each of the configurations would suppose a quite tedious task whenever a structural change had to be made, as all of the repos would have to be updated with that change.
Any idea on this?
Thank you.
As #EldadAK suggests, creating a repo for each configuration and importing core functionalities from other repos seems a nice idea.
However, I still can't figure out how to arrange "same but slightly different" components, which differ in some subtle features but share their core.
Is it a code architecture issue? Should the components be refactored so that the main core and the differing features lay in different components, which are related using custom builds for each configuration?
IMHO, you should keep all changes in your main branch. Managing all branches or even multiple repositories is not scalable and will eventually get out of hand.
I don't think you should care about the size of your release branch. By keeping it all together, you will always know where you are, what is included in a release and when you really need to branch, have all the changes accumulated to that point.
It's my personal opinion that you should try and keep it simple to manage looking many revisions and years ahead...
Note - Project managers tend to think about next week. You need to think about next year...
I hope this helps.
effective solution when it comes to versioning subtle variations of a component
While I can't see any serious drawbacks from using named branches in one repo, you can use another ("default" de-facto) solution for configuration management inside Mercurial: MQ
You have only slightly adopt your workflow (same amount of branches, same amount of repos) to MQ
Scenario:
Various products made up combinations of the smaller projects. A few different versions of each product in dev, release and maintennace (bugs/patches/minor releases).
Most the the team use various third party tools and libraries for dev and for release (common two are XUnit for dev, and AutoMapper in product). They're fans of version controlling these tools/libraries where it makes sense.
I cannot seem understand the best way to organise the structure in mercurial. In the the central SVN style, I'd organise by having the the third party tools as their own projects and then having small builds for the projects that would grab the output of the other projects, and then a release project that would be built from the built projects. All would be in a hierarchy,
(dev branch)
Root/dev/ProjectX/
Root/dev/ProjectY/
Root/dev/ThirdParty/XXX -- could be a 3rd party lib
Root/dev/ThirdParty/YYY -- could be a 3rd party lib
(branch 1)
Root/release1/ProjectX/
Root/release1/ProjectY/
Root/release1/ThirdParty/XXX
Root/release1/ThirdParty/YYY
(branch 2)
Root/release2/ProjectX/
Root/release3/ProjectY/
Root/release2/ThirdParty/XXX
Root/release2/ThirdParty/YYY
etc.
Here comes the rub, due to the way that developers keep their machines upto date (using NUGET package manager) the third party items all have to be in the ThirdParty folder to to ensure that the devs don't have to have multiple copies of these libraries for each project.
My question is this:
If they implement mercurial should implement a simmilar strategy (big repo) and clone/branch here or should they break up the repository say at the project level and clone/branch these. In the latter case would they have a product/version branch/repo? I know they'd prefer a distributed model if it works better in the long term, even if the pain of learning a new workflow is hard initially.
I've read http://nvie.com/posts/a-successful-git-branching-model/ and a number of articles but I'm still unsure as to how to organise.
Basically, you make a separate repo for each product, each project, and each third-party library, and then you combine them as appropriate in subrepositories. On your central server (or servers), I would probably set up a structure of bare repos like this:
products/ProductA/
ProductB/
ProductC/
projects/ProjectA/
ProjectB/
ProjectC/
thirdparty/ThirdPartyA/
ThirdPartyB/
ThirdPartyC/
This way you have exactly one copy of each repo on your central server. You don't need to make separate repos for each branch, because mercurial can hold multiple branches in one repository.
Then, when someone checks out a Product in their local repository, you use the subrepo mechanism to also check out the working trees for the appropriate projects and third party libraries. On your local disk, the structure will look like this:
ProductA/
ProjectA/
ProjectB/
ThirdParty/
ThirdPartyC/
This looks different than on the central server because there you don't have working trees, only pointers into the subrepos.
At my organization, we'd like to switch from using CVS to Mercurial for a variety of reasons. We've done a lot of investigation when trying to determine how we should organize our Hg repositories based on what we have in our codebase and how we tend to work. We've come up with satisfactory answers to most of our questions, but there's one point that's stumping us a little, and it's driving me mad because the most convenient way to organize the repo for our workflow just seems like the wrong way to go from a conceptual standpoint. I'm trying to figure out whether our perception of how this is "supposed" to work is wrong, or whether we're just bumping up against a legitimate feature gap in the available tooling.
Primarily we maintain a medium sized codebase consisting of a suite of applications that get all get released in the same package. Conceptually you can divide our code into three categories:
Shared code
Application code for our primary suite (uses the shared code)
Miscellaneous small utilities that are infrequently maintained (uses the shared code)
This doesn't seem unusual to me, but I want to stress the point that we maintain the application code and the shared code at the same time and always want them to be bleeding edge with respect to each other. That is, we want all our application builds to always use the latest version and the same version of the shared code. We frequently add to or modify application code and shared code at the same time. Currently, the shared code is in one CVS module, and the applications are all in their own separate modules. The shared code and application modules are checked out such that the shared code gets built once and then linked into each application. We frequently do cvs commits that include changes across shared and application modules at once. We would really like to keep that ability.
I understand that commits in Hg are atomic within repositories -- that's fine but I'd like to be able to diff and commit to an application and a shared library at the "same time" (i.e. I don't care if it's really atomic but I don't want to have to manually do two separate diffs and two separate commit actions).
Conceptually, it seems like it would be correct to have one or a few repos for the shared code, and a separate repo for each application and each little utility program. This means you'd need to check out multiple repos for each build but that isn't a problem for us. The problem is there doesn't seem to be any tooling that lets you view updates or changes on multiple repos at once, or diff multiple repos at once and then sequentially commit them for you. This would be easy to script, but that wouldn't help developers who want to use various GUI frontends to complement the command line.
It seems like in order to be able to commit across multiple codebases at once (from a user's perspective) and keep everything on the bleeding edge together, the only two solutions are:
Use a monolithic repo with EVERYTHING in it.
Create some subrepos but access/commit everything through a big monolithic "main" repo that contains all the subrepos to keep everything on the latest revisions (which doesn't seem any better than (1) to me).
It can't be that unusual to want to work with multiple "peer" repositories at the same time, even if the actions aren't truly atomic across all of them -- and yet I'm not finding tons of articles or posts clamoring for this ability.
In summary:
We would like to organize our code such that we can diff and commit application code and shared code at the same time from the user's perspective (they need not truly be atomic).
It seems like we should be putting application code and shared code in separate repositories.
Subrepositories tie parent repositories to specific revisions, which we do not want.
What am I missing here?
In my shop, we have many projects that are simply in separate repos, but the main application's repo has 2 other projects in it. One is a module that shares a significant amount of code with the main application, and the other is for database migrations for the application (it's even in a different language). I wanted related changes in both the application and the migrator to be committed together, inseparably. Altogether, all source files in this repo are between 10 and 11 MB.
So if putting everything in one repository is really what makes sense because you don't want to deal with subrepositories, then there's nothing wrong with putting everything in one repository. The one of mine is on the small side of medium, in my opinion. TortoiseHg's source is around 20 MB, OGRE is over 100 MB.
Without knowing more about your projects and their relationships, the impression I get is that a single repository would work just fine, and that you're not looking at this incorrectly.
If you change your mind, hg convert can help you extract projects into their own repository, maintaining the history of those files.
If the one-repository approach is not for you, then I think subrepos should be given a chance, as that is the only other method I know of for treating multiple repos cohesively that is supported in TortoiseHg (see the Recommendations section).
However, I'm not sure how you would deal with the inter-department access, given that it doesn't seem there is an established subset already shared with others.