(disclaimer: I am completely new to mercurial and version control)
So I have a folder structure
Programs
CPPLib1
CPPProject11
CPPProject12
CPPLib2
CPPProject21
CPPProject22
Each group of three is completely independent of the other group, but within each group the code is related and I'd like to manage it under version control as a whole (commit/extract everything in one transaction). As I understood by googling it, I must have a repository for each group in their common parent (Programs), but I cannot have 2 different repositories there, right? Does it mean I must have this structure instead:
Programs
Group1
CPPLib1
CPPProject11
CPPProject12
Group2
CPPLib2
CPPProject21
CPPProject22
A related question, this site http://help.fogcreek.com/8169/using-more-than-one-repository says
"Since Mercurial and Git are Distributed Version Control Systems (DVCSs), you should use at least use one separate repository per project, including shared projects and libraries."
So what does this advice mean? I can't have a separate repository for each of
CPPLib1
CPPProject11
CPPProject12
and manage them as a whole. I am confused.
For each of you project groups you'll need to create one repository in a separate directory. On how you structure that beneath is up to debate and depends a bit on your preferences.
You say that you want everything in that project group managed within a single repository. It means you can simply create a directory structure as you described with the sub-projects residing in different directories within this repository.
Within each group, you can take it further and make each of these directories (library, programme 1, programme 2, ...) a separate repository which in turn become a sub-repository to the main repository, as described in the link given by Lasse Karlsen (Subrepository).
You could also handle it differently, if you allow a more flexible layout and let go of checking out one group in its entirety: For instance you could declare the libraries a sub-repository to each of the programmes which uses the library. It would have the advantage that the programme defines this way directly which library version it depends on
Further, before jumping to sub-repositories, you might want to look at the alternative implementation of guest repositories as well. They handle the dependency less strict, thus a failure to find the sub-repository becomes less fatal: https://bitbucket.org/selinc/guestrepo
Related
I'm creating an application that is going to work on two separate versions of the same software. These frameworks will have completely different modules with a shared dependency to a JavaScript framework I've built.
Let's say
dave/version1
dave/version2
Which both share the dependency via a require of
dave/framework
I want to maintain this framework (dave/framework) in one repository that both of the parent modules can require. However the location that these framework files need to be placed are slightly different between the two modules, along with slightly different requirements for the composer.json files to ensure everything gets moved around correctly (the two versions of these software implement composer rather differently).
With my limited knowledge of composer and Git I've formulated a couple of solutions:
Create three repositories, two wrapper repositories with specific composer.json files to support each different version of the software. With another dependency to a third repository which contains the actual framework. I'm unsure if this would ever work outside of a theory. Also ends up being a little messy.
Use some form of clever tagging and have version1 and version2 depend on separate versions of the framework which would in turn have slightly different makeup. Composer would then struggle with pulling the latest version of the module as we'd be running two ever so slightly different code bases in weird versions.
However both of these seem potentially messy and the incorrect way of structuring what I'm trying to achieve.
Is there a nice way to achieve this? or am I better of maintaining two separate repositories for the framework?
With some correct planning, your first option is going to be easier than you think.
Git has a system called Submodules built right in.
Version1 and Version2 repos would include the Frameworks repo within them. Then when you update Framework, you can pull those updates no problem in Version1/2. You control when and how to, so you can make sure you don't introduce a bug down the road either.
Here is some documentation for submodules: https://git-scm.com/book/en/v2/Git-Tools-Submodules
https://git-scm.com/docs/git-submodule
In our project, we currently have two different configurations. The first one builds the assemblies. The other packages (including moving stuff to the right directories etc.) everything for InstallShield.
Now, we can't agree if it's better to move all the build steps into a single configuration and run it as a whole chain or if it's better to keep the build process separate from creating installation package.
Googling results in guides on how to do that but not in what way to do that (and our confusion is mainly due to the architecture of the configurations' order). We'll be using a few steps from PowerShield in order to move a number of files between different directories due to certain local considerations. The total number of steps will land on 5 or less.
The suggestion that I have is the following three configurations. They run separately, independently and their build steps overlap (being super sets of each other, consecutively regarded).
Configuration Build.
Configuration Build and test.
Configuration Build, test and package.
The main point of my suggestion is that e.g. the step that compiles the software is implemented in each configuration (as opposed to reusing the artifacts from an independent run of other configuration).
I would argue like this:
if you ever need to perform just one of the two steps - then leave them as separate steps.
This gives you the flexibility to run one, or the other, or both steps. E.g. could it be that you need to just build the solution, but not create the final installation package? E.g. for local testing?
However, if you never ever use one of the steps separately (you always run both together), then I'd probably just merge them together into one - having two separate steps doesn't make much sense to me
What is "vendoring" exactly? How would you define this term?
Does it mean the same thing in different programming languages? Conceptually speaking, not looking at the exact implementation.
Based on this answer
Defined here for Go as:
Vendoring is the act of making your own copy of the 3rd party packages
your project is using. Those copies are traditionally placed inside
each project and then saved in the project repository.
The context of this answer is in the Go language, but the concept still applies.
If your app depends on certain third-party code to be available you could declare a dependency and let your build system install the dependency for you.
If however the source of the third-party code is not very stable you could "vendor" that code. You take the third-party code and add it to your application in a more or less isolated way. If you take this isolation seriously you should "release" this code internally to your organization/working environment.
Another reason for vendoring is if you want to use certain third-party code but you want to change it a little bit (a fork in other words). You can copy the code, change it, release it internally and then let your build system install this piece of code.
Vendoring means putting a dependency into you project folder (vs. depending on it globally) AND committing it to the repo.
For example, running cp /usr/local/bin/node ~/yourproject/vendor/node & committing it to the repo would "vendor" the Node.js binary – all devs on the project would use this exact version. This is not commonly done for node itself but e.g. Yarn 2 ("Berry") is used like this (and only like this; they don't even install the binary globally).
The committing act is important. As an example, node_modules are already installed in your project but only committing them makes them "vendored". Almost nobody does that for node_modules but e.g. PnP + Zero Installs of Yarn 2 are actually built around vendoring – you commit .yarn/cache with many ZIP files into the repo.
"Vendoring" inherently brings tradeoffs between repo size (longer clone times, more data transferred, local storage requirements etc.) and reliability / reproducibility of installs.
Summarizing other, (too?) long answers:
Vendoring is hard-coding the often forked version of a dependency.
This typically involves static linking or some other copy but it doesn't have to.
Right or wrong, the term "hard-coding" has an old and bad reputation. So you won't find it near projects openly vendoring, however I can't think of a more accurate term.
As far as I know the term comes from Ruby on Rails.
It describes a convention to keep a snapshot of the full set of dependencies in source control, in directories that contain package name and version number.
The earliest occurrence of vendor as a verb I found is the vendor everything post on err the blog (2007, a bit before the author co-founded GitHub). That post explains the motivation and how to add dependencies. As far as I understand the code and commands, there was no special tool support for calling the directory vendor at that time (patches and code snippets were floating around).
The err blog post links to earlier ones with the same convention, like this fairly minimal way to add vendor subdirectories to the Rails import path (2006).
Earlier articles referenced from the err blog, like this one (2005), seemed to use the lib directory, which didn't make the distinction between own code and untouched snapshots of dependencies.
The goal of vendoring is more reproducibility, better deployment, the kind of things people currently use containers for; as well as better transparency through source control.
Other languages seem to have picked up the concept as is; one related concept is lockfiles, which define the same set of dependencies in a more compact form, involving hashes and remote package repositories. Lockfiles can be used to recreate the vendor directory and detect any alterations. The lockfile concept may have come from the Ruby gems community, but don't quote me on that.
The solution we’ve come up with is to throw every Ruby dependency in vendor. Everything. Savvy? Everyone is always on the same page: we don’t have to worry about who has what version of which gem. (we know) We don’t have to worry about getting everyone to update a gem. (we just do it once) We don’t have to worry about breaking the build with our libraries. […]
The goal here is simple: always get everyone, especially your production environment, on the same page. You don’t want to guess at which gems everyone does and does not have. Right.
There’s another point lurking subtlety in the background: once all your gems are under version control, you can (probably) get your app up and running at any point of its existence without fuss. You can also see, quite easily, which versions of what gems you were using when. A real history.
I'm currently looking at structuring my teams projects into a consistent manner that properly utilises packages and is easily version-controlled (via SVN).
I'm interested in any 'best practise' with regards to project structuring and how to use consistent packaging without lumping everything into a gigantic com.domainname.projects folder structure whilst maintaining that package structure. I'm also keen to use the src/bin/lib folder structure within each project.
I guess I'm asking 'how do you do it?' and 'why?'. Sorry if this is a bit abstract for Stack Overflow but you guys give the best answers I've found.
You might need to provide more information for a "complete" answer, like your shop size, number of active projects, how you build them, and how you reuse code.
Here are some insights anyway... hope I understood the direction of you question :)
If your projects are coupled and you release new versions of each reusable artifact frequently, a single svn repo keeping a folder for each should do it. Tag/branches should remain at the project level.
Having separate repos for code that is linked may become a maintainability headache.
Package1
src\com\domainname\Package1
source files
Package2
src\com\domainname\Package2
source files
Package3
src\com\domainname\Package3
source files
Lib
All packages export swcs into this folder
Problem with re-usability is going backwards -- sometimes changing something in the corelib will break all sorts of stuff going backwards, so I suggest doing running builds against it to create one large swc with all the packages in it.
I don't know why you're keen to use the bin\lib folder -- it makes more sense to have them point at things explicity rather than having parent projects export into children ...
I am currently working on automating/improving the release process for packaging my shop's entire product. Currently the product is a combination of:
Java server-side codebase
XML configuration and application files
Shell and batch scripts for administrators
Statically served HTML pages
and some other stuff, but that's most of it
All or most of which have various versioning information contained in them, used for varying purposes. Part of the release packaging process involves doing a lot of finding, grep'ing and sed'ing (in scripts) to update the information. This glue that packages the product seems to have been cobbled together in an organic, just-in-time manner, and is pretty horrible to maintain. For example, some Java methods create Date objects for the time of release, the arguments for which are updated by a textual replacement, without compiler validation... just, urgh.
I'm trying avoid giving examples of actual software used (i.e. CVS, SVN, ant, etc.) because I'd like to avoid the "use xyz's feature to do this" and concentrate more on general practices. I'd like to blame shoddy design for the problem, but if I had to start again, still using varying technologies, I'd be unsure how best to go about handling this, beyond laying down conventions.
My questions is, are there any best practices or hints and tips for maintaining and updating versioning information across different technologies, filetypes, platforms and version control systems?
Create a properties file that contains the version number and have all of the different components reference the properties file
Java files can reference the properties through
XML can use includes?
HTML can use a JavaScript to write the version number from the properties in the HTML
Shell scripts can read in the file
Indeed, to complete Craig Angus's answer, the rule of thumb here should be to not include any meta-informations in your normal delivery files, but to report those meta-data (version number, release date, and so on) into one special file -- included in the release --.
That helps when you use one VCS (Version Control System) tool from the development to homologation to pre-production.
That means whenever you load a workspace (either for developing, or for testing or for preparing a release into production), it is the versionning tool which gives you all the details.
When you prepare a delivery (a set of packaged files), you should ask that VCS tool about every meta-information you want to keep, and write them in a special file itself included into the said set of files.
That delivery should be packaged in an external directory (outside any workspace) and:
copied to a shared directory (or a maven repository) if it is a non-official release (but just a quick packaging for helping the team next door who is waiting for your delivery). That way you can make 10 or 20 delivers a day, it does not matter: they are easily disposable.
imported into the VCS in order to serve as official deliveries, and in order to be deployed easily since all you need is to ask the versionning tool for the right version of the right deliver, and you can begin to deploy it.
Note: I just described a release management process mostly used for many inter-dependant projects. For one small single project, you can skip the import in the VCS tool and store your deliveries elsewhere.
In addition to Craig Angus' ones include the version of tools used.