Recently I've been revisiting an old project, which I last worked on about two years ago. Obviously, during this time I've learned new habits about how best to program, and I've got the itch to keep the tests, scrap the implementation, and re-implement the entire project. It's not a large project, and I believe I'll not be losing much by re-writing it.
However, I don't know what to do about the version history. It's likely that when I'm done updating it, the new version will share only 3-4% of its code with the old version. Furthermore, the changes tend to be so wide-reaching that trying to maintain clean changesets is an exercise in frustration and futility. Given this, it seems unnecessary to force potential developers to download the old irrelevant versions.
One option I've been considering is to move the trunk to a branch, something like old-trunk/, and begin development in an empty branch. I don't know if this is a good idea, and I'm concerned that having two trunks could lead to confusion. Which brings me to the question:
What does SO think? If you encountered a project that had "reset" its trunk, would you be confused by it?
Why not just label/tag the trunk with "OldVersion" and continue development in the same location? This way you avoid the double branch altogether and still maintain ways to get to the old version of your code. Unless you're developing a different product, you likely want to keep the same trunk.
I tag the $trunk, so there is a "copy". Tag it the last version released, or the date before you started back on it, etc. I actually tag on the date released, and name my tags the actual date itself. (use Label inplace of tag if that is what your system supports).
After it is tagged, delete/rename/overhaul as needed. The version history is there, just in case. And, you have a complete copy labelled/tagged for archiving purposes.
Assuming you're using SVN, there's really nothing extra to do. Just remember the revision from which you started refactoring and then continue in working on the trunk. It's better than moving to an empty branch since the history of the changes from the old code to a new one will be recorded.
However, if you plan to write the thing from scratch and just copy some little bits of old code, maybe you should think about starting it in a new repository.
I don't see the need to throw the old version away. Just make all changes and check in. New developers will not have to download any old code, they will always just check out the new version.
No, it wouldn't confuse me because the project documentation would show that before I even got into it. We ALL have crap that needs to be redone, and frankly, should be.
Tell everyone it's insecure and not HIPAA compliant and they'll let you re-code it ;o)
And then start posting the fodder onto The Daily WTF for the rest of us 80))
Related
Very soon we are going to start on open-source (py+qt) project which is supposed to be multi-platform (we're using FreeBSD as native platform) and we're not sure which DVCS/hosting to use.
In the past we were using darcs for very long time, but moved away from it due to not having adequate public hosting available. Played for some time with Monotone - nicely designed, but mostly niche today. Fossil is nice, but it uses non-standard wiki and its tracker is functional, but we expect more.
Considering that we won't work on kernel-like sized project we do not nee Git which we consider too complex to deal with it, especially for potential contributors which might use Windows OS and prefer GUI tools.
So, the story comes to Bazaar/Launchpad and Mercurial/Bitbucket...
Here are some pro/cons which we gathered together, but would like to hear if we missed something which might help us to decide...
Bazaar pro/cons:
2.4 is probably quick enough for our needs ,
simple to use,
has nice GUI tool (explorer),
handles empty directories,
(probably) less popular than Mercurial,
does not have equivalent of hg's named branches
The last point is probably not to important 'cause there are nicks and there is colo-branches plugin, so one can get same/similar functionality.
The most problematic quirk we find in Bazaar it its revision numbers scheme and problem which can arise if one pushes from feature branch into upstream which would change revids.
Maybe it's a lesser problem when using Launchpad...
As far s Launchpad is concerned:
- it has very nice bug tracker with email interface
- it's (maybe) more project-oriented than Bitbucket
- no private repos as with Bitbucket
- no wiki for projects - bug (https://bugs.launchpad.net/launchpad/+bug/240067) is more than 3 years old and still with 'Low priority'. LP is the only one amongst {LP,Sourceforge,Bitbucket, Google, Github} which lacks this feature and it really sucks and degrades, otherwise, nice hosting solution.
What we've found in The other camp...
Mercurial is:
(probably) more popular than Bazaar,
quick,
simple to use,
there is nice TortoiseHG for non cli-savvy users,
we like named branches,
some quirks like handling empty directories (or https://www.mercurial-scm.org/bts/issue29)
However, what we like the most over Bazaar is, as we believe, great merging capabilities without the hassle of changed revids due to revno:hash schema.
As far as Bitbucket:
we like to have unlimited/private repos
we like having wikis available for the project(s)
we miss email interface for the tracker and the tracker is (maybe) not on par with the one at LP (reviews etc.)
At the end, let's say that there are some projects which we are interested in which are under Git #github, so we would like to use single DVCS which can helps us inter-operate with git#github projects.
We find that bzr-git plugin is very capable and do not have experience with hg-git.
Although there is bzr-hg plugin (not as mature as bzr-git), but we do not know about something like hg-bzr except hg's convert extension which does the job of hg-bzr conversion.
Is there any important feature which we did miss having important consequence in deciding about the two?
Finally, we use DVCS for all our needs (simple project, writings...) and we'd prefer to settle on one DVCS/hosting which can serve all our purposes and be useful in contributing to git(hub) projects as well.
What do you recommend?
In Bazaar:
You can avoid the problem of revision numbers being renumbered by setting append_revisions_only in branch.conf, which will make sure people only merge into trunk, rather than switching the trunk around.
I like bzr-colo a lot for dealing with named colocated branches.
I would certainly like to see Launchpad get wikis. It's assigned and in progress at the moment so perhaps it will get done soon.
Update: Seing this comment makes it easier for us to abandon bzr/LP and embrace hg/bitbucket.
Is it a good policy to automate source control commits following successful builds?
Edit: I'm asking because I want more frequent, incremental commits between versions that make it easier to find the point where a bug was introduced than rolling back 2K+ new lines of code between v1.0 and v1.1.
No. A successful build does not imply a successful code change. Do you never test your code? IF you had some sort of automated unit testing, I could understand the question (although I would still recommend against it - I would not consider a code change tested until you verified its functionality yourself). But an automated commit after a successful build - not if you like your team mates, or if they have access to weapons.
No. Where would the meaningful commit messages come from? And the references to issue tracker items? How is the automated process supposed to know that a particular bit of work is complete?
With such a process in place, your repository would degrade to a glorified IDE undo buffer.
Yesterday when i checked out the latest version of our internal tool i saw about 30+ new versions. This got me curious since i thought that somebody finally fixed those annoying bugs and added that feature i was waiting for so long... and guess what? None of this happened, someone just thought it would be nice to update some headers and do a minor adjustment of two or three functions. Everything in a separate commit. Great.
This raised a discussion in our team - should this be considered ok, or should we prohibit such "abuse"? Arguably this really could fit in one or two commits, but 30 seems to much. How should this be handled - what is the best practice?
You should be committing any time you make a change and are about to move on to the next one.
You shouldn't commit anything that stops the project from building.
You should be filling in the commit message so people know what changes have been made.
That'll do for me.. I don't assume something has been done unless I see it in the commit message...
Generally I think a commit should relate to one logical task, e.g. fixing bug #103 or adding a new print function. This could be one file or several, that way you can see all changes made for a particular task. It is also easier to roll back the change if necessary.
If each file is checked in one by one, it is not easy to see the changes made for a particular update / task.
Also if multiple tasks are completed in one commit, it is not easy to see what changes belong to which task.
I wouldn't care about the number of commits as each commit keeps project consistency (build will still succeed). This is some internal count that shouldn't bother you. If you want to change something here, better tell people to use some structured commit messages (like "[bugfix]...", "[feature]...", "[minorfix]").
By the way, if you want to know if bugs have been fixed or features have been added, using a bug tracing system is much better than checking commits in a SVN-like tool.
The battle against code entropy is an ongoing team effort. Minor checkins where one just 'fixes broken windows' along ones way should be encouraged, not frowned upon. The source repository is the wrong tool for keeping track of bugfixes - that's what a bug tracker is for - so the inconvenience in locating fixes when scanning the code repository and not the bug repository seems utterly negligible to me.
I work in a moderate size team on a large code base (~1M LOC) with a huge history (~20Y). A lot of the code is a pile of mess - rotten branch logic, deprecated API, naming conventions, even random indentation often makes it a misery to read. I started a habit of minor "drive-by" readability improvements, to try and fight complete code rot, and am trying hard to get teammates to adopt the same habit.
Unless your circumstances are radically different, I would try and think favorably on any such initiative. The alternative (which I'm familiar with all to well) is fearful stagnation, which dooms any code to rot.
Later this year I want to release a PHP framework that I've been working on as open source. I do use source control (SVN), but it's on an extremely limited basis. I'm self-taught, I develop by myself and don't have the experience of working with large teams. I have some ideas about what can help make a project successful, but I'm fuzzy on some of the details. Since it's not yet released, I want to do everything I can to set up the right infrastructure from the beginning. What do I need to know in order to setup and manage a successful project?
Some ideas that I have to make it successful (beyond marketing it):
Good documentation and tutorials
Automated unit tests and builds to
push update to the website
A clear roadmap
Bug Tracking integrated with the
source control
A style guide to keep the code
consistent
A forum for the community to get
support, share ideas, etc.
A good example application built with
the framework
A blog to keep the community informed
Maintaining backwards compatibility
wherever possible
Some of my questions:
How do I setup and automate a one
step submit-test-commit-generate API
docs-push update to website process? Edit: Would Ant or Maven be good candidates for this? If so, do you know of any resources for setting up a PHP project using them?
How do I handle (technically)
submissions from other users? How can
I ensure that those submissions must
be approved before being integrated?
What are some of the pitfalls that
can be avoided in terms of the
project community? I'd prefer to have
it be as friendly and helpful as
possible without a lot of drama.
I'd love to learn from your experience on any of these points. If you think I'm missing anything big, please share that as well. Any resources (preferably geared toward a beginner) that you could point me towards would also be greatly appreciated.
I'm just getting started in community projects, but I'll give you some advice on what I know.
How do I setup and automate a one step submit-test-commit-generate API docs-push update to website process?
I've never implemented it as one process. You could just have a checklist, and possibly even create some scripts to do certain tasks. I've never worked with any source control that automates the uploading and such to be done by a script. Most of the time, there is some web interaction to be involved.
You don't want to push API changes until it's an official release.
EDIT: Working Environment
For PHP, most of the time, I either edit directly on the server and test it there, using a beta.example.com, or similar, before pushing to example.com. You could also set up an web environment on your home PC (using XAMPP for Windows, or the standard LAMP installation on Linux). You would probably just use a mirror of your repository here, so you'd do svn commit, or whichever is appropriate for the VCS or DVCS you choose.
The fun part is testing this with different PHP versions. I've not done this myself, but you could probably use a .htaccess file to run a different PHP binary in order to test it out. I'm not really sure what the best option is for this is.
I've not done much with API, as I've never created a library, but just doing a quick search I found http://www.phpdoc.org/. It looks like a mature project, so that might be a starting point.
As far as creating releases go, I generally create a script that only includes the files that are part of the distribution (it will filter out any VCS files, and anything that you don't want in the distributed file). You could write a script around find on linux (which is what I do most of the time), or there may be other better options.
How do I handle (technically) submissions from other users? How can I ensure that those submissions must be approved before being integrated?
This is mostly handled by the bug tracker, and limited access in the Version Control System. Usually, you, and the people you allow, can commit to the VCS. Other users can submit patches, but then you might have someone review the patch, test the patch, and commit. You could split these tasks up as a team, or assign a patch to one person and have them do it all.
What are some of the pitfalls that can be avoided in terms of the project community? I'd prefer to have it be as friendly and helpful as possible without a lot of drama.
I would just make sure to keep it as positive as possible with the project members and community. There's going to be some disagreements, and it will drive a few people away, but as long as you have a stable product that meets the needs of most people, I think that's all that anyone can expect.
One minor suggestion that's worked well for me: start using first-person plural pronouns, rather than singular ones. That is, talk about "we" and "us" rather than "I" and "me." It encourages other people to participate when they feel like part of team, rather than when they feel like they're contributing your own self-aggrandizement.
The most important thing you have to do is to attract users. Without users, you won't get any contributions and developers helping you out. Because developers are users first, and then they decide to extend/fix something they use and might become contributors.
So to get users, you should consider
describe what your framework does in one or two sentences at the top of your project page
mention how your framework can be used and for what, what situations it is most useful for
add a lot of examples on how to use it
mention whether your framework is stable, beta or alpha. That's important because user need to know that before they start using it
also mention whether you want to keep improving it and keep working on it - most users don't want to use a framework that's abandoned (also keep in mind that a lot of users check your commits to see whether you really are working on it - if your last commit to the repository was months ago then you're not really working on it, so cheating isn't possible)
If you got all this, and people start submitting patches, you can use a patch tool to apply those to your source. Depending on your version control system, you can either use the GNU patch, a diff/patch tool that comes with your version control or maybe even a GUI tool that helps you with this. SVN doesn't have a patch tool (yet), but 'svn diff' will create a patchfile which you can then apply with the GNU patch tool, or in case you're using TortoiseSVN, right-drag the patchfile to your working copy and have TortoiseMerge apply it for you.
And on how to best deal with the community:
answer questions in time, don't wait more than two or three days to answer questions
try to be nice, even with upset and angry people. Only if they keep bothering tell them to (still in a nice way if possible) go elsewhere
always keep discussions about the project on a mailing list. You don't want to repeat the same discussions over and over again - if you have a mailing list, just point users to the archives before the discussion starts all over again
And you should watch the talk "How Open Source Projects Survive Poisonous People (And You Can Too)" - it's really good and tells you a lot on how to deal not just with 'poisonous people' but also how to deal with all people involved in your project.
I'd like to add that you should make it as easy as possible for your users to get the whole thing running and modify the code - these 'power users' can be 'converted' into developers or at least people who send smaller patches.
Don't try to do it all yourself - for open source projects there are several hosting providers that solve most of the problems. I recommend codeplex or google code.
Setting up build scripts will depend a certain amount on what platform you set up, but in general it's easy to add any tool you want into the script once you start using any sort of build script.
If you really need the one step process you describe, you need a build server. I use TeamCity, which I have set up to watch for any changes in svn and trigger build/test whenever something is checked in. The build server will generally be able to perform any steps that you put into the build script.
Read up on Git as an alternative to SVN
free public repository/bug tracker/wiki/fork-enabled community in Github (which hosts symfony and PHPUnit amongst others)
"How do I handle (technically) submissions from other users? How can I ensure that those submissions must be approved before being integrated?" - with Git, pull what you/your closest team finds most interesting to the master branch
Consistent API
be inspired of other public API:s
only change in major versions
guessable
Interesting for both users & developers
clear goal (your roadmap - excellent)
useful, contra everything else available
easy to use, but still not easy-enough-to-write/maintain-yourself
You could check out either Ant or Phing to build your project. Include CodeSniffer in the build and you'll save time checking for basic formatting errors/differences.
These are all technical tips, about the soft part... treat humans with respect, a lot of interest and be overly excited about their contributions and make them feel that they're not wasting their time. That would appeal to me.
Take a look at Karl Fogel's book on Producing Open Source Software. It probably has everything that you asked.
You should also plan for engaging the community. I'd recommend reading Jono Bacon's The Art of Community [http://www.artofcommunityonline.org/].
You have a great set of ideas to start. You might have to start by trimming them down! Ask yourself what's necessary for a first release.
For automating the builds and tests, the scripting can be done with ant, maven or phing for PHP projects.
You'll probably need a host so you can demo the product. For PHP that is pretty easy to find.
You need an open source hosting provider-- especially github (but also google code, source forge, etc). Github provides bug tracking, default licenses, blog and great mechanisms for accepting changes from the community. Built on git, it facilitates distributed projects quite well.
Although it's good to have a one-step build and install in place, automating integration of others changes probably isn't important (or desirable) off the bat.
Good luck!
Context: I work at a small software company that has traditionally done research-type work, and does not have much experience in the commercial space. We are now trying to push into the commercial world. Due to our origins in research we are used to a very rapid development cycle and very little structure in terms of maintaining proper versions of projects.
Problem: The lack of structure is now proving to be somewhat of a hindrance, as every developer has a slightly different view of the code base. A problem one developer discovers is not reproducible by another developer, and problems found in one build may disappear in the next (or worse, new problems may appear). This makes for a very frustrating experience for someone who is responsible for integrating all the projects and ensuring quality and performance standards are met - i.e. myself.
Potential solution: Personally I am convinced we need to enforce better structure via fixed version numbers and regular releases. It should be self-evident how proper versioning would help with many of our problems, but of course it is not without problems - developers need to do extra work to perform and test releases, and will no longer be able to use the latest versions of everything.
Question: To come to a point - what sorts of strategies do you recommend for ensuring the process and effort required for releases occurs as smoothly as possible? We are using git for version control, maven for our build system, and we have bug tracking and continuous integration systems running, so I believe the tools are there. I am simply unsure about what a proper release process should look like.
You have the big three in place: version control, one-click build via Maven and your continuous build server, and bug tracking. It sounds like you guys are gravitating towards Agile methodologies, and so you ought to be trying to keep the trunk version of your product in a near deliverable state at all times.
When you decide to make your first release, create a branch off of your trunk version for that release. Decide on a labelling scheme and be sure to label the branch version. For example, your first release could be 1.0.4530, where the 1 means first version, the 0 means it's the first release candidate, and the 4530 is the version control change number. You test this release branch and fix important bugs on it. After a while you issue another release candidate, say 1.1.4807. This process iterates a couple more times (say), your release becomes good enough, and you ship version 1.3.5167.
Meanwhile, your new development occurs only in the trunk version, and from time to time you'll need to merge bug fixes from the 1.x release branch back to the trunk. Later, you'll split off a 2.x branch from the trunk to repeat the process for your second release. You'll generally have several active branches (plus the trunk), with development limited to the trunk and each branch kept pristine and independent from development.
You guys will get the hang of things and your developer coordination problems will become less frequent. But these problems are nearly all going to be limited to the trunk, not the release branches.
A problem one developer discovers is
not reproducible by another developer,
and problems found in one build may
disappear in the next (or worse, new
problems may appear). This makes for a
very frustrating experience for
someone who is responsible for
integrating all the projects and
ensuring quality and performance
standards are met - i.e. myself.
Potential solution: Personally I am
convinced we need to enforce better
structure via fixed version numbers
and regular releases.
I don't think you need to have very frequent releases just to coordinate internally. You can do that through version control. Just have people talk about specific git revisions when reporting issues. Also note that you will have to coordinate any external dependencies/libraries too. Some kind of vendor branches could help with this.
It sound like the developers need to use "test branches" and respect the "stable/production branch" a little bit more.
Sell in the concept of "do your wild west stuff in this branch", and when you are happy with the results then you merge it into this "boring stable production branch"....
(or something like that)
There are books written about the general topic; Amazon search even returns three titles for specialized "version control with git."
I think you will benefit from defining a canonical view of the code base. Call it Test. A problem is a problem if it appears in Test. If a problem does not appear in some developer's view, it is up to that developer to figure out what is the important difference; and likewise for a problem that appears in a developer's view, but not in Test.
One convention is for Test to be re-built from sources on a nightly basis. A more strenuous convention is for Test to be re-built upon every update. If your team is small (five or fewer) and not dispersed over great distances or multiple timezones, a reasonable first approximation is to make Test a git workspace on a server upon which your toolchain has been installed along with some cron jobs so that this workspace is updated and rebuilt every night (usually).