Everyone managing open-source-software runs into the problem, that with the time the process of releasing a new version gets more and more work. You have to tag the release in your version-control, create the distributions (that should be easy with automated builds), upload them to your website and/or open-source-hoster. You have to announce the new release with nearly the same message on chosen web-forums, the news-system on sourceforge, mailinglists and your blog or website. And you have to update the entry of your software on freshmeat. Possible more tasks have to be done for the release.
Do you developed techniques to automate some of these tasks? Does software exist that supports you with this?
Pragmatic Project Automation shows how to do all of that. They use Ant for practically everything in the book, so if you know Ant you can make different targets to do any step in the build-release cycle.
For my Perl stuff, I wrote Module::Release. In the top-level directory I type a single command:
% release
If checks several things and dies if anything is wrong. If everything checks out, it uploads the distribution.
It automates my entire process:
Test against multiple versions of Perl
Test distribution files
Check the status of source control
Check for code and distribution quality metrics
Update changes file
Determine new version number
Release code to multiple places
Tag source control with new version number
Everyone seems to write their own release automator though. Most people like their process how they like their process, so general solutions don't work out that well socially.
Brad Fitzpatrick has ShipIt which is a Perl program to automate releases. There's slightly more info in his original announcement.
Related
Following only the instructions here - https://www.chromium.org/developers/how-tos/get-the-code I have been able to successfully build and get a Chromium executable which I can then run.
So, I have been playing around with the code (adding new buttons to the browser etc.) for learning purposes. So each time I make a change (like adding a new button in the settings toolbar) and I use the ninja command to build it takes over 3 hours to finish before I can run the executable. It builds each and every file again I guess.
I have a decently powerful machine (i7, 8GB RAM) running 64-bit Ubuntu. Are there ways to speed up the builds? (At the moment, I have literally just followed the instructions in the above mentioned link and no other optimizations to speed it up.)
Thank you very very much!
If all you're doing is modifying a few files and rebuilding, ninja will only rebuild the objects that were affected by those files. When you run ninja -C ..., the console displays the number of targets that need to be built. If you're modifying only a few files, that should be ~2000 at the high end (modifying popular header files can touch lots of objects). Modifying a single .cpp would result in rebuilding just that object.
Of course, you still have to relink which can take a very long time. To make linking faster, try using a component build, which keeps everything in separate shared libraries rather than one big onw that needs to be relinked for any change. If you're using GN, add is_component_build=true to gn args out/${build_dir}. For GYP, see this page.
You can also peruse faster linux builds and see if any of those tips apply to you. Unfortunately, Chrome is a massive project so builds will naturally be long. However, once you've done the initial build, incremental builds should be on the order of minutes rather than hours.
Follow the recently updated instructions here:
https://chromium.googlesource.com/chromium/src/+/HEAD/docs/windows_build_instructions.md#Faster-builds
In addition to using component builds you can disable nacl, use jumbo builds, turn off symbols for webcore, etc. Jumbo builds are still experimental at this point but they already help build times and they will gradually help more.
Full builds will always take a long time even with jumbo builds, but component builds should let incremental builds be quite fast in many cases.
For building on Linux, you can see how to build faster at: https://chromium.googlesource.com/chromium/src/+/master/docs/linux_build_instructions.md#faster-builds
Most of them require add build argments. To edit build arguments, you can see GN build configuration at: https://www.chromium.org/developers/gn-build-configuration.
You can edit the build arguments on a build directory by:
$ gn args out/mybuild
We have a group of developers moving from C++ to C# and WinRT. We used D'Oxygen as part of our C++ developer builds, and I'd like to continue to have document generation as part of the developer build in C#/WinRT.
It's easy to turn on XML Doc generation, and I believe that will provide warnings for malformed tags, but without actual HTML output, I think our developers will be missing valuable feedback.
Looks like NDoc is now defunct, and I took a quick look at Sandcastle, but found it rather complex. Ideally, I'm looking for something that doesn't unduly burden developers, or require them to remember extra steps as they edit, build, test, and commit. In other words, the best solution would be something that "just happens", like a post-build step, and doesn't add significantly to each developer's build time.
If anyone has had some experience doing this in C#/WinRT, I'd sure like some advice.
Thanks in advance!
Get Sandcastle Help File Builder.
Create a help project for your library in the Visual Studio solution.
Remove Build check mark from Debug solution configuration to build the documentation project only in Release configurations, since Debug is most often used during development. For release build testing or performance testing you can either create another solution configuration or simply switch the option back and forth.
Build the documentation once
Include the documentation file in the solution so it shows up in the Pending Changes window when the file changes.
Kindly ask your developers to build with the release configuration that updates the documentation before check-in or use any other policy to require updating the documentation.
I don't think it makes sense to build the documentation all the time, but it helps to make it easy to do so that when you actually need an updated version - you can build it really quickly.
You can also make sure to use FXCop or StyleCop (forgot which) and configure it to treat missing XML documentation warnings as errors - at least in release builds. Doing it for debug configurations might slow down development and make changes difficult since developers often want to try things out before committing to a final implementation worth documenting.
EDIT*
Sandcastle provides various output formats as shown in the project properties:
I would like to mention ForgeDoc (of which I'm the developer), it could be what you are looking for. It is designed to be fast and simple, and it generates proper MSDN-like HTML output. It also has a command-line interface so you can just call it from a post-build event command in Visual Studio.
I think you should give it a try, as I would really like to hear about your thoughts.
In the company I am I was asked to write an autoupdate function a la chrome. I.e. It should check periodically whether a new version is available, download the new version and apply it silently the next time the application starts.
I already have something up and running but it is more like a dirty hack than something I feel happy about it. So, I would like to know how to design and implement such a solution. My horrible hack works as this:
Have a mechanism to check whether a new version exists (a database query or a web service)
Download a full zip with the whole new version.
Check file signature. If everything went alright, set a registry value: must update to true.
When the application restarts, if the must update value is true, launch an update program and exist.
The update deletes the contents of the application folder, unzips the update and replaces the old contents, launches the application and exits.
Now, I would like to change it, so it works cleaner. I am planning to send the update as a bsdiff file. It gets downloaded. But the question is, what happens next?
When do apply the update?
Who is in charge of applying the patch? is it the program itself or is it a third program, as I did, which is in charge of applying the patch and relaunch the application?
If your going down the C++ route you can go to chromium and download the Chrome source code and dig around to see how the update is done, this might give you a better idea on how to approach it. Here's an article that might help.
If your familiar with .NET the recently release nuget also has an auto update feature that might be useful to look at, you can get the source code from here. David Ebbo has a blog about how its done here.
I'm not up to date on Delphi but you might be able to use either of the above options.
The workflow you proposed is more or less like it should work, but there's no need to re-invent the wheel - there are plenty libraries out there that will do this for you. Using a 3rd party library has the benefit of keeping your code cleaner while making sure the dirty process of auto-update is contained and working flawlessly.
Trust me, I know. I'm the author of NAppUpdate, an app update framework for .NET (which you might want to try out or learn from).
So, after giving it a lot of though, this is what I came with (for active directory I will refer to the directory where the main program lies, active program is the main program and update program is the one that replaces the active program and its resource files):
The active program checks if there is a new version every certain amount of time. If so, download it
Prepare new version in a separate folder (this can be done by copying the contents of the directory with the program to a subdirectory and applying a binary patch, or simply unziping the new version).
Set a flag that indicates that a new version is ready.
When a program is exiting (and one has to control for different interrupts here):
The active program checks the new version ready flag. Launch the update program and exit.
The update program checks if it can write in the active directory. If so, replaces the contents with the prepared version.
The update program has to recheck links and update them accordingly.
So guys, if you have a better workflow, please tell me.
You could literally use the Google Chrome update workflow by using the Google Chrome updater:
http://code.google.com/p/omaha/
They open sourced it Feb 2009.
I want to give credit to all open source libraries we use in our (commercial) application. I thought of showing a HTML page in our about dialog. Our build process uses ant and the third party libs are committed in svn.
What do you think is the best way of generating the HTML-Page?
Hard code the HTML-Page?
Switch dependency-management to apache-ivy and write some ant task to generate the html
Use maven-ant-tasks and write some ant task to generate the HTML
Use maven only to handle the dependencies and the HTML once, download them and commit them. The rest is done by the unchanged ant-scripts
Switch to maven2 (Hey boss, I want to switch to maven, in 1 month the build maybe work again...)
...
What elements should the about-dialog show?
Library name
Version
License
Author
Homepage
Changes made with link to source archive
...
Is there some best-practise-advice? Some good examples (applications having a nice about-dialog showing the dependencies)?
There are two different things you need to consider.
First, you may need to identify the licenses of the third-party code. This is often down with a THIRDPARTYLICENSE file. Sun Microsystems does this a lot. Look in the install directory for OpenOffice.org, for example. There are examples of .txt and .html versions of such files around.
Secondly, you may want to identify your dependencies in the About box in a brief way (and also refer to the file of license information). I would make sure the versions appear in the About box. One thing people want to quickly check for is an indication of whether the copy of your code they have needs to be replaced or updated because one of your library dependencies has a recently-disclosed bug or security vulnerability.
So I guess the other thing you want to include in the about box is a way for people to find your support site and any notices of importance to users of the particular version (whether or not you have a provision in your app for checking on-line for updates).
Ant task seems to be the best way. We do a similar thing in one of our projects. All the open source libraries are present in a specified folder. An Ant task reads the manifest of these libraries, versions and so on and generates an HTML, copies into another specified folder from where it is picked up by the web container.
Generating the page with each build would be wasteful if the libraries are not going to change often. Library versions may change, but the actual libraries don't. Easier to just create a HTML page would be the easiest way out, but that's one more maintenance head ache. Generate it once and include it with the package. The script can always be run again in case some changes are being made to the libraries (updating versions, adding new libraries).
I am currently working on automating/improving the release process for packaging my shop's entire product. Currently the product is a combination of:
Java server-side codebase
XML configuration and application files
Shell and batch scripts for administrators
Statically served HTML pages
and some other stuff, but that's most of it
All or most of which have various versioning information contained in them, used for varying purposes. Part of the release packaging process involves doing a lot of finding, grep'ing and sed'ing (in scripts) to update the information. This glue that packages the product seems to have been cobbled together in an organic, just-in-time manner, and is pretty horrible to maintain. For example, some Java methods create Date objects for the time of release, the arguments for which are updated by a textual replacement, without compiler validation... just, urgh.
I'm trying avoid giving examples of actual software used (i.e. CVS, SVN, ant, etc.) because I'd like to avoid the "use xyz's feature to do this" and concentrate more on general practices. I'd like to blame shoddy design for the problem, but if I had to start again, still using varying technologies, I'd be unsure how best to go about handling this, beyond laying down conventions.
My questions is, are there any best practices or hints and tips for maintaining and updating versioning information across different technologies, filetypes, platforms and version control systems?
Create a properties file that contains the version number and have all of the different components reference the properties file
Java files can reference the properties through
XML can use includes?
HTML can use a JavaScript to write the version number from the properties in the HTML
Shell scripts can read in the file
Indeed, to complete Craig Angus's answer, the rule of thumb here should be to not include any meta-informations in your normal delivery files, but to report those meta-data (version number, release date, and so on) into one special file -- included in the release --.
That helps when you use one VCS (Version Control System) tool from the development to homologation to pre-production.
That means whenever you load a workspace (either for developing, or for testing or for preparing a release into production), it is the versionning tool which gives you all the details.
When you prepare a delivery (a set of packaged files), you should ask that VCS tool about every meta-information you want to keep, and write them in a special file itself included into the said set of files.
That delivery should be packaged in an external directory (outside any workspace) and:
copied to a shared directory (or a maven repository) if it is a non-official release (but just a quick packaging for helping the team next door who is waiting for your delivery). That way you can make 10 or 20 delivers a day, it does not matter: they are easily disposable.
imported into the VCS in order to serve as official deliveries, and in order to be deployed easily since all you need is to ask the versionning tool for the right version of the right deliver, and you can begin to deploy it.
Note: I just described a release management process mostly used for many inter-dependant projects. For one small single project, you can skip the import in the VCS tool and store your deliveries elsewhere.
In addition to Craig Angus' ones include the version of tools used.