Related
I'm trying to generate a VxWorks 6.9.4.8 kernel configuration that is identical to another kernel workbench project. The Workbench 3.3.6 only allows GUI configuration.
Is there an underlying kernel configuration file, produced by the GUI, which can be replaced?
After updating the kernel configuration using the Workbench GUI, I see the following files have changed:
linkSyms.c,
prjComps.h,
prjConfig.c, and
prjParams.h
I guess my question is, which one, if any uniquely identifies the kernel as built?
prjComps.h will contain all the component's names, as you have chosen in your kernel configuration GUI.
First step to create new Kernel configuration based on some other Kernel configuration is to use GUI configurator and add the missing component in prjComps.h, Better use some diff tool like 'beyond compare', and keep reducing the differences by adding/removing the components. Remember not to edit this file directly, but via GUI configurator only. As the tool calculates the dependent component and adds/removes them.
Second step is to create the new prjParams.h as above.
The Workbench actually allows to use command line to edit Kernel configuration via vxprj tool in vxworks 6.9(this tool has been replaced by "wrtool" in vxworks 7), you can right click on the Image project and chose 'Open Wind River vxWorks 6.9 Developement Shell'.
If you want to add a component for e.g. telnet client (INCLUDE_TELNET_CLIENT)
, you can use the following command
vxprj component add INCLUDE_TELNET_CLIENT
To remove a component
vxprj component remove INCLUDE_TELNET_CLIENT
For more of vxprj tool, you can look up the documentation in the workbench itself.
The project configuration is held in a handful of files in the kernel project directory.
These are:
.project
.cproject
.wrproject
projectname.wpj
Files such as prjComps.h, prjParams.h prjConfig.c are all generated by the configuration tool, however these are not configuration files themselves. Instead, this is generated C code that contains, amongst other things, a list of selected components.
These files are also re-generated, I believe, when you rebuild the project.
As such, these are not really the authoritative source you are interested in.
For this, you need to look at the project files. In terms of a list of components, the most interesting is the .wpj file, which contains amongst other things a list of explicitly and implicitly included components.
The explicitly included components are those you manually selected in the Kernel Configuration GUI, the implicitly included are those that were then included to satisfy dependencies.
This distinction can sometimes make comparing kernel configurations tricky, then you may want to fall back on the generated files eg prjComps.h, however you should always remember that this is a representation of the configuration, not the source.
The .project etc configuration files are big and complex, but a decent diff tool, such as BeyondCompare can make comparisons of the project directories fairly easy
Thanks for the clue, #endTunnel. I looked at that file, and noticed that a few files get modified when I save my GUI selections.
prjComps.h - all the components #included in the kernel build
prjParams.h - the additional parameters set for the enabled components
prjConfig.c - the configuration and initialization calls for each module included.
'linkSyms.c' also gets modified. Not sure how that is used, yet.
I can now use diff to compare kernel configurations, and perhaps even duplicate a configuration (haven't tried that yet).
The issue that I consider is how to write code that can easily know the location of a required config file and yet is portable, without any edit, from an environment to another. We don't want to edit the location of the configuration file to adapt the code to each new environment, say each time we move the code from a development environment to production. The method should not rely on resources that are not universally available, such as an access to user-defined environment variables or an access to a specific directory. For example, it may seem that using the DOCUMENT_ROOT as a base location for the config file is the way to go, but that is not universal. First, in a command line environment the DOCUMENT_ROOT makes no sense. Second, a programmer might be given access to a sub-folder of the DOCUMENT_ROOT only. Another requirement is that the configuration file could depend on values known at run time, say the user who call the application, as in this question How to load a config file based on user selection from "unknown" location .
The question is not what is the best location of the configuration file in specific environments, such as Location to put user configuration files in windows . The programmers would still have to figure out the best location so that end users could easily find the configuration file. The question is how this location, whatever it is, even if it depends on values known at run time, can be passed to the code in a portable manner.
One approach is to design any script file with in mind that it is to be included in another file and so on until we get to a wrapper script that only defines the directory of the config file to the benefit of the included file and other included files therein. Once this directory path is known, other configuration values can be obtained from a named configuration file within it. This works because the wrapper scripts are not updated when we update the code from a repository or testing environment. This approach seems universally applicable : no special support of any kind such as an access to user defined environment variables or to some specific directory in the server is needed. As long as you have access to the code, which is a strict minimum to expect, it works. Also, scripts are often naturally designed to be included in another file - so it is natural.
The approach only requires that we agree on a convention for the name of the constant, say CONFIG_DIRECTORY. If every programmer would agree to search at the location specified by this constant for the config file, then any user of the code could put the config file anywhere and just define this constant accordingly.
In Linux, they have the folder /etc for config files. So, the notion of an universally agreed standard in a very large context is already there. This is the same idea than the one proposed here, except that it is the same constant for all machines and someone might not have access to that level of the server. Moreover, we lose the possibility to have different configuration directories for different wrapper scripts. Allowing the universal standard to be a constant name, say 'CONFIG_DIRECTORY', instead of being the fixed constant '/etc', seems just an extra flexibility with no additional inconvenient. It does require that we define this constant in some wrapper script, but we could fall back to the old approach if it is not defined. The outcome, if the approach is strictly applied, would be that all the scripts required in the server document root would only be simple wrappers that define a configuration directory. That seems cool. Often people say that it is safer to have important code outside the document root.
What is a good way to coordinate configuration changes through environments?
In an effort to decouple our code from the environment we've moved all environmental config to external files. So maybe the application will look for ${application.config.dir}/app.properties and app.properties could contain:
user.auth.endpoint=http://some.url/user
user.auth.apikey=abcd12314
The problem is, user.auth.endpoint needs to point to a test resource when on test, a staging resource when on the staging environment, and a production resource when on prod.
We could maintain different copies of the config file but this would violate DRY and become very unwieldy (there are 20+ production environments).
What's a good way to manage this? What tools should I be searching for?
Externalizing config is a good idea, you could externalize them all the way to environment variables.
Env vars are easy to change between deploys without changing any code;
unlike config files, there is little chance of them being checked into
the code repo accidentally; and unlike custom config files, or other
config mechanisms such as Java System Properties, they are a language-
and OS-agnostic standard.
From http://12factor.net/config
I know of three approaches to this.
The first approach is to write, say, a Python "wrapper" script for your application. The script will find out some environmental details, such as hostname, user name and values of environment variables, and then construct the appropriate configuration file (or a set of command-line options) that is passed to the real application.
The second approach is to embed an interpreter for a scripting language (Python, Lua and Tcl come to mind) into your application. This makes it possible for you to write a configuration file in the syntax of that embedded scripting language. In this way, the configuration file can make use of the scripting language's features, such as the ability to query environment variables or execute a shell command (such as hostname) and use if-then-else statements to set variables appropriately.
The third approach (if you are using C++ or Java) is to use the open-source Config4* library (disclaimer, I am the main developer of that). I recommend you read Chapter 2 of the "Config4* Getting Started" manual to see examples of how its flexible syntax can enable a single configuration file adapt to multiple environments.
You can take a look at http://www.configapp.com. You work with 1 configuration file, and switch/tab between the environments. Internally it's just 1 configuration file, and it handles the environment variables and generates the config file for the specific environment. In Config terminology, you have 1 Prod environment with 20+ instances. You will have a Prod environment configuration and you can tweak the 20+ instances accordingly using a web interface.
You moved environment specific properties to a separate file, but with Config, you don't have to do that. With Config, you can have 1 configuration file, with environment variables support, and common configuration applied to all environments.
Note that I'm part of the Config team.
It would be nice to have a more or less complete list over what files and/or directories that shouldn't (in most cases) be under source control. What do you think should be excluded?
Suggestion so far:
In general
Config files with sensitive information (passwords, private keys etc.)
Thumbs.db, .DS_Store and desktop.ini
Editor backups: *~ (emacs)
Generated files (for instance DoxyGen output)
C#
bin\*
obj\*
*.exe
Visual Studio
*.suo
*.ncb
*.user
*.aps
*.cachefile
*.backup
_UpgradeReport_Files
Java
*.class
Eclipse
I don't know, and this is what I'm looking for right now :-)
Python
*.pyc
Temporary files
- .*.sw?
- *~
Anything that is generated. Binary, bytecode, code/documents generated from XML.
From my commenters, exclude:
Anything generated by the build, including code documentations (doxygen, javadoc, pydoc, etc.)
But include:
3rd party libraries that you don't have the source for OR don't build.
FWIW, at my work for a very large project, we have the following under ClearCase:
All original code
Qt source AND built debug/release
(Terribly outdated) specs
We do not have built modules for our software. A complete binary is distributed every couple weeks with the latest updates.
OS specific files, generated by their file browsers such as
Thumbs.db and .DS_Store
Some other Visual Studio typical files/folders are
*.cachefile
*.backup
_UpgradeReport_Files
My tortoise global ignore pattern for example looks like this
bin obj *.suo *.user *.cachefile *.backup _UpgradeReport_Files
files that get built should not be checked in
I would approach the problem a different way; what things should be included in source control? You should only source control those files that:
( need revision history OR are created outside of your build but are part of the build, install, or media ) AND
can't be generated by the build process you control AND
are common to all users that build the product (no user config)
The list includes things like:
source files
make, project, and solution files
other build tool configuration files (not user related)
3rd party libraries
pre-built files that go on the media like PDFs & documents
documentation
images, videos, sounds
description files like WSDL, XSL
Sometimes a build output can be a build input. For example, an obfuscation rename file may be an output and an input to keep the same renaming scheme. In this case, use the checked-in file as the build input and put the output in a different file. After the build, check out the input file and copy the output file into it and check it in.
The problem with using an exclusion list is that you will never know all the right exclusions and might end up source controlling something that shouldn't be source controlled.
Like Corey D has said anything that is generated, specifically anything that is generated by the build process and development environment are good candidates. For instance:
Binaries and installers
Bytecode and archives
Documents generated from XML and code
Code generated by templates and code generators
IDE settings files
Backup files generated by your IDE or editor
Some exceptions to the above could be:
Images and video
Third party libraries
Team specific IDE settings files
Take third party libraries, if you need to ship or your build depends on a third party library it wouldn't be unreasonable to put it under source control, especially if you don't have the source. Also consider some source control systems aren't very efficient at storing binary blobs and you probably will not be able to take advantage of the systems diff tools for those files.
Paul also makes a great comment about generated files and you should check out his answer:
Basically, if you can't reasonably
expect a developer to have the exact
version of the exact tool they need,
there is a case for putting the
generated files in version control.
With all that being said ultimately you'll need to consider what you put under source control on a case by case basis. Defining a hard list of what and what not to put under it will only work for some and only probably for so long. And of course the more files you add to source control the longer it will take to update your working copy.
Anything that can be generated by the IDE, build process or binary executable process.
An exception:
4 or 5 different answers have said that generated files should not go under source control. Thats not quite true.
Files generated by specialist tools may belong in source control, especially if particular versions of those tools are necessary.
Examples:
parsers generated by bison/yacc/antlr,
autotools files such as configure or Makefile.in, created by autoconf, automake, libtool etc,
translation or localization files,
files may be generated by expensive tools, and it might be cheaper to only install them on a few machines.
Basically, if you can't reasonably expect a developer to have the exact version of the exact tool they need, there is a case for putting the generated files in version control.
This exception is discussed by the svn guys in their best practices talk.
Temp files from editors.
.*.sw?
*~
etc.
desktop.ini is another windows file I've seen sneak in.
Config files that contain passwords or any other sensitive information.
Actual config files such a web.config in asp.net because people can have different settings. Usually the way I handle this is by having a web.config.template that is on SVN. People get it, make the changes they want and rename it as web.config.
Aside from this and what you said, be careful of sensitive files containing passwords (for instance).
Avoid all the annoying files generated by Windows (thumb) or Mac OS (.ds_store)
*.bak produced by WinMerge.
additionally:
Visual Studio
*.ncb
The best way I've found to think about it is as follows:
Pretend you've got a brand-new, store-bought computer. You install the OS and updates; you install all your development tools including the source control client; you create an empty directory to be the root of your local sources; you do a "get latest" or whatever your source control system calls it to fetch out clean copies of the release you want to build; you then run the build (fetched from source control), and everything builds.
This thought process tells you why certain files have to be in source control: all of those necessary for the build to work on a clean system. This includes .designer.cs files, the outputs of T4 templates, and any other artifact that the build will not create.
Temp files, config for anything other than global development and sensitive information
Things that don't go into source control come in 3 classes
Things totally unrelated to the project (obviously)
Things that can be found on installation media, and are never changed (eg: 3rd-party APIs).
Things that can be mechanically generated, via your build process, from things that are in source control (or from things in class 2).
Whatever the language :
cache files
generally, imported files should not either (like images uploaded by users, on a web application)
temporary files ; even the ones generated by your OS (like thumbs.db under windows) or IDE
config files with passwords ? Depends on who has access to the repository
And for those who don't know about it : svn:ignore is great!
If you have a runtime environment for your code (e.g. dependency libraries, specific compiler versions etc.) do not put the packages into the source control. My approach is brutal, but effective. I commit a makefile, whose role is to downloads (via wget) the stuff, unpack it, and build my runtime environment.
I have a particular .c file that does not go in source control.
The rule is nothing in source control that is generated during the build process.
The only known exception is if a tool requires an older version of itself to build (bootstrap problem). In that case you will need a known good bootstrap copy in source control so you can build from blank.
Going out on a limb here, but I believe that if you use task lists in Visual Studio, they are kept in the .suo file. This may not be a reason to keep them in source control, but it is a reason to keep a backup somewhere, just in case...
A lot of time has passed since this question was asked, and I think a lot of the answers, while relevant, don't have hard details on .gitignore on a per language or IDE level.
Github came out with a very useful, community collaborated list of .gitignore files for all sorts of projects and IDEs that is worth taking a look.
Here's a link to that git repo: https://github.com/github/gitignore
To answer the question, here are the related examples for:
C# -> see Visual Studio
Visual Studio
Java
Eclipse
Python
There are also OS-specific .gitignore files. Following:
Windows
OS X
Linux
So, assuming you're running Windows and using Eclipse, you can just concatenate Eclipse.gitignore and Windows.gitignore to a .gitignore file in the top level directory of your project. Very nifty stuff.
Don't forget to add the .gitignore to your repo and commit it!
Chances are, your IDE already handles this for you. Visual Studio does anyway.
And for the .gitignore files, If you see any files or patterns missing in a particular .gitignore, you can open a PR on that file with the proposed change. Take a look at the commit and pull request trackers for ideas.
I am always using www.gitignore.io to generate a proper one .ignore file.
Opinion: everything can be in source control, if you need to, unless it brings significant repository overhead such as frequently changing or large blobs.
3rd party binaries, hard-to-generate (in terms of time) generated files to speed up your deployment process, all are ok.
The main purpose of source control is to match one coherent system state to a revision number. If it would be possible, I'd freeze the entire universe with the code - build tools and the target operating system.
I've read many times and agree with avoiding the use of globals to keep code orthogonal. Does the use of the config file to keep read only information that your program uses similar to using Globals?
If you're using config files in place of globals, then yes, they are similar.
Config files should only be used in cases where the end-user (presumably a computer-savvy user, like a developer) needs to declare settings for an application or piece of code, while keeping their hands out of the code itself.
My first reaction would be that it is not the same. I think the problem with globals is the read+write scenario. Config-files are readonly (at least in terms of execution).
In the same way constants are not considered bad programming behaviour. Config-files, at least in the way I use them, are just easy-changable constants.
Well, since a config file and a global variable can both have the effect of propagating changes throughout a system - they are roughly similar.
But... in the case of a configuration file that change is usually going to take place in a single, highly-visible (to the developer) location, and global variables can affect change in very sneaky and hard to track down ways -- so in this way the two concepts are not similar.
Having a configuration file ususally helps with DRY concepts, and it shouldn't hurt the orthogonality of the system, either.
Bonus points for using the $25 word 'orthogonal'. I had to look that one up in Wikipedia to find out the non-Euclidean definition.
Configuration files are really meant to be easily editable by the end user as a way of telling the program how to run.
A more specialized form of configuration files, user preferences, are used to remember things between program executions.
Global is related to a unique instance for an object which will never change, whereas config file is used as container for reference values, for objects within the application that can change.
One "global" object will never change during runtime, the other object is initialized through config file, but can change later on.
Actually, those objects not only can change during the lifetime of the application, they can also monitor the config file in order to realize "hot-change" (modification of their value without stopping/restarting the application), if that config file is modified.
They are absolutely not the same or replacements for eachother. A config file, or object can be used non-globally, ie passed explicitly.
You can of course have a global variable that refers to a config object, and that would be defeating the purpose.