I'm having trouble finding good advice and common practices for the use of namespaces in Clojure. I realize that namespaces are not the same as Java packages so I'm trying to tease out the conventions in Clojure, which seem surprisingly hard to determine.
I think I have a pretty good idea how to split functions into clj files and even roughly how I'd want to organize those files into directories. But beyond that I'm having trouble finding the mechanics for my dev environment. Some inter-related questions:
Do I use the same uniqueness conventions for Clojure namespaces as I would normally use for Java packages? [ie backwards-company-domain.project.subsystem]
Should I save my files in a directory structure that matches my namespaces? [ala Java]
If I have multiple namespaces, do I need to compile all of my code into a jar and add it to my classpath to make it accessible?
Should each namespace compile to one jar? Or should I create a single jar that contains clj code from many namespaces?
Thanks...
I guess it's ok if you think it helps, but many Clojure projects don't do so -- cf. Compojure (using a top-level compojure ns and various compojure.* ns's for specific functionality), Ring, Leiningen... Clojure itself uses clojure.* (and clojure.contrib.* for contrib libraries), but that's a special case, I suppose.
Yes! You absolutely must do so, or else Clojure won't be able to find your namespaces. Also note that you musn't use the underscore in namespace names or the hyphen in filenames and wherever you use a hyphen in a namespace name, you must use an underscore in the filename (so that the ns my.cool-project is defined in a file called cool_project.clj in a directory called my).
You need to make sure all your stuff is on the classpath, but it doesn't matter if it's in a jar, multiple jars, a mixture of jars and directories on the filesystem... As long as it obeys the correct naming conventions (your point no. 2) you should be fine.
However, do not compile things ahead-of-time if there's no particular reason to do so -- this may prevent your code from being portable across various versions of Clojure without providing any benefits besides a slightly improved loading time.
You'll still need to use AOT compilation sometimes, notably in some Java interop scenarios -- the documentation of the relevant functions / macros always mentions that. There are examples of things requiring AOT in clojure.contrib; I've never needed it, so I can't provide much in the way of details.
I'd say you should use jars for functional units of code. E.g. Compojure and Ring get packaged as single jars containing many namespaces which together compose the whole package. Also, clojure.contrib is notably packaged as a single jar with multiple unrelated libraries; but that again may be a special case.
On the other hand, a single jar containing all of your project's code together with its dependencies might occasionally be useful for deployment. Check out the Leiningen build tool and its 'uberjar' facility if you think that sort of thing may be useful to you.
Strictly speaking, not necessary, though many Java projects have dropped that convention as well, especially for internal projects or private APIs. Do avoid single-segment namespaces though, which would result in classfiles being generated in the default package.
Yes.
Regarding 3 & 4, packaging and AOT compilation are entirely orthogonal to the question of namespace conventions.
Related
I have a couple questions about adding options/switches (with and without parameters) to procedures/commands. I see that tcllib has cmdline and Ashok Nadkarni's book on Tcl recommends the parse_args package and states that using Tcl to handle the arguments is much slower than this package using C. The Nov. 2016 paper on parse_args states that Tcl script methods are or can be 50 times slower.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Can both be easily included in a starkit?
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
Thank you for considering my questions.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Potentially. Procedures have overhead to do with managing the stack frame and so on, and code implemented in C can avoid a number of overheads due to the way values are managed in current Tcl implementations. The difference is much more profound for numeric code than for string-based code, as the cost of boxing and unboxing numeric values is quite significant (strings are always boxed in all languages).
As for which is the one to use, it really depends on the details as you are trading off flexibility for speed. I've never known it be a problem for command line parsing.
(If you ask me, fifty options isn't really that many, except that it's quite a lot to pass on an actual command line. It might be easier to design a configuration file format — perhaps a simple Tcl script! — and then to just pass the name of that in as the actual argument.)
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Performance? Details of how you describe things to the parser?
Can both be easily included in a starkit?
As long as any C code is built with Tcl stubs enabled (typically not much more than define USE_TCL_STUBS and link against the stub library) then it can go in a starkit as a loadable library. Using the stubbed build means that the compiled code doesn't assume exactly which version of the Tcl library is present or what its path is; those are assumptions that are usually wrong with a starkit.
Tcl-implemented packages can always go in a starkit. Hybrid packages need a little care for their C parts, but are otherwise pretty easy.
Many packages either always build in stubbed mode or have a build configuration option to do so.
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
We think we're about a month from the feature freeze for 8.7, and builds seem stable in automated testing so the beta phase will probably be fairly short. The list of what's in can be found here (filter for 8.7 and Final). However, bear in mind that we tend to feel that if code can be done in an extension then there's usually no desperate need for it to be in Tcl itself.
I've been having trouble sorting through the noise while trying to find an answer to this. There are certainly a lot of libraries available for handling configuration files but what I'm looking for is an answer to whether there is a solution available for this specific kind of split configuration.
On Linux systems I've found that it's not uncommon to find a program which has split its default configuration away from user modifications by instructing the user to place a subset of the default configuration into a separate folder (commonly found with a .d suffix). These changes override what is found in the default configuration and provide a very easy way to track at a later date what has been modified.
There is a wide variety of incompatible syntaxes used across different configuration files, and I am not aware of a library that can parse them all. But if you are willing to restrict yourself to just one configuration syntax, then two answers to your question come to mind.
The first is to use a scripting language as your configuration file syntax. Let's assume an application reads both default.cfg and override.cfg (in that order). If default.cfg contains name="john", then override.cfg might contain name="mary", which will have the effect of overriding the value of name. This occurs because the shell interpreter provides a common place to store the global variables assigned within the script files. The following pseudocode shows how your application could interact with a scripting language interpreter:
interp = new Interpreter();
interp.executeFile("default.cfg");
interp.executeFile("override.cfg");
name = interp.getValueOfVariable("name");
The second is to use Config4*, which is a Configuration-file parser I wrote; it is available in C++ and Java. I recommend you read Chapter 2 (Overivew of Config4* Syntax) in the Config4* Getting Started Guide. The "adaptive configuration" abilities of Config4* seem to fit the question I think you are asking. If, after reading Chapter 2, you agree, then read Chapter 3, which provides an overview of the C++ and Java API. That will probably be enough to get you started.
I'm just starting with TCL and trying to get my head around how to best define and integrate modules. There seem to be much effort put into the package+namespace concept, but from what I can tell interp is more powerful and lean for every thinkable scenario. In particular when it comes to hiding and renaming procedures, but also the lack of creep in the global namespace. The only reason to use package+namespaces seem to be because "once upon a time Sun said so".
When should I ever use package+namespace instead of interp?
Namespaces and packages work together. Interpreters are something else.
A namespace is a small scale naming context in Tcl. It can contain commands, variables and other namespaces. You can refer to entities in a namespace via either local names (foo) or via qualified names (bar::foo); if a qualified name starts with ::, it is relative to the (interpreter-)global namespace, and can be used to refer to its command or variable from anywhere in the interpreter. (FWIW, the TclOO object system builds extensively on top of namespaces; there is one namespace per object.)
A package is a high-level concept for a bunch of code supplied by some sort of library. Packages have abstract names (the name do not have to correspond to how the library's implementation is stored on disk) and a distinct version; you can ask for a particular version if necessary, though most of the time you don't bother. Packages can be implemented by multiple mechanisms, but they almost all come down to sourceing some number of Tcl scripts and loading some number of DLLs. Almost all packages declare commands, and they conventionally are encouraged to put those commands in a namespace with the same general name as the package. However, quite a few older packages do not do this for various reasons, mostly to do with compatibility with existing code.
An interpreter is a security context in Tcl. By default, Tcl creates one interpreter (plus another if it sets up the console window in wish). Named entities in one interpreter are completely distinct from named entities in another interpreter with a few key exceptions:
Channels have common names across all interpreters. This means that an interpreter can talk about channels owned by another interpreter, but merely being able to mention its name doesn't give permission to access the channel. (The stdin, stdout and stderr channels are shared by default.)
The interp alias command can be used to make alias commands, which are such that invoking a command (the alias) in one interpreter can cause a command (the implementation) in another interpreter to be called, with all arguments safely passed over. This allows one interpreter to expose whatever special calls it wants another interpreter to access without losing control, but it is up to the implementation of those commands to act safely on those arguments.
A safe interpreter is one with the unsafe commands of Tcl profiled out by default. (That's things like open, socket, source, load, cd, etc.) The parent interpreter that created the safe child interpreter can use the alias mechanism to add in exactly the functionality desired; it's very much analogous to an OS system call except you can easily make your own application-specific ones.
Tcl's threading package is designed to create one interpreter per thread (and the aliasing mechanism does not work across threads). That means that there's very little in the way of shared resources by default, and inter-thread communication is done via queued message passing.
In general, packages are required at most once per interpreter and are how you are recommended to access most third-party functionality. Namespaces are fairly lightweight and are used for all sorts of things, and interpreters are considered to be expensive; lots of quite thoroughly production-grade Tcl scripts only ever work with a single interpreter. (Threads are even more expensive than interpreters; it's good practice to match the number of threads you create to the hardware load that you wish to impose, probably through the use of suitable thread pools.)
The purpose of a module is to provide modular code, i.e. code that can easily be used by applications beyond the module writer's knowledge and control, and that encapsulates their own internals.
Package-namespace- and interpreter-based modules are probably equally good at encapsulation, but it's not as easy to make interpreter-based modules that play well with arbitrary applications (it is of course possible).
My own opinion is that interpreters are application level (I mostly use them for user input and for controlled evaluation), not module level. Both namespaces and packages have their warts, but in most cases they do what is expected of them with a minimum of fuss.
My recommendation is that if you are writing modules for your own benefit and interpreters serve you well, by all means use them. If you write modules that other people are to use, possibly including yourself in 18 months, you should stick with namespaces and packages.
what is the purpose of namespaces ?
and, more important, should they be used as objects in java (things that have data and functions and that try to achieve encapsulation) ? is this idea to far fetched ? :)
or should they be used as packages in java ?
or should they be used more generally as a module system or something ?
Given that you use the Clojure tag, I suppose that you'll be interested in a Clojure-specific answer:
what is the purpose of namespaces ?
Clojure namespaces, Java packages, Haskell / Python / whatever modules... At a very high level, they're all different names for the same basic mechanism whose primary purpose is to prevent name clashes in non-trivial codebases. Of course, each solution has its own little twists and quirks which make sense in the context of a given language and would not make sense outside of it. The rest of this answer will deal with the twists and quirks specific to Clojure.
A Clojure namespace groups Vars, which are containers holding functions (most often), macro functions (functions used by the compiler to generate macroexpansions of appropriate forms, normally defined with defmacro; actually they are just regular Clojure functions, although there is some magic to the way in which they are registered with the compiler) and occasionally various "global parameters" (say, clojure.core/*in* for standard input), Atoms / Refs etc. The protocol facility introduced in Clojure 1.2 has the nice property that protocols are backed by Vars, as are the individual protocol functions; this is key to the way in which protocols present a solution to the expression problem (which is however probably out of the scope of this answer!).
It stands to reason that namespaces should group Vars which are somehow related. In general, creating a namespace is a quick & cheap operation, so it is perfectly fine (and indeed usual) to use a single namespace in early stages of development, then as independent chunks of functionality emerge, factor those out into their own namespaces, rinse & repeat... Only the things which are part of the public API need to be distributed between namespaces up front (or rather: prior to a stable release), since the fact that function such-and-such resides in namespace so-and-so is of course a part of the API.
and, more important, should they be used as objects in java (things that have data and functions and that try to achieve encapsulation) ? is this idea to far fetched ? :)
Normally, the answer is no. You might get a picture not too far from the truth if you approach them as classes with lots of static methods, no instance methods, no public constructors and often no state (though occasionally there may be some "class data members" in the form of Vars holding Atoms / Refs); but arguably it may be more useful not to try to apply Java-ish metaphors to Clojure idioms and to approach a namespace as a group of functions etc. and not "a class holding a group of functions" or some such thing.
There is an important exception to this general rule: namespaces which include :gen-class in their ns form. These are meant precisely to implement a Java class which may later be instantiated, which might have instance methods and per-instance state etc. Note that :gen-class is an interop feature -- pure Clojure code should generally avoid it.
or should they be used as packages in java ?
They serve some of the same purposes packages were designed to serve (as already mentioned above); the analogy, although it's certainly there, is not that useful, however, just because the things which packages group together (Java classes) are not at all like the things which Clojure namespaces group together (Clojure Vars), the various "access levels" (private / package / public in Java, {:private true} or not in Clojure) work very differently etc.
That being said, one has to remember that there is a certain correspondence between namespaces and packages / classes residing in particular packages. A namespace called foo.bar, when compiled, produces a class called bar in the package foo; this means, in particular, that namespace names should contain at least one dot, as so-called single-segment names apparently lead to classes being put in the "default package", leading to all sorts of weirdness. (E.g. I find it impossible to have VisualVM's profiler notice any functions defined in single-segment namespaces.)
Also, deftype / defrecord-created types do not reside in namespaces. A (defrecord Foo [...] ...) form in the file where namespace foo.bar is defined creates a class called Foo in the package foo.bar. To use the type Foo from another namespace, one would have to :import the class Foo from the foo.bar package -- :use / :require would not work, since they pull in Vars from namespaces, which records / types are not.
So, in this particular case, there is a certain correspondence between namespaces and packages which Clojure programmers who wish to take advantage of some of the newer language features need to be aware of. Some find that this gives an "interop flavour" to features which are not otherwise considered to belong in the realm of interop (defrecord / deftype / defprotocol are a good abstraction mechanism even if we forget about their role in achieving platform speed on the JVM) and it is certainly possible that in some future version of Clojure this flavour might be done away with, so that the namespace name / package name correspondence for deftype & Co. can be treated as an implementation detail.
or should they be used more generally as a module system or something ?
They are a module system and this is indeed how they should be used.
A package in Java has its own namespace, which provides a logical grouping of classes. It also helps prevent naming collisions. For example in java you will find java.util.Date and java.sql.Date - two different classes with the same name differentiated by their namespace. If you try an import both into a java file, you will see that it wont compile. At least one version will need to use its explicit namespace.
From a language independant view, namespaces are a way to isolate things (i.e. encapsulate in a sens). It's a more general concept (see xml namespaces for example). You can "create" namespaces in several ways, depending on the language you use: packages, static classes, modules and so on. All of these provides namespaces to the objects/data/functions they contain. This allow to organize the code better, to isolate features, tends for better code reuse and adaptability (as encapsulation)
As stated in the "Zen of Python", "Namespaces are one honking great idea -- let's do more of those !".
Think of them as containers for your classes. As in if you had a helper class for building strings and you wanted it in your business layer you would use a namespace such as MyApp.Business.Helpers. This allows your classes to be contained in sensical locations so when you or some else referencing your code wants to cosume them they can be located easily. For another example if you wanted to consume a SQL connection helper class you would probably use something like:
MyApp.Data.SqlConnectionHelper sqlHelper = new MyApp.Data.SqlConnectionHelper();
In reality you would use a "using" statement so you wouldn't need to fully qualify the namespace just to declare the variable.
Paul
I am designing a class library designed to solve a wide scope of problems. One thing about this library is that it will be usable by several different languages and environments natively. For example, there will be a C++ version written entirely in C++, a .NET version written in C# and a Java version written in Java, without any dependencies on each other... as opposed to writing the core library in C++ and simply providing .NET and Java bindings to it.
The library in each of its different forms sets out to solve a different but sometimes very similar set of problems. For example, there might be many classes whose members will be functionally identical in each language, and there will also be many classes that will be present in only one or two language-versions of the library, but not the others. Take a class or struct representing a program's version number. .NET already has such as class (System.Version) so I would not include it in my .NET version but the C++ and Java libraries would provide one.
The problem I am facing is that for classes which will exist in most or all versions of the library, the documentation will remain relatively the same (obviously). The brief text for both the C++ and Java version for a Version struct would be something like "Represents a software version number in the form major.minor.build.revision"... as would the detailed class description, and all the members' documentation, etc. As you know, .NET, Java and C++ all have their own documentation syntax. Is there any way I can attempt to consolidate documentation in a language-neutral way (WITHOUT writing the documentation separately from the source code - e.g. manual documentation as opposed to generating it using doxygen/sandcastle/javadoc) or am I stuck copying and pasting the same text into the source files of each version?
I was having the same issues and decided there were just two options for me:
Using the same documentation generator in all languages. If you use doxygen (or ROBODoc, or whatever) for all of them, you would have just one doc syntax for all languages. This means that you have to break with language-specific conventions, though.
Write your own doc parser. Which is hard work, especially for a language with quite complex syntactic rules (as C++.)
We are currently using doxygen for such projects.