Me and my team have been working on multiple tools flow for awhile now.
We keep adding new procedure in either a same file or creating new file in the same directory. There are also a lot of nested procedures; one calls others.
The number of procedures will only keep growing, and the flow involves at least 10 people who love to do their own things.
My question is, how would we go about managing all these procedures in a tidy manner?
We assume you follow good practices with general software engineering (keeping files in source control, etc.) as without those you're stuck anyway.
Tcl doesn't really support nested procs; you can call proc from inside another procedure, but it doesn't do any kind of scoping.
You should be thinking in terms of dividing up your code into pieces of “coherent API”. What exactly that is depends on your application, but it is only rarely a single procedure; a particular dialog box or screen is a much more useful unit. That might end up as one procedure, but it's often several related ones.
Once you've identified these coherent pieces, they form the contents of what you put in a file, typically one coherent piece per file, though if the file is rather long when you do that, using a group of files instead (probably in their own directory) makes a lot of sense. At the same time, you probably should make the variables and commands defined by each coherent piece be all in a Tcl namespace, which isolates the piece a little bit from the rest of the world, mostly to stop code from treading on the toes of other code.
Now that you've done that, and if you've got what you think is a stable API to your coherent piece, you can make that piece into a Tcl package. That's just done by giving it a higher-level name and version number; you put this in one of your files in the coherent piece:
package provide YourPackageName 1.0
and then (usually in the same directory) you make a pkgIndex.tcl file with contents like this:
package ifneeded YourPackageName 1.0 [list source [file join $dir yourFilename.tcl]]
That is, it says that to get YourPackageName version 1.0 in a Tcl interpreter, you source the file $dir/yourFilename.tcl; the $dir is a convenience in package index files that refers to the directory containing the current package index file. Then the rest of your code can stop thinking about “reading the right files”, and start thinking in terms of “use this defined API”. (This is great if you then choose to start implementing the package using a mixture of Tcl and C or even pure C code; a change to the index file to use load of the right thing and everything else can be oblivious.) It does that by doing:
package require YourPackageName
# Give the version if necessary, of course
Then write yourself some documentation (even if it is just listing the entry point commands into the package) and tests, and you've migrated to being a very well behaved piece of code indeed.
There are some additional techniques that can help you in some cases with making coherent pieces. In particular, if you're using an OO system like TclOO, iTcl, or XOTcl, each class is almost certainly a candidate coherent piece. Also, it's sometimes better to put several related coherent pieces together in a package. However, there's absolutely no hard and fast rule on that.
Finally, Tcl uses a bunch of techniques to find packages, but they mostly come down to looking using the auto_path global variable. In your application main script, it's best (if the rest of your code is mostly in the library directory) to use something like this as one of the first steps:
lappend auto_path [file join [file dirname [info script]] library]
You can also gather the contents of many pkgIndex.tcl files in one place, provided you take into account any pathname changes needed from moving things around.
So regarding the TCL, You can look for creating the packages and namespaces. Let me know if that can help. So more details can be provided
Related
I have a couple questions about adding options/switches (with and without parameters) to procedures/commands. I see that tcllib has cmdline and Ashok Nadkarni's book on Tcl recommends the parse_args package and states that using Tcl to handle the arguments is much slower than this package using C. The Nov. 2016 paper on parse_args states that Tcl script methods are or can be 50 times slower.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Can both be easily included in a starkit?
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
Thank you for considering my questions.
Are Tcl methods really signicantly slower? Is there some minimum threshold number of options to be reached before using a package?
Potentially. Procedures have overhead to do with managing the stack frame and so on, and code implemented in C can avoid a number of overheads due to the way values are managed in current Tcl implementations. The difference is much more profound for numeric code than for string-based code, as the cost of boxing and unboxing numeric values is quite significant (strings are always boxed in all languages).
As for which is the one to use, it really depends on the details as you are trading off flexibility for speed. I've never known it be a problem for command line parsing.
(If you ask me, fifty options isn't really that many, except that it's quite a lot to pass on an actual command line. It might be easier to design a configuration file format — perhaps a simple Tcl script! — and then to just pass the name of that in as the actual argument.)
Is there any reason to use parse_args (not in tcllib) over cmdline (in tcllib)?
Performance? Details of how you describe things to the parser?
Can both be easily included in a starkit?
As long as any C code is built with Tcl stubs enabled (typically not much more than define USE_TCL_STUBS and link against the stub library) then it can go in a starkit as a loadable library. Using the stubbed build means that the compiled code doesn't assume exactly which version of the Tcl library is present or what its path is; those are assumptions that are usually wrong with a starkit.
Tcl-implemented packages can always go in a starkit. Hybrid packages need a little care for their C parts, but are otherwise pretty easy.
Many packages either always build in stubbed mode or have a build configuration option to do so.
Is this included in 8.7a now? (I'd like to use 8.7a but I'm using Manjaro Linux and am afraid that adding it outside the package manager will cause issues that I won't know how to resolve or even just "undo").
We think we're about a month from the feature freeze for 8.7, and builds seem stable in automated testing so the beta phase will probably be fairly short. The list of what's in can be found here (filter for 8.7 and Final). However, bear in mind that we tend to feel that if code can be done in an extension then there's usually no desperate need for it to be in Tcl itself.
I think this is more of a Tcl configuration question rather than a Tcl coding question...
I inherited a whole series of Tcl scripts that are used within a simulation tool that my company built in-house. In my scripts, I'm finding numerous instances where there are function calls to functions that don't seem to be declared anywhere. How can I trace the path to these phantom functions?
For example, rather than use source, someone build a custom include function that they named INCLUDE. Tclsh obviously balks when I try to run it there, but with my simulation software, it runs fine.
I've tried grep-ing through the entire simulation software for INCLUDE, but I'm not having any luck. Are there any other obvious locations outside the simulation software where a Tcl function might be defined?
The possibilities:
Within your software. (you have checked for this).
Within some other package included by the software.
Check and see if the environment variable TCLLIBPATH is set.
Also check and see if the simulation software sets TCLLIBPATH.
This will be a list of directories to search for Tcl packages, and you
will need to search the packages that are located outside of the
main source tree.
Another possibility is that the locations are specified in the pkgIndex.tcl file.
Check any pkgIndex.tcl files and look for locations outside the main source tree.
Within an unknown command handler. This could be in
your software or within some other package. You should be able to find
some code that processes the INCLUDE statement.
Within a binary package. These are shared libraries that are loaded
by Tcl. If this is the case, there should be some C code used to
build the shared library that can be searched.
Since you say there are numerous instances of unknown functions, my first
guess is that you have
not found all the directories where packages are loaded from. But an
''unknown'' command handler is also a possibility.
Edit:
One more possibility I forgot. Check and see if your software sets the auto_path variable. Check any directories added to the auto_path for
other packages.
This isn't a great answer for you, but I suspect it is the best you're going to get...
The procedure could be defined in a great many places. Your best bet for finding it is to use a tool like findstr (on Windows) or grep -R (on POSIX platforms) to search across all the relevant source files. But that still might not help! It might not be a procedure but instead a general command, which could be implemented in C and not as a procedure, or it could be defined in a packaged application archive (which are usually awkward to look inside). There are also other types of script-implemented command too, which could make things awkward. Generally searching and investigating is your best bet, but it might not work.
Tcl doesn't really differentiate strongly between different types of command except in some introspection operations. If you're lucky, you could find that info body tells you the definition of the procedure (and info args and info default tell you about the arguments) but that won't help with other command types at all. Tcl 8.7 will include a command (info cmdtype) that would help a lot with narrowing down what to do next, but that's no use to you now and it definitely doesn't exist in older versions.
I'm just starting with TCL and trying to get my head around how to best define and integrate modules. There seem to be much effort put into the package+namespace concept, but from what I can tell interp is more powerful and lean for every thinkable scenario. In particular when it comes to hiding and renaming procedures, but also the lack of creep in the global namespace. The only reason to use package+namespaces seem to be because "once upon a time Sun said so".
When should I ever use package+namespace instead of interp?
Namespaces and packages work together. Interpreters are something else.
A namespace is a small scale naming context in Tcl. It can contain commands, variables and other namespaces. You can refer to entities in a namespace via either local names (foo) or via qualified names (bar::foo); if a qualified name starts with ::, it is relative to the (interpreter-)global namespace, and can be used to refer to its command or variable from anywhere in the interpreter. (FWIW, the TclOO object system builds extensively on top of namespaces; there is one namespace per object.)
A package is a high-level concept for a bunch of code supplied by some sort of library. Packages have abstract names (the name do not have to correspond to how the library's implementation is stored on disk) and a distinct version; you can ask for a particular version if necessary, though most of the time you don't bother. Packages can be implemented by multiple mechanisms, but they almost all come down to sourceing some number of Tcl scripts and loading some number of DLLs. Almost all packages declare commands, and they conventionally are encouraged to put those commands in a namespace with the same general name as the package. However, quite a few older packages do not do this for various reasons, mostly to do with compatibility with existing code.
An interpreter is a security context in Tcl. By default, Tcl creates one interpreter (plus another if it sets up the console window in wish). Named entities in one interpreter are completely distinct from named entities in another interpreter with a few key exceptions:
Channels have common names across all interpreters. This means that an interpreter can talk about channels owned by another interpreter, but merely being able to mention its name doesn't give permission to access the channel. (The stdin, stdout and stderr channels are shared by default.)
The interp alias command can be used to make alias commands, which are such that invoking a command (the alias) in one interpreter can cause a command (the implementation) in another interpreter to be called, with all arguments safely passed over. This allows one interpreter to expose whatever special calls it wants another interpreter to access without losing control, but it is up to the implementation of those commands to act safely on those arguments.
A safe interpreter is one with the unsafe commands of Tcl profiled out by default. (That's things like open, socket, source, load, cd, etc.) The parent interpreter that created the safe child interpreter can use the alias mechanism to add in exactly the functionality desired; it's very much analogous to an OS system call except you can easily make your own application-specific ones.
Tcl's threading package is designed to create one interpreter per thread (and the aliasing mechanism does not work across threads). That means that there's very little in the way of shared resources by default, and inter-thread communication is done via queued message passing.
In general, packages are required at most once per interpreter and are how you are recommended to access most third-party functionality. Namespaces are fairly lightweight and are used for all sorts of things, and interpreters are considered to be expensive; lots of quite thoroughly production-grade Tcl scripts only ever work with a single interpreter. (Threads are even more expensive than interpreters; it's good practice to match the number of threads you create to the hardware load that you wish to impose, probably through the use of suitable thread pools.)
The purpose of a module is to provide modular code, i.e. code that can easily be used by applications beyond the module writer's knowledge and control, and that encapsulates their own internals.
Package-namespace- and interpreter-based modules are probably equally good at encapsulation, but it's not as easy to make interpreter-based modules that play well with arbitrary applications (it is of course possible).
My own opinion is that interpreters are application level (I mostly use them for user input and for controlled evaluation), not module level. Both namespaces and packages have their warts, but in most cases they do what is expected of them with a minimum of fuss.
My recommendation is that if you are writing modules for your own benefit and interpreters serve you well, by all means use them. If you write modules that other people are to use, possibly including yourself in 18 months, you should stick with namespaces and packages.
this is a strange question and i searched but couldn't find any satisfactory answer.
I have a compiled tcl file i.e. a .tbc file. So is there a way to convert this .tbc file back to .tcl file.
I read here and someone mentioned about ::tcl_traceCompile and said this could be used to disassemble the .tbc file. But being a novice tcl user i am not sure if this is possible, or to say more, how exactly to use it.
Though i know that tcl compiler doesn't compile all the statements and so these statements can be easily seen in .tbc file but can we get the whole tcl back from .tbc file.
Any comment would be great.
No, or at least not without a lot of work; you're doing something that quite a bit of effort was put in to prevent (the TBC format is intended for protecting commercial code from prying eyes).
The TBC file format is an encoding of Tcl's bytecode, which is not normally saved at all; the TBC stands for Tcl ByteCode. The TBC format data is only produced by one tool, the commercial “Tcl Compiler” (originally written by either Sun or Scriptics; the tool dates from about the time of the transition), which really is a leveraging of the built-in compiler that every Tcl system has together with some serialization code. It also strips as much of the original source code away as possible. The encoding used is unpleasant; you want to avoid writing your own loader of it if you can, and instead use the tbcload extension to do the work.
You'll then need to use it with a custom build of Tcl that disables a few defensive checks so that you can disassemble the loaded code with the tcl::unsupported::disassemble command (which normally refuses to take apart anything coming from tbcload); that command exists from Tcl 8.5 onwards. After that, you'll have to piece together what the code is doing from the bytecodes; I'm not aware of any tools for doing that at all, but the bytecodes are mostly fairly high level so it's not too difficult for small pieces of code.
There's no manual page for disassemble; it's formally unsupported after all! However, that wiki page I linked to should cover most of the things you need to get started.
I can say partially "yes" and conditionaly too. That condition is if original tcl code is written in namespace and procs are defined within namespace curly braces. Then you source tbc file in tkcon/wish and see code using info procs and namespace command. Offcourse you need to know namespace name. However that also can be found.
I am writing some support code in the common subset of Matlab/Octave, which comes in the form of a bunch of functions. Let's call it a package.
I want to be able to organize the package, i.e.,
put all the relevant function files in a single place, where users
are not supposed to store their code;
have some internal organization ('subpackages');
prevent namespace pollution;
have some mechanism for user code to 'import' parts of the package;
I don't necessarily want all functions I provide to be
visible from user clients.
On the Matlab side of things, this functionality is pretty much provided by package directories and the 'import' mechanism. This functionality doesn't appear to be available in Octave though (as of 3.6.1).
Given that, I wonder what options remain for organizing my support code package in Octave.
The option of putting everything in a directory and just have the user code do an ADDPATH feels rather unrefined, and doesn't give the level of control I want -- it only addresses point #1 of the list above.
There is plenty documentation here and examples in OctaveForge. Just browse the SVN.
Also there are personal packages all around. For example this one
Happy coding!