I worked on a TCL script that takes a bit of time to execute. In order to to run it faster, I would like to use multi-threading using the Thread package. The problem is that I execute my TCL script in an engineering software called Hypermesh. It supports TCL scripting quite well since everything that can be done in the software has an associated TCL command.
For example if the user was to use Hypermesh to create a node at coordinate (x, y, z) then the same result can be obtained using the command *createnode $x $y $z.
Back to the multi-thread problem: I can create threads just fine but inside the thread, I cannot use the command *createnode because it is not recognized. I am guessing it is because some packages are not loaded inside the thread. I would like to load those missing packages with "package require" but no matter how I tried, it could not find any package.
I found out that inside a thread, the auto_path variable does not exist. I tried to create it, but it still would not work.
It looks like my thread can execute basic TCL buit-in commands and nothing else.
On the other hand, when I created a Thread using wish, I did not have this problem at all. All the packages "outside" the thread were also directly loaded "inside".
Here is how my code looks like:
package require Thread
package require Ttrace
set scriptToSend {
package require Trf
return 0
}
set t1 [thread::create]
thread::send -async $t1 $scriptToSend result
puts $result
Trf is just a random package that can be loaded just fine when "package require" is called outside of t1
Is there any way I could setup the thread to have all the commands that exist outside the thread? For example by making the use of package require possible?
EDIT:
I spent some more time on this problem and tried to use tpool instead of thread. The threads created in tpool seem "better". Unlike with thread, they do have an auto_path variable and I am able to use it to successfully load packages when using threads. However now I have more of an Hypermesh issue since it does not let me load all the necessary packages to use procs such as *createnode.
I will try to go around the problem by making my threads do as much as they can without using any Hypermesh function. It will still speed up the script quite a bit.
Tcl uses a limited version of the auto_path global in threads by default because it doesn't set it up in quite the same way (e.g., it doesn't evaluate your ~/.tclshrc). To use the same packages as in the main thread, you'll need to transfer a copy of that variable over.
thread::send $otherThread [list set ::auto_path $::auto_path]
Do that before sending other scripts to the subthread in order to synch things up.
On my system I'd see this sort of thing too:
bash$ tclsh8.6
% puts $::auto_path
/opt/local/lib/tcl8.6 /opt/local/lib /opt/local/lib/tcllib1.19
% package require Thread
2.8.4
% set otherThread [thread::create]
tid0x7000053ab000
% puts [thread::send $otherThread {set ::auto_path}]
/opt/local/lib/tcl8.6 /opt/local/lib
There's also the module path (controllable via the tcl::tm::path command) but that's the same for me in the subthread as in the main thread.
Related
I would like a way to limit what packages can be loaded into tcl.
I can't just remove them from the path, as it has some packages that should be loaded.
Is there a way to handle this without separating the packages to different directories ?
Also, is there a way to add a trace to every package being loaded ? So I can see after the fact which packages have been loaded ?
Thanks
Tcl loads packages in several stages, which are described in a reasonable amount of detail here:
If you ask for a package it does not already known about, it calls the handler registered with package unknown to search the package path (with various origins of the bits of the path) for package index files (pkgIndex.tcl) which provide descriptors of the packages that can be loaded. These descriptors are files that call package ifneeded to give a script to call to load a particular version of a particular package name. If it still doesn't know about the package after doing the loading, you get an error message.
Most pkgIndex.tcl scripts are incredibly simple, though they tend to be dynamically generated during the package installation process. A couple of minor points though; they are not evaluated in the global context, but rather are inside a scope that defines a local dir variable that says the name of the directory containing the pkgIndex.tcl script, and they are always assumed to be UTF-8 encoded on all platforms (I recommend only using ASCII characters in them anyway, which tends to be pretty easy to do precisely because the script is entirely under programmer control).
If you ask for a package it does know about, because the package is either already loaded or there is a descriptor present for it, it checks the version numbers for a match and if the package isn't actually loaded, it evaluates the script provided through package ifneeded in the global context to actually load the package. That will usually source Tcl scripts and/or load DLLs. Some packages do more complicated things than that, but it's common to not put too much complexity in the descriptor scripts themselves: complicated stuff goes into the code they load in, which can be as complex as the package needs it to be.
Now, there is another package subcommand that is useful to you, package forget, which makes Tcl forget the package descriptor and whether the package is loaded. It does not forget the implementation of the package if it is actually loaded but if you can forget the descriptor before stage two, you deny the Tcl interpreter the ability to load the package. If you've got a whitelist of allowed package names and versions, all you need to do is intercept the package unknown handler, delegate to the old implementation, and then strip out anything you don't want before returning. Fortunately, the unknown package handler system is designed to be interceptable like this.
# These packages are to not be version-checked
set builtins {zlib TclOO tcl::tommath Tcl}
# These are the external packages we want to support filtering; THIS IS A WHITELIST!
set whitelist {
Thread {2.8.1}
yaml {0.3.1}
http {2.8.11}
}
proc filterPackages {wrappedCall args} {
# Do the delegation first
uplevel 1 [list {*}$wrappedCall {*}$args]
# Now filter out anything unexpected
global builtins whitelist
foreach p [package names] {
# Check if this is a package we should just tolerate first
if {$p in $builtins} continue
set scripts {}
foreach v [package versions $p] {
if {[dict exists $whitelist $p] && $v in [dict get $whitelist $p]} {
# This is a version we want to keep
lappend scripts [list \
package ifneeded $p $v [package ifneeded $p $v]]
# Everything else is going to be forgotten in a few moments
}
}
# Forget all those package descriptors...
package forget $p
# ...but put back the ones we want.
foreach s $scripts { eval $s }
}
}
# Now install our filter
package unknown [list filterPackages [package unknown]]
As long as install this filter early in your script, before any package require commands have been done (except of the small list of true builtins), you'll have a system which can only package require the packages that you want to allow.
I'm probably missing something obvious here but maybe you can help me.
I have a tcl script. Here it is in its entirety:
puts [encodeSDR 25]
proc encodeSDR {input} {
set sdr "test"
return $sdr
}
when I run this script I get an error:
c:\temp>tclsh testenviro.tcl
invalid command name "encodeSDR"
while executing
"encodeSDR"
invoke from within
"puts [encodeSDR 25]"
(file "testenviro.tcl" line 1)
What am I missing? Seems like this should work perfectly.
PS. Found possible reason:
There when I put the puts call below the proc it worked - so does it not load the entire script first? seems weird if that's the case but maybe thats it.
You are correct in that the entire script is not loaded before any execution takes place. A TCL script is interpreted one command at a time, and then will move to the next one only once it is finished. There is no compiling beforehand to check for procedures you may be referencing down below.
In your case, it first tried to interpret the puts command, which involved calling encodeSDR. Since that had not been defined in memory as of yet, the interpreter had no idea what you were trying to do.
One thing to watch out for is if you define the procedure by itself (say during testing/debug), and then later add it into a script in a way like your example. It will work just fine until you close out of the session, and TCL's memory gets released. The next time you load the script, however, it will fail because the procedure was never loaded for that session.
I have a C extension to Tcl where command mytest is defined. The extension is compiled correctly (I am on Linux, extension is *.so). For example, I can start tclsh and use it like this:
$ tclsh
% load /path/extension.so
% mytest abc
...
But, if I create a file myscript.tcl with the following content:
load /path/extension.so
mytest abc
then I get error:
$ tclsh myscript.tcl
invalid command name "mytest"
while executing
"mytest abc"
(file "myscript.tcl" line 2)
I am using bash on Ubuntu 14.04. Tcl 8.6.
EDIT 1: My question/problem is that I want to use tclsh with a script as an argument - this script should properly load extensions in such a way that mytest and other implemented functions are working without error.
EDIT 2: Uhh, If I use command "source myscript.tcl" inside tcl shell the result is the same. If I use absolute path for myscript.tcl the error is still the same --- "load" executes without warning but I am not sure about it because I get invalid command name "mytest". Maybe the problem is with scope, but it is working correctly when tclsh is used interactively.
If you are using the full path of the extension library in both cases, that part should work identically. It probably is doing though; if it couldn't load it, it would generate an error (which might or might not be helpful, as some of the ways that things fail give very little information; Tcl reports what it has got, but that's sometimes not enough, as it is dependent on the OS to tell it some things). Instead, the problem is probably elsewhere.
The main difference between interactive use and scripted use is that in interactive use, the unknown command will expand unknown command names to Tcl commands that the thing you typed is an unambiguous prefix of. This is convenient, but when converting to a script, you should always use the full command name. OK, not the full full command name — you mostly don't want or need the :: namespace on the front — but without abbreviation, so don't use lappe for lappend. (In interactive use, Tcl will also exec things as external programs without requiring you to type the exec explicitly; again, that's turned off in scripts as it is rather fragile.)
Could it be that this is what is going on? You can check by setting the global variable tcl_interactive to 0 before typing in your code (I recommend using cut-n-paste for that typing, so that you know exactly what is going in). If that fails, it's the interactive-mode helpfulness that is tripping you up. Check what commands you might have as an expansion for a prefix with info commands (after the load, of course):
info commands mytest*
If that just reports mytest, my theory is wrong. (Well, if it does that and the length of that string is 6; there could theoretically be extra invisible characters have been put on the command name, which would be legal Tcl but very nasty and DON'T DO THAT!)
I am making this GUI'ed TCL script with ActiveTCL & Expect.
But for some reason Expect doesn't work with telnet that comes with windows 8 64bit, so I figured a way to use a custom telnet tcl script. It works fine, but I need now to wrap my script with the telnet script and some logo images into a single .exe to run without extra files in directory, but I can't for the life of me get it to work.
I click add files to the tclapp wrapper but it says file not found on the script when it tries to call for the telnet script.
When you wrap Tcl code into a single-file executable, everything goes inside. Scripts, libraries, any images (assuming you're making a GUI), the lot. Tcl transparently extracts things and pretends you've got a real filesystem. However, when you execute a program (whether via exec, open |… or spawn) then the OS must be involved as you are creating a subprocess — the OS is always involved in that, as process management is one of the main things that the OS kernel does — and it needs to have a real executable to run. If you have packaged up your telnet-replacement as its own single-file executable and stored it inside the parent process's VFS, you have to make that subordinate process executable real.
Copy the telnet-executable out to some location (e.g., to the temporary directory, which I think should be described in $::env(TEMP)) and execute that.
set realTelnetExe [file join $::env(TEMP) mytelnet.exe]
file copy .../the/stored/copy/mytelnet.exe $realTelnetExe
spawn $realTelnetExe
# ...
You probably want to file delete the copy once you're done using it.
Relevant background material:
comp.lang.tcl thread
Run tclsh command without any tcl file, the interpreter will go into interactive mode.
Can I simply disable this feature by modifying the tclsh source code ?
I can't imagine why you would want to bother doing this, given that supplying any script file will turn off interactive mode. The script you supply will have full access to the additional arguments passed in (a list in the global argv variable) and the standard IO channels (stdin, stdout and stderr). It can exit when it is done. Literally anything you want can be done at that point; you've just got to write a script to do it.
If you're including Tcl in your own program, the behaviour of tclsh is implemented in the C function Tcl_Main. If you never call that — instead just using Tcl_FindExecutable, Tcl_CreateInterp and Tcl_Eval/Tcl_EvalFile — then you never get any of that interactive behaviour. While theoretically you could modify the Tcl source itself to do what you want — it's all open source — why would you bother when you could just not call that code in the first place?