TCL/Expect - $argv VS $::argv VS {*}$argv - tcl

What is difference between following variables:
$argv
$::argv
{*}$argv
First two are possible to print via puts command and they returns following output:
param0 param1 {param 2} param3
param0 param1 {param 2} param3
The arguments that was passed to script were:
param0 param1 "param 2" param3
The last one end up with error:
wrong # args: should be "puts ?-nonewline? ?channelId? string"
while executing
"puts {*}$argv"
I've done some research in this area using following code:
if {[array exists $argv]} {
puts "\$argv IS ARRAY"
} else {
puts "\$argv IS NOT AN ARRAY"
}
if {[string is list $argv]} {
puts "\$argv IS LIST"
} else {
puts "\$argv IS NOT LIST"
}
if {[array exists $::argv]} {
puts "\$::argv IS ARRAY"
} else {
puts "\$::argv IS NOT AN ARRAY"
}
if {[string is list $::argv]} {
puts "\$::argv IS LIST"
} else {
puts "\$::argv IS NOT LIST"
}
if {[array exists {*}$argv]} {
puts "{*}\$::argv IS ARRAY"
} else {
puts "{*}\$::argv IS NOT AN ARRAY"
}
if {[string is list {*}$argv]} {
puts "{*}\$::argv IS LIST"
} else {
puts "{*}\$::argv IS NOT LIST"
}
The last two if-else statements which contain {*}$argv ends with following error:
wrong # args: should be "array exists arrayName"
while executing
"array exists {*}$argv"
invoked from within
"if {[array exists {*}$argv]} {
puts "{*}\$::argv IS ARRAY"
} else {
puts "{*}\$::argv IS NOT AN ARRAY"
}"
Commenting out those two statements shows that $argv and $::argv are lists:
argv IS NOT AN ARRAY
$argv IS NOT AN ARRAY
argv IS LIST
$argv IS LIST
Both those lists can be traversed as standard list e.g.:
foreach item $argv {
puts $item
}
or
foreach item $::argv {
puts $item
}
Attempt to traverse {*}$argv the same way leads to following error again:
wrong # args: should be "foreach varList list ?varList list ...? command"
while executing
"foreach item {*}$argv {
puts $item
}"
I am using TCL version 8.5

What is difference between following variables:
$argv
$::argv
{*}$argv
There are two types of difference here.
Unqualified and Qualified Variables
In Tcl, unqualified and qualified variables can be a bit different, but it depends on the context (in a pretty simple way though). Firstly, a qualified variable name is one that contains at least one :: within it. If the variable name (the thing after the $ — in Tcl, $ just means “read this variable now and use its contents here”) starts with ::, it is an absolute variable name, otherwise a qualified variable name is a relative variable name and is resolved with respect to the current namespace (which you can find out with namespace current if you're uncertain). Absolute variable names always refer to the same thing, in all contexts. Thus, ::argv is an absolute variable name, and indeed it refers to a variable called argv in the top-level, global namespace. That happens to be a variable that tclsh and wish write their arguments into.
But if there is no ::, it is an unqualified variable name. If you are not in a procedure (or procedure-like thing, which includes a lambda term such as you'd use with apply or the methods defined by various OO systems) then the variable is (mostly) treated as if it was a relative variable name and resolved with respect to the current namespace. namespace eval and namespace code are two of the things that can change the current namespace (the others are more obscure). All this is provided you use variable to declare all your namespace variables. Otherwise, you can hit some weird problems with variable resolution which are really nasty. So do use variable. Really.
If you are in a procedure(-like entity) though, that unqualified name refers to a local variable, whose life is coupled to that of the stack frame pushed on the stack when the procedure is entered. That can be linked to variables in other scopes (including the global namespace) through various commands: global, upvar, variable, and namespace upvar. However, the actual resolution of the variable is to something local.
Finally, there might also be a custom variable resolver in place. Since you're using Tcl 8.5, the place where you're most likely to see this in use is if you're using Incr Tcl, an object system for Tcl. Custom variable resolvers can do some complex stuff. (If you were using Tcl 8.6, the most likely place to see a custom variable resolver at work is in TclOO. The variable resolver there is very conservative and cautious, but allows local variables to be bound to object variables without having to explicitly declare this in each method).
Normal and Expanding Substitution
The difference between $argv and {*}$argv is totally different.
$argv is a normal substitution. It says “read this variable here and use the contents of it instead”. It can be used in the middle of a word, so $argv$argv$argv is a thing, consisting of the concatenation of the contents of the argv variable three times.
{*}, when placed at the start of a word (it's not special elsewhere), marks that word for expansion. When a word is expanded, it's parsed as a Tcl list after all other normal substitutions have been done, and the words of that list are used as words in the resulting command being built up. {*}$argv is a degenerate case where the remainder of the word is just the a read from a variable; the words that are used in the command are the elements of the list in the argv variable. Since that's normally a list, this is all hunky-dory.
Here's an example:
set abc {a b c}
set dabcf [list d $abc f]
puts $dabcf; # ===> “d {a b c} f”
set eabcg [list e {*}$abc g]
puts $eabcg; # ===> “e a b c g”
See the difference? One produces three elements in the list, the other produces five. It makes even more sense with something somewhat longer:
set options {
-foreground blue
-background yellow
-text "This is eye-watering stuff!"
}
button .b1 {*}$options -command {puts "Ouch 1"}
button .b2 {*}$options -command {puts "Ouch 2"}
button .b3 {*}$options -command {puts "Ouch 3"}
pack .b1 .b2 .b3
With expansion, that all Just Works™. Without, you'd have to do something horrific with eval:
eval [list button .b1] [lrange $options 0 end] [list -command {puts "Ouch 1"}]
# etc.
This was difficult to get right, and tedious, so it caused lots of people (including Tcl and Tk maintainers!) many problems because they tended to take shortcuts and get it wrong. It was to address this that expansion syntax was created in Tcl 8.5 to make this all less error prone. (The prototypical example in plain Tcl tends to involve things with exec, and meant that quite a few people actually had security holes because of this.)
As a bonus, using {*} is much faster than using eval since expansion can guarantee that it is never doing complicated reparsing of things. In Tcl, faster virtually always correlates with safer.
Note that this is independent of whether the variable is qualified. Yes, that means that you can also have {*}$::argv if you want.

You confuse the effects of substitution with the effects of argument expansion.
Please study the Dodekalogue http://wiki.tcl.tk/10259.
You mix the Rule #5: Argument Expansion (the {*} thing) with Variable Substitution (Rule #8).
The three forms you listed above are equivalent to the following:
$argv -> [set argv]
Get the value of a simple variable in the currently active scope.
$::argv -> [namespace eval :: { set argv }] -> [set ::argv]
Get the value of the variable in the namespace :: (the global namespace)
{*}$argv -> [eval [set argv]]
Expand the variables content to multiple arguments.

Related

info vars command is not working properly inside a proc

I am tring to do some variable auto-completion using TCL (this is intended for jimtcl)
I have tried the following sequence in both tclsh and jimsh:
% set VAR1 1
1
% set VAR2 2
2
% info vars
.... tcl_pkgPath VAR1 tcl_patchLevel VAR2 argc ...
% set pattern \$V*
$V*
% set vars_pattern [string range $pattern 1 end]
V*
% puts [lsort [info vars $vars_pattern]]
VAR1 VAR2
%
this is fine.
but once I get this into a proc
% proc autocomplete_helper pattern {
# check for variables auto-completion
puts "pattern '$pattern'"
if {[regexp {\$\S+$} $pattern match]} {
set vars_pattern [string range $match 1 end]
puts "pattern '$vars_pattern'"
return [lsort [info vars $vars_pattern]]
}
puts "other stuff to do"
}
% autocomplete_helper zerazer
pattern 'zerazer'
other stuff to do
% autocomplete_helper \$V*
pattern '$V*'
pattern 'V*
%
do you have any idea why this is not working ?
The info vars command is sensitive to what its current context (obviously; it returns the currently-visible variables) and moving things into a procedure changes that. The right fix for this is to use uplevel to run the command in a different context, either uplevel 1 to run in the caller's context or uplevel #0 to run in the global context (the one at the top of the stack).
In this case, we need to be a little careful because the pattern could have metacharacters in it (it'd be weird but legal) and uplevel is eval-like; the list command will ensure we've got a well-formed command. Putting this line into your procedure at the obvious point (everything else unchanged)
# The double quotes around #0 are to fool the highlighter used on Stack Overflow
return [lsort [uplevel "#0" [list info vars $vars_pattern]]]
With that, I can do this:
% autocomplete_helper {$e*}
pattern '$e*'
pattern 'e*'
env errorCode errorInfo
Which looks right to me.
This is a namespace problem.
A proc has its own namespace. When you're running info vars at the tclsh prompt, that's the global :: namespace.
The simplest thing to do in your proc would be to add :: to your argument to info vars
return [lsort [info vars ::$vars_pattern]]
The return values will include the :: namespace prefix, so remove that first if you need to.
Funny that you're seeing this problem with an auto-completion application. I've written a Tcl script to dump out all my procs, commands, namespaces, etc into json files that I read into Vim for a custom auto-completion plugin. I found the very same problems while writing that.

Tcl: Evaluate the variable value when passing argument to class

I have a piece of code like this
oo::class create class_test {
variable title_text
variable result
method title {t} {
set title_text $t
set result [format "%-6s %-6s" {*}$title_text]
}
method print {} {
return $result
}
}
set a "abcde"
set b "fghij"
class_test create foo
foo title {"$a" "$b"}
puts [foo print]
The real output is
$a $b
While the expected output is
abcde efghi
Could someone help to fix it?
Change
foo title {"$a" "$b"}
to
foo title [list $a $b]
so that the variables get substituted by their values.
You want to expand substitutions inside a {brace-quoted} string (logically) after the point that it is written in the script. This isn't usually recommended (not when you can construct arguments with list correctly), but you can do it.
method title {t} {
set title_text [lmap value $t {
uplevel 1 [list subst $value]
}]
set result [format "%-6s %-6s" {*}$title_text]
}
We do the transform on each word in the argument (lmap) and the transform is to apply subst to it, which must be done in the caller's context (uplevel 1). The use of list in there is so that we guarantee that we make a substitution-free script to run in the outer context, a very strongly recommended practice.
A feature of TclOO is that you don't need to take special precautions to use uplevel (or upvar) when using it, unlike some other older object systems for Tcl. That makes doing this sort of thing in a method no more tricky than doing it in a normal procedure. This is true even when inheritance is present.
Could someone help to fix it?
I fail to see why you, first, pack variable references into a single value and, then, uplevel-substitute them. In addition, the number of value arguments to format seem fixed. Why not just use two separate formal parameters to your title method and use them directly?
method title {v1 v2} {
set result [format "%-6s %-6s" $v1 $v2]
}
Then just call it like so:
foo title $a $b
Update
to generate the title in different length
then better use args like so?
method title {args} {
set result [format [join [lrepeat [llength $args] "%-6s"] " "] {*}$args]
}
args is the natural way of having a method (proc) with variable arguments in Tcl.

Is there any Tcl package/add-on that handles named arguments?

In Python, Ruby 2.0, Perl 6, and some hardware description languages, one can use named arguments. See this example. This makes the code more readable, easy to maintain, etc. Is there a way of getting it done/extension, in TCL 8.6, other than using a dictionary as a workaround?
In 8.6, use a dictionary parsed from args. The dict merge command can help:
proc example args {
if {[llength $args] % 2} {
return -code error "wrong # args: should be \"example ?-abc abc? ?-def def?\""
}
set defaults {
-abc 123
-def 456
}
set args [dict merge $defaults $args]
set abc [dict get $args -abc]
set def [dict get $args -def]
puts "abc=$abc, def=$def"
}
example; # abc=123, def=456
example -abc 789; # abc=789, def=456
example -def 789; # abc=123, def=789
example -def 246 -abc 135; # abc=135, def=246
You can go further than that with verifying (the tcl::prefix command can help) but it's a lot more work and doesn't buy you a lot more in production code. Not that that has stopped people from trying.
There are two proposals to add full named argument handling
(TIP #457, TIP #479) to 8.7 at the moment, but I'm not sure that either have really gained traction. (The problem from my perspective is the extra runtime cost that has to be borne by code that doesn't volunteer to support named arguments. There might be other issues too, such as disagreement over preferred syntax; I've not paid so much attention to that as I'm still fretting over the performance implications in a pretty hot piece of code.)
There is an entire page on the tcler's wiki that discusses named arguments: http://wiki.tcl.tk/10702
You can do it yourself with a little creativity. There are several mechanisms that allow you to do this:
procs can define other procs
proc behave just like a proc (the function definition system is not a syntax, it is a function call)
procs can use the args argument instead of positional parameter and manually process the list of arguments
you can execute code in any parent stack frame using uplevel
you can pull variables from any parent stack frame using upvar
everything is a string!
etc.
I'm not sure I've listed all the possible mechanisms.
My personal implementation of this idea is optproc: http://wiki.tcl.tk/20066
The implementation itself is quite simple:
proc optproc {name args script} {
proc $name args [
string map [list ARGS $args SCRIPT $script] {
foreach var {ARGS} {
set [lindex $var 0] [lindex $var 1]
}
foreach {var val} $args {
set [string trim $var -] $val
}
SCRIPT
}
]
}
I basically used string manipulation (string map) to directly insert the function body ($script) into the defined function without any substitutions etc. This is to avoid any $ or [] from being processed. There are many ways to do this but my go-to tool is string map.
This is similar to Donald's answer except for two things:
I don't transform args into a dict instead I manually process args and declare each local variable in a loop.
This is a meta solution. Instead of processing args I created another function (syntax) to create a function that processes args.
Usage (stolen from Donald's answer) would be:
optproc example {abc def} {
puts "abc=$abc, def=$def"
}
But note that my solution is not the only solution. There are many ways to do this limited only by creativity.

About passing around Tcl arrays holding lists

First off: I could fix my problem by myself, but I don't understand why my original solution did not work, and this is what I am interested in. I tried to make a compact example here:
I am dynamically building arrays, each array value being a list. Let's start with the following program:
# 'collector' is a callback function, expecting a container array, and some
# data used to populate the array.
proc generate { collector arr_name } {
eval $collector $arr_name first XXX YYY
eval $collector $arr_name second UUU VVV
}
# This is the callback function used in our example
proc collect { container_name key valuex valuey } {
upvar $container_name container
lappend container($key) [list $valuex $valuey]
}
# Procedure to write out an array
proc dump { arr_name } {
upvar $arr_name arr
puts $arr_name:
foreach key [array names arr] {
puts "$key : $arr($key)"
}
}
# Main program
array set containerA {}
generate [namespace code { collect }] containerA
dump containerA
Up to this point, nothing spectacular. Running this program produces the output
containerA:
second : {UUU VVV}
first : {XXX YYY}
But now let's extend this program somewhat
# Wrapper function to call 'generate' using a fixed collector function
# ("Currying" the first argument to generate)
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace code { collect }] $container_name ; # This works
# This would not work:
#generate [namespace code { collect }] container
}
array set containerB {}
coll_gen containerB
dump containerB
As written here, this would work too, and we get the output
containerB:
second : {UUU VVV}
first : {XXX YYY}
Now to my question: As you already can guess from the comments in the code, I had first written coll_gen as
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace code { collect }] container
}
My reasoning was that, since container is an alias to the array, the name of which was passed via the parameter list, I could equally well pass on the name of this alias to the 'generate' function. However, when I run the code (Tcl 8.5), it turns out that containerB is empty.
Why is it that it didn't work this way too?
The issue is one of evaluation scope.
Let's write out the call stack at the point where you're inside collect in the case where things don't work:
::
coll_gen containerB
generate {namespace inscope :: { collect }} container
namespace inscope :: { collect } container first XXX YYY
collect container first XXX YYY
Whoops! What's that namespace inscope? Where are the inner layers upvaring to? The result of namespace code is a wrapping with namespace inscope (which you shouldn't write directly; use namespace code or namespace eval) that arranges for the script formed by appending the other arguments (with appropriate metacharacter protection) to be run in the given namespace (:: in your case, I assume). This “run in the given namespace” requires adding another stack frame, and that's what the upvar is then poking into (it's probably created a global array called container, since the namespace inscope frame is a namespace-coupled one, not a “procedure local” stack frame).
You could use upvar 2 or maybe even upvar 3 (I'm not quite sure which) inside collect to work around this, but that's horrific and fragile.
You're better off writing your code like this:
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace which collect] container
}
proc generate { collector arr_name } {
upvar 1 $arr_name collectorVar
eval $collector collectorVar first XXX YYY
eval $collector collectorVar second UUU VVV
}
With that, the call stack will become this:
::
coll_gen containerB
generate ::collect container
::collect collectorVar first XXX YYY
Annotating with what the array is called inside each level…
:: ### containerB
coll_gen containerB ### container (→ containerB)
generate ::collect container ### collectorVar (→ container → containerB)
::collect collectorVar first XXX YYY ### container (→ collectorVar → container → containerB)
Tcl is very literal, and I find it helps to think in terms of strings as far as possible, similar to how you think in terms of symbols when using Lisp but even more pervasive. When you use upvar, what you get isn't anything like a reference variable in some other languages. You just get to refer to a Tcl_Obj that was originally referenced in another stack frame (or the same stack frame if you upvar 0) using a local name. In the invocation
generate [namespace code { collect }] container
the second argument to generate doesn't carry over any kind of reference to the Tcl_Obj that container referred to inside coll_gen: the argument is just a Tcl_Obj containing the string "container". If that string is equal to a valid name in one of the stack frames, you can upvar the name to get/be able to set a value in the associated object (and if you've managed the stack frames correctly, it will even be the object you wanted to access).
The commands upvar and uplevel have important uses, but you really don't need them here. If you just go with names and don't try to drag your objects with you through each stack frame, your code becomes easier to read and easier to maintain:
proc generate args {
# use eval $args first XXX YYY if you have Tcl 8.4 or earlier
{*}$args first XXX YYY
{*}$args second UUU VVV
}
proc collect {container_name key args} {
lappend ${container_name}($key) $args
}
proc dump arr_name {
puts $arr_name:
dict for {key val} [array get $arr_name] {
puts "$key : $val"
}
}
proc coll_gen container_name {
generate [namespace code collect] $container_name
}
array set containerB {}
set container_name [namespace which -variable containerB]
foreach cmd {coll_gen dump} {$cmd $container_name}
A variable created (by assignment or the variable command) in the global scope will be a namespace variable that exists independent of stack frames: every proc in the program will be able to reach it using an absolute reference (such as created by namespace which or simply prepending the namespace to the variable name).
Local variables, OTOH, are disambiguated by name and stack frame. Within a stack frame, every use of a certain variable name will reference the same object. In the simple case, a proc will execute in one stack frame only, but the uplevel command may cause some piece of code to execute in another stack frame. In that case, the same name may be used to refer to different objects in the same code body. There is no ambiguity, though: the level of execution determines what object a name refers to.
When using the upvar command, two different name + stack frame permutations can be used to reference the same object residing on some stack level, or the same name can be used to reference objects from different stack levels:
proc foo {} {set abc foo ; bar}
proc bar {} {set abc bar ; baz}
proc baz {} {set abc baz ; qux}
proc qux {} {
set abc qux
foreach n {3 2 1 0} {
upvar $n abc var
lappend res $var
}
puts [join $res { }]
}
foo
# => foo bar baz qux
Again, there is never any ambiguity, since the name + stack level designation makes the identity of the object clear.
The uplevel and upvar commands can be wonderfully convenient as long as you can keep the stack frames straight, and I for one use them all the time. As you saw in Donal's answer, though, even a Tcl ace can't always keep the stack frames straight, and in those cases namespace variables are much simpler and safer.
Documentation: array, dict, foreach, lappend, namespace, proc, puts, set, {*}, uplevel, upvar

"can't read:variable is array" error in ns2

In ns2, I declared a simple array using
array set ktree {}
then I tried to use it as a GOD variable as
create-god $ktree
but this gives the error
can't read "ktree": variable is array
while executing
"create-god $ktree {}"
Any help is greatly appreciated.
In Tcl, $varName means “read from the variable called varName” and is not a general reference to the variable (unlike some other languages, notably Perl and PHP, which do rather different things). Reading from a whole array, instead of an element of that array, is always an error in Tcl.
To pass an array to a command, you pass the name of that array in. It's then up to that command to access it as it sees fit. For procedures and methods written in Tcl, it'll typically involve upvar to link the array into a local view. (Things written directly in C or C++ have far fewer restrictions as they don't automatically push a Tcl stack frame.)
Note however that the command must be expecting the name of an array when you pass that name in. (Good programmers will document this fact, of course.) Whether create-god does, I really have no idea; it's not a general Tcl command but rather something that's more specific. (Part of ns2? Or maybe your own code.)
Example of passing in an array
An example of passing in an array by name is the parray command that should be part of every Tcl distribution. It's a procedure that prints an array out. Here's the source code without a few boiler-plate comments:
proc parray {a {pattern *}} {
upvar 1 $a array
if {![array exists array]} {
error "\"$a\" isn't an array"
}
set maxl 0
set names [lsort [array names array $pattern]]
foreach name $names {
if {[string length $name] > $maxl} {
set maxl [string length $name]
}
}
set maxl [expr {$maxl + [string length $a] + 2}]
foreach name $names {
set nameString [format %s(%s) $a $name]
puts stdout [format "%-*s = %s" $maxl $nameString $array($name)]
}
}
The key thing here is that we first see upvar 1 to bind the named variable in the caller to a local variable, and a test with array exists to see if the user really passed in an array (so as to give a good error message rather than a rubbishy one). Everything else then is just the implementation of how to actually pretty-print an associative array (finding out the max key length and doing some formatted output); it's just plain Tcl code.