Is there any Tcl package/add-on that handles named arguments? - tcl

In Python, Ruby 2.0, Perl 6, and some hardware description languages, one can use named arguments. See this example. This makes the code more readable, easy to maintain, etc. Is there a way of getting it done/extension, in TCL 8.6, other than using a dictionary as a workaround?

In 8.6, use a dictionary parsed from args. The dict merge command can help:
proc example args {
if {[llength $args] % 2} {
return -code error "wrong # args: should be \"example ?-abc abc? ?-def def?\""
}
set defaults {
-abc 123
-def 456
}
set args [dict merge $defaults $args]
set abc [dict get $args -abc]
set def [dict get $args -def]
puts "abc=$abc, def=$def"
}
example; # abc=123, def=456
example -abc 789; # abc=789, def=456
example -def 789; # abc=123, def=789
example -def 246 -abc 135; # abc=135, def=246
You can go further than that with verifying (the tcl::prefix command can help) but it's a lot more work and doesn't buy you a lot more in production code. Not that that has stopped people from trying.
There are two proposals to add full named argument handling
(TIP #457, TIP #479) to 8.7 at the moment, but I'm not sure that either have really gained traction. (The problem from my perspective is the extra runtime cost that has to be borne by code that doesn't volunteer to support named arguments. There might be other issues too, such as disagreement over preferred syntax; I've not paid so much attention to that as I'm still fretting over the performance implications in a pretty hot piece of code.)

There is an entire page on the tcler's wiki that discusses named arguments: http://wiki.tcl.tk/10702
You can do it yourself with a little creativity. There are several mechanisms that allow you to do this:
procs can define other procs
proc behave just like a proc (the function definition system is not a syntax, it is a function call)
procs can use the args argument instead of positional parameter and manually process the list of arguments
you can execute code in any parent stack frame using uplevel
you can pull variables from any parent stack frame using upvar
everything is a string!
etc.
I'm not sure I've listed all the possible mechanisms.
My personal implementation of this idea is optproc: http://wiki.tcl.tk/20066
The implementation itself is quite simple:
proc optproc {name args script} {
proc $name args [
string map [list ARGS $args SCRIPT $script] {
foreach var {ARGS} {
set [lindex $var 0] [lindex $var 1]
}
foreach {var val} $args {
set [string trim $var -] $val
}
SCRIPT
}
]
}
I basically used string manipulation (string map) to directly insert the function body ($script) into the defined function without any substitutions etc. This is to avoid any $ or [] from being processed. There are many ways to do this but my go-to tool is string map.
This is similar to Donald's answer except for two things:
I don't transform args into a dict instead I manually process args and declare each local variable in a loop.
This is a meta solution. Instead of processing args I created another function (syntax) to create a function that processes args.
Usage (stolen from Donald's answer) would be:
optproc example {abc def} {
puts "abc=$abc, def=$def"
}
But note that my solution is not the only solution. There are many ways to do this limited only by creativity.

Related

Tcl - Differentiate between list/dict and anonymous proc

I wrote the following proc, which simulates the filter function in Lodash (javascript library) (https://lodash.com/docs/4.17.4#filter). You can call it in 3.5 basic formats, seen in the examples section. For the latter three calling options I would like to get rid of the the requirement to send in -s (shorthand). In order to do that I need to differentiate between an anonymous proc and a list/dict/string.
I tried looking at string is, but there isn't a string is proc. In researching here: http://wiki.tcl.tk/10166 I found they recommend info complete, however in most cases the parameters would pass that test regardless of the type of parameter.
Does anyone know of a way to reliable test this? I know I could leave it or change the proc definition, but I'm trying to stay as true as possible to Lodash.
Examples:
set users [list \
[dict create user barney age 36 active true] \
[dict create user fred age 40 active false] \
]
1. set result [_filter [list 1 2 3 4] {x {return true}}]
2. set result [_filter $users -s [dict create age 36 active true]]
3. set result [_filter $users -s [list age 36]]
4. set result [_filter $users -s "active"]
Proc Code:
proc _filter {collection predicate args} {
# They want to use shorthand syntax
if {$predicate=="-s"} {
# They passed a list/dict
if {[_dictIs {*}$args]} {
set predicate {x {
upvar args args
set truthy 1
dict for {k v} {*}$args {
if {[dict get $x $k]!=$v} {
set truthy false
break
}
}
return $truthy
}}
# They passed just an individual string
} else {
set predicate {x {
upvar args args;
if {[dict get $x $args]} {
return true;
}
return false;
}}
}
}
# Start the result list and the index (which may not be used)
set result {}
set i -1
# For each item in collection apply the iteratee.
# Dynamically pass the correct parameters.
set paramLen [llength [lindex $predicate 0]]
foreach item $collection {
set param [list $item]
if {$paramLen>=2} {lappend param [incr i];}
if {$paramLen>=3} {lappend param $collection;}
if {[apply $predicate {*}$param]} {
lappend result $item
}
}
return $result
}
Is x {return true} a string, a list, a dictionary or a lambda term (the correct name for an anonymous proc)?
The truth is that it may be all of them; it would be correct to say it was a value that was a member of any of the mentioned types. You need to describe your intent more precisely and explicitly rather than hiding it inside some sort of type magic. That greater precision may be achieved by using an option like -s or by different main command names, but it is still necessary either way. You cannot correctly and safely do what you seek to do.
In a little more depth…
All Tcl values are valid as strings.
Lists have a defined syntax and are properly subtypes of strings. (They're implemented differently internally, but you are supposed to ignore such details.)
Dictionaries have a syntax that is equivalent to lists with even numbers of elements where the elements at the even indices are all unique from each other.
Lambda terms are lists with two or three elements (the third element is the name of the context namespace, and defaults to the global namespace if it is absent). The first element of the list needs to be a valid list as well.
A two-element list matches the requirements for all the above. In Tcl's actual type logic, it is simultaneously all of the above. A particular instantiation of the value might have a particular implementation representation under the covers, but that is a transient thing that does not reflect the true type of the value.
Tcl's type system is different to that of many other languages.

TCL/Expect - $argv VS $::argv VS {*}$argv

What is difference between following variables:
$argv
$::argv
{*}$argv
First two are possible to print via puts command and they returns following output:
param0 param1 {param 2} param3
param0 param1 {param 2} param3
The arguments that was passed to script were:
param0 param1 "param 2" param3
The last one end up with error:
wrong # args: should be "puts ?-nonewline? ?channelId? string"
while executing
"puts {*}$argv"
I've done some research in this area using following code:
if {[array exists $argv]} {
puts "\$argv IS ARRAY"
} else {
puts "\$argv IS NOT AN ARRAY"
}
if {[string is list $argv]} {
puts "\$argv IS LIST"
} else {
puts "\$argv IS NOT LIST"
}
if {[array exists $::argv]} {
puts "\$::argv IS ARRAY"
} else {
puts "\$::argv IS NOT AN ARRAY"
}
if {[string is list $::argv]} {
puts "\$::argv IS LIST"
} else {
puts "\$::argv IS NOT LIST"
}
if {[array exists {*}$argv]} {
puts "{*}\$::argv IS ARRAY"
} else {
puts "{*}\$::argv IS NOT AN ARRAY"
}
if {[string is list {*}$argv]} {
puts "{*}\$::argv IS LIST"
} else {
puts "{*}\$::argv IS NOT LIST"
}
The last two if-else statements which contain {*}$argv ends with following error:
wrong # args: should be "array exists arrayName"
while executing
"array exists {*}$argv"
invoked from within
"if {[array exists {*}$argv]} {
puts "{*}\$::argv IS ARRAY"
} else {
puts "{*}\$::argv IS NOT AN ARRAY"
}"
Commenting out those two statements shows that $argv and $::argv are lists:
argv IS NOT AN ARRAY
$argv IS NOT AN ARRAY
argv IS LIST
$argv IS LIST
Both those lists can be traversed as standard list e.g.:
foreach item $argv {
puts $item
}
or
foreach item $::argv {
puts $item
}
Attempt to traverse {*}$argv the same way leads to following error again:
wrong # args: should be "foreach varList list ?varList list ...? command"
while executing
"foreach item {*}$argv {
puts $item
}"
I am using TCL version 8.5
What is difference between following variables:
$argv
$::argv
{*}$argv
There are two types of difference here.
Unqualified and Qualified Variables
In Tcl, unqualified and qualified variables can be a bit different, but it depends on the context (in a pretty simple way though). Firstly, a qualified variable name is one that contains at least one :: within it. If the variable name (the thing after the $ — in Tcl, $ just means “read this variable now and use its contents here”) starts with ::, it is an absolute variable name, otherwise a qualified variable name is a relative variable name and is resolved with respect to the current namespace (which you can find out with namespace current if you're uncertain). Absolute variable names always refer to the same thing, in all contexts. Thus, ::argv is an absolute variable name, and indeed it refers to a variable called argv in the top-level, global namespace. That happens to be a variable that tclsh and wish write their arguments into.
But if there is no ::, it is an unqualified variable name. If you are not in a procedure (or procedure-like thing, which includes a lambda term such as you'd use with apply or the methods defined by various OO systems) then the variable is (mostly) treated as if it was a relative variable name and resolved with respect to the current namespace. namespace eval and namespace code are two of the things that can change the current namespace (the others are more obscure). All this is provided you use variable to declare all your namespace variables. Otherwise, you can hit some weird problems with variable resolution which are really nasty. So do use variable. Really.
If you are in a procedure(-like entity) though, that unqualified name refers to a local variable, whose life is coupled to that of the stack frame pushed on the stack when the procedure is entered. That can be linked to variables in other scopes (including the global namespace) through various commands: global, upvar, variable, and namespace upvar. However, the actual resolution of the variable is to something local.
Finally, there might also be a custom variable resolver in place. Since you're using Tcl 8.5, the place where you're most likely to see this in use is if you're using Incr Tcl, an object system for Tcl. Custom variable resolvers can do some complex stuff. (If you were using Tcl 8.6, the most likely place to see a custom variable resolver at work is in TclOO. The variable resolver there is very conservative and cautious, but allows local variables to be bound to object variables without having to explicitly declare this in each method).
Normal and Expanding Substitution
The difference between $argv and {*}$argv is totally different.
$argv is a normal substitution. It says “read this variable here and use the contents of it instead”. It can be used in the middle of a word, so $argv$argv$argv is a thing, consisting of the concatenation of the contents of the argv variable three times.
{*}, when placed at the start of a word (it's not special elsewhere), marks that word for expansion. When a word is expanded, it's parsed as a Tcl list after all other normal substitutions have been done, and the words of that list are used as words in the resulting command being built up. {*}$argv is a degenerate case where the remainder of the word is just the a read from a variable; the words that are used in the command are the elements of the list in the argv variable. Since that's normally a list, this is all hunky-dory.
Here's an example:
set abc {a b c}
set dabcf [list d $abc f]
puts $dabcf; # ===> “d {a b c} f”
set eabcg [list e {*}$abc g]
puts $eabcg; # ===> “e a b c g”
See the difference? One produces three elements in the list, the other produces five. It makes even more sense with something somewhat longer:
set options {
-foreground blue
-background yellow
-text "This is eye-watering stuff!"
}
button .b1 {*}$options -command {puts "Ouch 1"}
button .b2 {*}$options -command {puts "Ouch 2"}
button .b3 {*}$options -command {puts "Ouch 3"}
pack .b1 .b2 .b3
With expansion, that all Just Works™. Without, you'd have to do something horrific with eval:
eval [list button .b1] [lrange $options 0 end] [list -command {puts "Ouch 1"}]
# etc.
This was difficult to get right, and tedious, so it caused lots of people (including Tcl and Tk maintainers!) many problems because they tended to take shortcuts and get it wrong. It was to address this that expansion syntax was created in Tcl 8.5 to make this all less error prone. (The prototypical example in plain Tcl tends to involve things with exec, and meant that quite a few people actually had security holes because of this.)
As a bonus, using {*} is much faster than using eval since expansion can guarantee that it is never doing complicated reparsing of things. In Tcl, faster virtually always correlates with safer.
Note that this is independent of whether the variable is qualified. Yes, that means that you can also have {*}$::argv if you want.
You confuse the effects of substitution with the effects of argument expansion.
Please study the Dodekalogue http://wiki.tcl.tk/10259.
You mix the Rule #5: Argument Expansion (the {*} thing) with Variable Substitution (Rule #8).
The three forms you listed above are equivalent to the following:
$argv -> [set argv]
Get the value of a simple variable in the currently active scope.
$::argv -> [namespace eval :: { set argv }] -> [set ::argv]
Get the value of the variable in the namespace :: (the global namespace)
{*}$argv -> [eval [set argv]]
Expand the variables content to multiple arguments.

About passing around Tcl arrays holding lists

First off: I could fix my problem by myself, but I don't understand why my original solution did not work, and this is what I am interested in. I tried to make a compact example here:
I am dynamically building arrays, each array value being a list. Let's start with the following program:
# 'collector' is a callback function, expecting a container array, and some
# data used to populate the array.
proc generate { collector arr_name } {
eval $collector $arr_name first XXX YYY
eval $collector $arr_name second UUU VVV
}
# This is the callback function used in our example
proc collect { container_name key valuex valuey } {
upvar $container_name container
lappend container($key) [list $valuex $valuey]
}
# Procedure to write out an array
proc dump { arr_name } {
upvar $arr_name arr
puts $arr_name:
foreach key [array names arr] {
puts "$key : $arr($key)"
}
}
# Main program
array set containerA {}
generate [namespace code { collect }] containerA
dump containerA
Up to this point, nothing spectacular. Running this program produces the output
containerA:
second : {UUU VVV}
first : {XXX YYY}
But now let's extend this program somewhat
# Wrapper function to call 'generate' using a fixed collector function
# ("Currying" the first argument to generate)
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace code { collect }] $container_name ; # This works
# This would not work:
#generate [namespace code { collect }] container
}
array set containerB {}
coll_gen containerB
dump containerB
As written here, this would work too, and we get the output
containerB:
second : {UUU VVV}
first : {XXX YYY}
Now to my question: As you already can guess from the comments in the code, I had first written coll_gen as
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace code { collect }] container
}
My reasoning was that, since container is an alias to the array, the name of which was passed via the parameter list, I could equally well pass on the name of this alias to the 'generate' function. However, when I run the code (Tcl 8.5), it turns out that containerB is empty.
Why is it that it didn't work this way too?
The issue is one of evaluation scope.
Let's write out the call stack at the point where you're inside collect in the case where things don't work:
::
coll_gen containerB
generate {namespace inscope :: { collect }} container
namespace inscope :: { collect } container first XXX YYY
collect container first XXX YYY
Whoops! What's that namespace inscope? Where are the inner layers upvaring to? The result of namespace code is a wrapping with namespace inscope (which you shouldn't write directly; use namespace code or namespace eval) that arranges for the script formed by appending the other arguments (with appropriate metacharacter protection) to be run in the given namespace (:: in your case, I assume). This “run in the given namespace” requires adding another stack frame, and that's what the upvar is then poking into (it's probably created a global array called container, since the namespace inscope frame is a namespace-coupled one, not a “procedure local” stack frame).
You could use upvar 2 or maybe even upvar 3 (I'm not quite sure which) inside collect to work around this, but that's horrific and fragile.
You're better off writing your code like this:
proc coll_gen { container_name } {
upvar $container_name container
generate [namespace which collect] container
}
proc generate { collector arr_name } {
upvar 1 $arr_name collectorVar
eval $collector collectorVar first XXX YYY
eval $collector collectorVar second UUU VVV
}
With that, the call stack will become this:
::
coll_gen containerB
generate ::collect container
::collect collectorVar first XXX YYY
Annotating with what the array is called inside each level…
:: ### containerB
coll_gen containerB ### container (→ containerB)
generate ::collect container ### collectorVar (→ container → containerB)
::collect collectorVar first XXX YYY ### container (→ collectorVar → container → containerB)
Tcl is very literal, and I find it helps to think in terms of strings as far as possible, similar to how you think in terms of symbols when using Lisp but even more pervasive. When you use upvar, what you get isn't anything like a reference variable in some other languages. You just get to refer to a Tcl_Obj that was originally referenced in another stack frame (or the same stack frame if you upvar 0) using a local name. In the invocation
generate [namespace code { collect }] container
the second argument to generate doesn't carry over any kind of reference to the Tcl_Obj that container referred to inside coll_gen: the argument is just a Tcl_Obj containing the string "container". If that string is equal to a valid name in one of the stack frames, you can upvar the name to get/be able to set a value in the associated object (and if you've managed the stack frames correctly, it will even be the object you wanted to access).
The commands upvar and uplevel have important uses, but you really don't need them here. If you just go with names and don't try to drag your objects with you through each stack frame, your code becomes easier to read and easier to maintain:
proc generate args {
# use eval $args first XXX YYY if you have Tcl 8.4 or earlier
{*}$args first XXX YYY
{*}$args second UUU VVV
}
proc collect {container_name key args} {
lappend ${container_name}($key) $args
}
proc dump arr_name {
puts $arr_name:
dict for {key val} [array get $arr_name] {
puts "$key : $val"
}
}
proc coll_gen container_name {
generate [namespace code collect] $container_name
}
array set containerB {}
set container_name [namespace which -variable containerB]
foreach cmd {coll_gen dump} {$cmd $container_name}
A variable created (by assignment or the variable command) in the global scope will be a namespace variable that exists independent of stack frames: every proc in the program will be able to reach it using an absolute reference (such as created by namespace which or simply prepending the namespace to the variable name).
Local variables, OTOH, are disambiguated by name and stack frame. Within a stack frame, every use of a certain variable name will reference the same object. In the simple case, a proc will execute in one stack frame only, but the uplevel command may cause some piece of code to execute in another stack frame. In that case, the same name may be used to refer to different objects in the same code body. There is no ambiguity, though: the level of execution determines what object a name refers to.
When using the upvar command, two different name + stack frame permutations can be used to reference the same object residing on some stack level, or the same name can be used to reference objects from different stack levels:
proc foo {} {set abc foo ; bar}
proc bar {} {set abc bar ; baz}
proc baz {} {set abc baz ; qux}
proc qux {} {
set abc qux
foreach n {3 2 1 0} {
upvar $n abc var
lappend res $var
}
puts [join $res { }]
}
foo
# => foo bar baz qux
Again, there is never any ambiguity, since the name + stack level designation makes the identity of the object clear.
The uplevel and upvar commands can be wonderfully convenient as long as you can keep the stack frames straight, and I for one use them all the time. As you saw in Donal's answer, though, even a Tcl ace can't always keep the stack frames straight, and in those cases namespace variables are much simpler and safer.
Documentation: array, dict, foreach, lappend, namespace, proc, puts, set, {*}, uplevel, upvar

"can't read:variable is array" error in ns2

In ns2, I declared a simple array using
array set ktree {}
then I tried to use it as a GOD variable as
create-god $ktree
but this gives the error
can't read "ktree": variable is array
while executing
"create-god $ktree {}"
Any help is greatly appreciated.
In Tcl, $varName means “read from the variable called varName” and is not a general reference to the variable (unlike some other languages, notably Perl and PHP, which do rather different things). Reading from a whole array, instead of an element of that array, is always an error in Tcl.
To pass an array to a command, you pass the name of that array in. It's then up to that command to access it as it sees fit. For procedures and methods written in Tcl, it'll typically involve upvar to link the array into a local view. (Things written directly in C or C++ have far fewer restrictions as they don't automatically push a Tcl stack frame.)
Note however that the command must be expecting the name of an array when you pass that name in. (Good programmers will document this fact, of course.) Whether create-god does, I really have no idea; it's not a general Tcl command but rather something that's more specific. (Part of ns2? Or maybe your own code.)
Example of passing in an array
An example of passing in an array by name is the parray command that should be part of every Tcl distribution. It's a procedure that prints an array out. Here's the source code without a few boiler-plate comments:
proc parray {a {pattern *}} {
upvar 1 $a array
if {![array exists array]} {
error "\"$a\" isn't an array"
}
set maxl 0
set names [lsort [array names array $pattern]]
foreach name $names {
if {[string length $name] > $maxl} {
set maxl [string length $name]
}
}
set maxl [expr {$maxl + [string length $a] + 2}]
foreach name $names {
set nameString [format %s(%s) $a $name]
puts stdout [format "%-*s = %s" $maxl $nameString $array($name)]
}
}
The key thing here is that we first see upvar 1 to bind the named variable in the caller to a local variable, and a test with array exists to see if the user really passed in an array (so as to give a good error message rather than a rubbishy one). Everything else then is just the implementation of how to actually pretty-print an associative array (finding out the max key length and doing some formatted output); it's just plain Tcl code.

SWIG + TCL flags

Will the ownership of a pointer last only in the block in which we set the -acquire flag for it?
Eg.:
{
{
$xyz -acquire
}
}
Firstly, Tcl doesn't define blocks with {/}. The scope is defined by the procedure call or namespace.
Secondly, Tcl commands are always defined to have lifetime that corresponds to the namespace that owns them; they are never† scoped to a procedure call. They must be manually disposed one way or another; there are two ways to do this manual disposal: calling $xyz -delete or rename $xyz "" (or to anything else that is the empty string). Frankly, I prefer the first method.
But if you do want the lifespan to be tied to a procedure call, that's actually quite possible to do. It just requires some extra code:
proc tieLifespan args {
upvar 1 "____lifespan handle" v
if {[info exists v]} {
trace remove variable v unset $v
set args [concat [lindex $v 1] $args]
}
set v [concat Tie-Garbage-Collect $args]
trace add variable v unset $v
}
proc Tie-Garbage-Collect {handles var dummy1 dummy2} {
upvar 1 $var v
foreach handle $handles {
# According to SWIG docs, this is how to do explicit destruction
$handle -delete
# Alternatively: rename $handle ""
}
}
That you'd use like this in the scope that you want to tie $xyz's life to:
tieLifespan $xyz
# You can register multiple objects at once too
And that's it. When the procedure (or procedure-like entity if you're using Tcl 8.5 or later) exits, the tied object will be deleted. It's up to you to decide if that's what you really want; if you later disown the handle, you probably ought to not use tying.
† Well, hardly ever; some extensions do nasty things. Discount this statement as it doesn't apply to SWIG-generated code!