I have implemented many TCL extensions for a specific tool in the domain of formal methods (extensions are implemented in C but I do not want solution to rely on this fact). Thus, the users of my tool can use TCL for prototyping algorithms. Many of them are just linear list of commands (they are powerfull), e.g.:
my_read_file f
my_do_something a b c
my_do_something_else a b c
Now, I am interested in timing. It is possible to change the script to get:
puts [time [my_read_file f] 1]
puts [time [my_do_something a b c] 1]
puts [time [my_do_something_else a b c] 1]
Instead of this I want to define procedure xsource that executes a TCL script and get/write timing for all my commands. Some kind of a profiler. I wrote a naive implementation where the main idea is as follows:
set f [open [lindex $argv 0] r]
set inputLine ""
while {[gets $f line] >= 0} {
set d [expr [string length $line] - 1]
if { $d >= 0 } {
if { [string index $line 0] != "#" } {
if {[string index $line $d] == "\\"} {
set inputLine "$inputLine [string trimright [string range $line 0 [expr $d - 1]]]"
} else {
set inputLine "$inputLine $line"
set inputLine [string trimleft $inputLine]
puts $inputLine
puts [time {eval $inputLine} 1]
}
set inputLine ""
}
}
}
It works for linear list of commands and even allows comments and commands over multiple lines. But it fails if the user uses if statements, loops, and definition of procedures. Can you propose a better approach? It must be pure TCL script with as few extensions as possible.
One way of doing what you're asking for is to use execution traces. Here's a script that can do just that:
package require Tcl 8.5
# The machinery for tracking command execution times; prints the time taken
# upon termination of the command. More info is available too (e.g., did the
# command have an exception) but isn't printed here.
variable timerStack {}
proc timerEnter {cmd op} {
variable timerStack
lappend timerStack [clock microseconds]
}
proc timerLeave {cmd code result op} {
variable timerStack
set now [clock microseconds]
set then [lindex $timerStack end]
set timerStack [lrange $timerStack 0 end-1]
# Remove this length check to print everything out; could be a lot!
# Alternatively, modify the comparison to print more stack frames.
if {[llength $timerStack] < 1} {
puts "[expr {$now-$then}]: $cmd"
}
}
# Add the magic!
trace add execution source enterstep timerEnter
trace add execution source leavestep timerLeave
# And invoke the magic, magically
source [set argv [lassign $argv argv0];set argv0]
# Alternatively, if you don't want argument rewriting, just do:
# source yourScript.tcl
Then you'd call it like this (assuming you've put it in a file called timer.tcl):
tclsh8.5 timer.tcl yourScript.tcl
Be aware that this script has a considerable amount of overhead, as it inhibits many optimization strategies that are normally used. That won't matter too much for uses where you're doing the real meat in your own C code, but when it's lots of loops in Tcl then you'll notice a lot.
You can wrap your commands which you want to measure. And name wrappers exactly as the original ones (renaming original procs before). After that, when instrumented command is executed it actually executes the wrapper, which executes the original procedure and measure the time of execution. The example below (Tcl 8.5).
proc instrument {procs} {
set skip_procs {proc rename instrument puts time subst uplevel return}
foreach p $procs {
if {$p ni $skip_procs} {
uplevel [subst -nocommands {
rename $p __$p
proc $p {args} {
puts "$p: [time {set r [__$p {*}\$args]}]"
return \$r
}
}]
}
}
}
proc my_proc {a} {
set r 1
for {set i 1} {$i <= $a} {incr i} {
set r [expr {$r * $i}]
}
return $r
}
proc my_another_proc {a b} {
set r 0
for {set i $a} {$i <= $b} {incr i} {
incr r $i
}
return $r
}
instrument [info commands my_*]
puts "100 = [my_proc 100]"
puts "200 = [my_proc 100]"
puts "100 - 200 = [my_another_proc 100 200]"
You might want to look at the command "info complete". It can tell you if what you have accumulated so far looks complete from the point of view of most common Tcl syntax markers. It will deal with command input that might be spread across multiple physical lines.
Related
I have a variable, let's say xx, with a list of index 0 and index 1 values. I want to modify a script (not mine) which previously defines a function, pptable, i.e.,
proc pptable {l1 l2} {
foreach i1 $l1 i2 $l2 {
puts " [format %6.2f $i1]\t[format %6.2f $i2]"
}
}
so that it displays the output into two columns using
pptable [lindex $xx 1] [lindex $xx 0]
However, I want to write the output directly to a file. Could you tell me how I can send the data to a file instead to the display?
One of the neatest ways of doing this is to stack on a channel transform that redirects stdout to where you want it to go. This works even if the write to stdout happens from C code or in a different thread as it plugs into the channel machinery. The code is a little bit long (and requires Tcl 8.6) but is reliable and actually mostly very simple.
package require Tcl 8.6; # *REQUIRED* for [chan push] and [chan pop]
proc RedirectorCallback {targetHandle op args} {
# The switch/lassign pattern is simplest way of doing this in one procedure
switch $op {
initialize {
lassign $args handle mode
# Sanity check
if {$mode ne "write"} {
close $targetHandle
error "this is just a write transform"
}
# List of supported subcommands
return {initialize finalize write}
}
finalize {
lassign $args handle
# All we need to do here is close the target file handle
close $targetHandle
}
write {
lassign $args handle buffer
# Write the data to *real* destination; this does the redirect
puts -nonewline $targetHandle $buffer
# Stop the data going to *true* stdout by returning empty string
return ""
# If we returned the data instead, this would do a 'tee'
}
default {
error "unsupported subcommand"
}
}
}
# Here's a wrapper to make the transform easy to use
proc redirectStdout {file script} {
# Stack the transform onto stdout with the file handle to write to
# (which is going to be $targetHandle in [redirector])
chan push stdout [list RedirectorCallback [open $file "wb"]]
# Run the script and *definitely* pop the transform after it finishes
try {
uplevel 1 $script
} finally {
chan pop stdout
}
}
How would we actually use this? It's really very easy in practice:
# Exactly the code you started with
proc pptable {l1 l2} {
foreach i1 $l1 i2 $l2 {
puts " [format %6.2f $i1]\t[format %6.2f $i2]"
}
}
# Demonstrate that stdout is working as normal
puts "before"
# Our wrapped call that we're capturing the output from; pick your own filename!
redirectStdout "foo.txt" {
pptable {1.2 1.3 1.4} {6.9 6.8 6.7}
}
# Demonstrate that stdout is working as normal again
puts "after"
When I run that code, I get this:
bash$ tclsh8.6 stdout-redirect-example.tcl
before
after
bash$ cat foo.txt
1.20 6.90
1.30 6.80
1.40 6.70
I believe that's precisely what you are looking for.
You can do this with less code if you use Tcllib and TclOO to help deal with the machinery:
package require Tcl 8.6
package require tcl::transform::core
oo::class create WriteRedirector {
superclass tcl::transform::core
variable targetHandle
constructor {targetFile} {
set targetHandle [open $targetFile "wb"]
}
destructor {
close $targetHandle
}
method write {handle buffer} {
puts -nonewline $targetHandle $buffer
return ""
}
# This is the wrapper, as a class method
self method redirectDuring {channel targetFile script} {
chan push $channel [my new $targetFile]
try {
uplevel 1 $script
} finally {
chan pop $channel
}
}
}
Usage example:
proc pptable {l1 l2} {
foreach i1 $l1 i2 $l2 {
puts " [format %6.2f $i1]\t[format %6.2f $i2]"
}
}
puts "before"
WriteRedirector redirectDuring stdout "foo.txt" {
pptable {1.2 1.3 1.4 1.5} {6.9 6.8 6.7 6.6}
}
puts "after"
I assume you don't want or can't modify the existing script and proc pptable, correct?
If so, there are different options, depending on your exact situation:
Redirect stdout: tclsh yourscript.tcl > your.out
Redefine puts (for a clearly defined scope):
rename ::puts ::puts.orig
proc puts args {
set fh [open your.out w];
::puts.orig $fh $args;
close $fh
}
# run pptable, source the script
This theme has been covered before, e.g., tcl stop all output going to stdout channel?
Rewire Tcl's stdout channel (not necessarily recommended):
close stdout
open your.out w
# run pptable, source the script
This has also been elaborated on before, e.g. Tracing stdout and stderr in Tcl
I need to get invocation line number of tcl proc inside it’s body.
Starting from 8.5 tcl have info frame command which allows following:
proc printLine {} {
set lineNum [dict get [info frame 1] line]
}
I need the same for 8.4
It's not available in 8.4; the data wasn't collected at all. I guess you could search for a unique token in the line, but that'd be about all.
proc lineNumber {uniqueToken} {
set name [lindex [info level 1] 0]
set body [uplevel 2 [list info body $name]]
set num 0
foreach line [split $body \n] {
incr num
if {[string first $uniqueToken $line] >= 0} {
return $num
}
}
error "could not find token '$uniqueToken'"
}
Note that 8.4 is not supported any more. Upgrade.
I'm using tcl 8.5, but it should work on version 8.4. here is:
#!/usr/bin/tclsh
puts "tcl version: $tcl_version"
proc linum {} {
if {![string equal -nocase precompiled [lindex [info frame -1] 1]]} {
return [lindex [info frame -1] 3]
} else {
return Unknown
}
}
puts "call proc #line:[linum]"
and the result is:
tcl version: 8.5
call proc #line:13
you can reference info frame for more details
I have a file like this:
set position {0.50 0.50}
set visibility false
set text {ID: {entity.id}\n Value: {entity.contour_val}}
And I want to do something similar to source, but I want to use a file handle only.
My current attempt looks like this:
proc readArray {fileHandle arrayName} {
upvar $arrayName arr
set cl 0
while {! [eof $fileHandle]} {
set cl [expr "$cl + 1"]
set line [gets $fileHandle]
if [$line eq {}] continue
puts $line
namespace eval ::__esg_priv "
uplevel 1 {*}$line
"
info vars ::__esg_priv::*
foreach varPath [info vars ::__esg_priv::*] {
set varName [string map { ::__esg_priv:: "" } $varPath]
puts "Setting arr($varName) -> [set $varPath]"
set arr($varName) [set $varPath]
}
namespace delete __esg_priv
}
puts "$cl number of lines read"
}
In place of uplevel I tried many combinations of eval and quoting.
My problem is, it either fails on the lines with lists or it does not actuall set the variables.
What is the right way to do it, if the executed commands are expected to be any valid code.
An extra question would be how to properly apply error checking, which I haven't tried yet.
After a call to
readArray [open "myFile.tcl" r] arr
I expect that
parray arr
issues something like:
arr(position) = 0.50 0.50
arr(text) = ID: {entity.id}\n Value: {entity.contour_val}
arr(visibility) = false
BTW: The last line contains internal {}, which are supposed to make it into the string variables. And there is no intent to make this a dict.
This code works, but there are still some problems with it:
proc readArray {fileHandle arrayName} {
upvar $arrayName arr
set cl 0
while {! [eof $fileHandle]} {
incr cl ;# !
set line [gets $fileHandle]
if {$line eq {}} continue ;# !
puts $line
namespace eval ::__esg_priv $line ;# !
foreach varPath [info vars ::__esg_priv::*] {
set varName [string map { ::__esg_priv:: "" } $varPath]
puts "Setting arr($varName) -> [set $varPath]"
set arr($varName) [set $varPath]
}
namespace delete __esg_priv
}
puts "$cl number of lines read"
}
I've taken out a couple of lines that didn't seem necessary, and changed some lines a bit.
You don't need set cl [expr "$cl + 1"]: incr cl will do.
if [$line eq {}] continue will fail because the [...] is a command substitution. if {$line eq {}} continue (braces instead of brackets) does what you intend.
Unless you are accessing variables in another scope, you won't need uplevel. namespace eval ::__esg_priv $line will evaluate one line in the designated namespace.
I didn't change the following, but maybe you should:
set varName [string map { ::__esg_priv:: "" } $varPath] works as intended, but set varName [namespace tail $varPath] is cleaner.
Be aware that if there exists a global variable with the same name as one of the variables in your file, no namespace variable will be created; the global variable will be updated instead.
If you intend to use the value in the text variable as a dictionary, you need to remove either the \n or the braces.
According to your question title, you want to evaluate the file line by line. If that requirement can be lifted, your code could be simplified by reading the whole script in one operation and then evaluating it with a single namespace eval.
ETA
This solution is a lot more robust in that it reads the script in a sandbox (always a good idea when writing code that will execute arbitrary external code) and redefines (within that sandbox) the set command to create members in your array instead of regular variables.
proc readArray {fileHandle arrayName} {
upvar 1 $arrayName arr
set int [interp create -safe]
$int alias set apply {{name value} {
uplevel 1 [list set arr($name) $value]
}}
$int eval [read $fileHandle]
interp delete $int
}
To make it even more safe against unexpected interaction with global variables etc, look at the interp package in the Tcllib. It lets you create an interpreter that is completely empty.
Documentation: apply, continue, eof, foreach, gets, if, incr, info, interp package, interp, list, namespace, proc, puts, set, string, uplevel, upvar, while
I know I have been asking a lot of questions but I'm still learning tcl and I haven't found anything that similar to this issue anywhere so far. Is it at all possible to replace a set f commands in tcl with one variable function0 for example?
I want to be able to replace the following code;
set f [listFromFile $path1]
set f [lsort -unique $f]
set f [lsearch -all -inline $f "test_*"]
set f [regsub -all {,} $f "" ]
set len [llength $f]
set cnt 0
with a variable function0 because this same code appears numerous times within the script. I should mention it appears both in a proc and not in a proc
The above code relates to similar script as
while {$cnt < $len} {
puts [lindex $f $cnt]
incr cnt
after 25; #not needed, but for viewing purposes
}
Variables are for storing values. To hide away (encapsulate) some lines of code you need a command procedure, which you define using the proc command.
You wanted to hide away the following lines
set f [listFromFile $path1]
set f [lsort -unique $f]
set f [lsearch -all -inline $f "test_*"]
set f [regsub -all {,} $f "" ]
set len [llength $f]
set cnt 0
to be able to just invoke for instance function0 $path1 and have all those calculations made in one fell swoop. Further, you wanted to use the result of calling the procedure in code like this:
while {$cnt < $len} {
puts [lindex $f $cnt]
# ...
Which means you want function0 to produce three different values, stored in cnt, len, and f. There are several ways to have a command procedure return multiple values, but the cleanest solution here is to make it return a single value; the list that you want to print. The value in len can be calculated from that list with a single command, and the initialization of cnt is better performed outside the command procedure. What you get is this:
proc function0 path {
set f [listFromFile $path]
set f [lsort -unique $f]
set f [lsearch -all -inline $f test_*]
set f [regsub -all , $f {}]
return $f
}
which you can use like this:
set f [function0 $path1]
set len [llength $f]
set cnt 0
while {$cnt < $len} {
puts [lindex $f $cnt]
incr cnt
after 25; #not needed, but for viewing purposes
}
or like this:
set f [function0 $path1]
set len [llength $f]
for {set cnt 0} {$cnt < $len} {incr cnt} {
puts [lindex $f $cnt]
after 25; #not needed, but for viewing purposes
}
or like this:
set f [function0 $path1]
foreach item $f {
puts $item
after 25; #not needed, but for viewing purposes
}
This is why I didn't bother to create a procedure returning three values: you only really needed one.
glenn jackman makes a very good point (or two points, actually) in another answer about the use of regsub. For completeness, I will repeat it here.
Tcl is a bit confusing because it usually allows string operations (like string substitution) on data structures that aren't formally strings. This makes the language very powerful and expressive, but also means that newbies do not always get the kick in the shins that a regular type system would give them.
In this case you created a list structure inside listFromFile by reading a string from a file and then using split on it. From that point on it's a list and you should only perform list operations on it. If you wanted to take out all commas in your data you should either perform that operation on each item in the list, or else perform the operation inside listFromFile, before splitting the text.
String operations on lists will work, but sometimes the result will be garbled, so mixing them should be avoided. The other good point was that in this case string map is preferable to regsub, if nothing else it makes the code a bit clearer.
Documentation: for, foreach, lindex, llength, lsearch, lsort, proc, puts, regsub, set, split, string, while
(more of a comment than an answer, but I want the formatting)
One thing to be aware of: $f holds a list, then you use the string command regsub on it, then you treat the result of regsub as a list again.
Use list commands with list values. I'd replace the regsub command with
set f [lmap elem $f {string map {"," ""} $elem} ]
for Tcl version 8.5 or earlier, you could do this:
for {set i 0} {$i < [llength $f]} {incr i} {
lset f $i [string map {, ""} [lindex $f $i]]
}
Tcl syntax is very simple and consistant in the sense of its command / arguments structure. Sometimes I miss the dot notation of other languages like ruby. In ruby you can right something like this:
-199.abs # => 199
"ice is nice".length # => 11
"ruby is cool.".index("u") # => 1
"Nice Day Isn't It?".downcase.split("").uniq.sort.join # => " '?acdeinsty"
In Radical Language Modification and Let unknown know there are ideas of how to modify the language with the unknown command, e.g.:
proc know {cond body} {
proc unknown {args} [string map [list #c# $cond #b# $body] {
if {![catch {expr {#c#}} res] && $res} {
return [eval {#b#}]
}
}][info body unknown]
}
know {[regexp {^([a-z]+)\.([a-z]+)$} [lindex $args 0] -> from to]} {
set res {}
while {$from<=$to} {lappend res $from; incr from}
set res
}
# % puts [1..5]
# 1 2 3 4 5
How can I modify the previous code, so I can write commands with dot notation as in the Ruby example.
You can do it for specific operations, but not all, and there are some syntactic limitations. For example:
know {[regexp {^(.*)\.length$} [lindex $args 0] -> value]} {
string length $value
}
puts [abc.length]
# ---> 3
set thevar "abc def"
puts [$thevar.length]
# ---> 7
puts ["abc def".length]
# ---> extra characters after close-quote
That is, the value must still be syntactically-valid Tcl; that last example is not. You can chain the know handlers by using [$value] instead of plain $value in the handler, provided you've got a handler for the base case.
know {[regexp {^(.*)\.length$} [lindex $args 0] -> value]} {
string length [$value]
}
know {[regexp {^(.*)\.repeat\((\d+)\)$} [lindex $args 0] -> value count]} {
string repeat [$value] $count
}
# Base case for simple words
know {[regexp {^'(.*)'$} [lindex $args 0] -> value]} {
set value
}
puts ['abc\ def'.repeat(5).length]
# ---> 35
Ultimately, while you can do all sorts of stuff like this, it's not how Tcl is designed to work. It is going to be slow (the unknown calling mechanism is not an optimised path) and you're going to hit limitations. Better to learn to do things the normal way:
puts [string length [string repeat "abc def" 5]]