I am new to tcl and I need to read a binary file and compare first and last byte of data but I discovered that binary scan is not working for me (even the examples)
Here is the snippet
set fp [open $temp_file_name]
defer { close $fp }
fconfigure $fp -translation binary
# ...
seek $fp 0
set m [binary scan [read $fp 1] h* v]
puts $v
# can't read "v": no such variable
# m is 0
With the example
> puts [binary scan abcdefg s3s first second]
> puts $first
## can't read "first": no such variable
> puts $second
## can't read "second": no such variable
What am I doing wrong?
The result of binary scan is the number of format groups that have been satisfied. If we consider the case:
binary scan abcdefg s3s first second
The result of this is 1, because there are enough bytes to only satisfy the first format group, s3. That consumes six bytes (three little-endian 16-bit quantities, using up abcdef). There's insufficient data to satisfy the following s; there'd need to be another byte for that.
% info patchlevel
8.6.10
% puts [binary scan abcdefg s3s first second]
1
% puts $first
25185 25699 26213
% puts $second
can't read "second": no such variable
With the other code:
set m [binary scan [read $fp 1] h* v]
This will read a byte from the stream, convert into little-endian hex digits (I don't like the h conversion very much; H seems more sensible to me, especially for single bytes) and stores all those hex characters in the variable v. It should return 1 if anything is read from the stream at all.
I've no idea at all why binary scan is not working for you. It does not appear to be functioning in accordance with documented behaviour at all. To double-check, you haven't accidentally replaced the ::binary or ::tcl::binary::scan (which is the default implementation of binary scan) commands, have you?
Since things are behaving massively out of spec, it's hard to suggest how to fix that.
Does anyone know of a way to select a specific value in a list that consists of floating values (i.e. an equivalent method to Lindex used for integers in a list)?
Tcl's lindex command can work on any arbitrary list, but the indices themselves have to be either integers or end-relative (e.g., end-1). The values in the list can most definitely be floating point numbers (or any other value, including strings and lists and variable names and snippets of code and database handles and …).
set theList [list 1.23 2.34 3.45 [expr {4.56 + 5.67}]]
puts [lindex $theList 3]
The indices have to be integers because they are logically counting positions from the start of the list (or from the end of the list for end-relative, of course). It makes no sense at all to count positions using floating point numbers.
If you're trying to find where a floating point number would belong in a sortd list of floating point numbers, the lsearch command is the right tool (with the options below).
set idx [lsearch -sorted -real -bisect $theList 6.78]
# Now $idx is the index where the value is *or* the index before where it would be inserted
# In particular, $idx+1 indicates the first element later than the value
The options above are:
-sorted — Tells the lsearch command that the list is sorted (so it can use a binary search algorithm instead of a linear one)
-real — Tells the lsearch command that it is using floating point comparisons
-bisect — Tells the lsearch command to find the slot for the value (and not return -1 if it isn't already in there)
I would like to divied a bigger ::math::linearalgebra::mkMatrix to multiple smaller ones and store them in a list or any other container. Unfortunatly I did not manage to find a way to do this with lists. Is it possible to store multiple ::math::linearalgebra::mkMatrices in a list, array or even in a dictionary.
I tried to store the names in a list and then whenever I needed the mkMtarix I used [lindex matrices 0]. However this didn't work.
Is there any good material around about this?
Most of the commands in the math::linearalgebra expect to take the name of a variable holding the matrix. That means that while their value can go nicely in a list for storage, you can't really manipulate them like that. You're much better off using a Tcl array for what you're doing. Then you can go:
# An all-zero 3x3 matrix
set collection(0) [math::linearalgebra::mkMatrix 3 3 0.0]
# Turn it into an identity matrix; notice the use of a variable to name the element
set matrixID 0
for {set i 0} {$i < 3} {incr i} {
math::linearalgebra::setelem collection($matrixID) $i $i 1.0
}
The only things to be careful of are that elements of a Tcl array are not ordered, but in compensation you can use non-trivial keys into the array as well as simple integers. This means you can use composite keys like 1,2 (or fred,wilma), which you can generate like $x,$y, i.e., as in:
set matrixX 1
set matrixY 2
for {set i 0} {$i < 3} {incr i} {
math::linearalgebra::setelem collection($matrixX,$matrixY) $i $i 1.0
}
A powerful technique that you might find very useful.
I know that you can "hack" nesting associative arrays in tcl, and I also know that with dictionaries (which I have no experience with) you can nest them pretty easily. I'm trying to find a way to store the values of a function that has two variables, and then at the end I just want to print out a table of the two variables (column and row headers) with the function values in the cells. I can make this work, but it is neither succinct nor efficient.
Here's what should be printed. The rows are values of b and columns are values of a (1,2,3,4,5 for simplicity):
b
1 2 3 4 5
1 y(1,1) y(1,2) y(1,3) y(1,4) y(1,5)
2 y(2,1) y(2,2) y(2,3) y(2,4) y(2,5)
a 3 y(3,1) y(3,2) y(3,3) y(3,4) y(3,5)
4 y(4,1) y(4,2) y(4,3) y(4,4) y(4,5)
5 y(5,1) y(5,2) y(5,3) y(5,4) y(5,5)
To store this, I imagine I would simply do two nested for loops over a and b and somehow store the results in nested dictionaries. Like have one dictionary with 5 entries, 1 for each value of b, and each entry in this is another dictionary for each value of b.
To print it, the only way I can think of is to just explicitly print out each table line and call each dictionary entry. I'm not too versed in output formatting with tcl, but I can probably manage there.
Can anyone think of a more elegant way to do this?
Here are a couple of examples on how you might use the struct::matrix package.
Example 1 - Simple Create/Display
package require struct::matrix
package require Tclx
# Create a 3x4 matrix
set m [::struct::matrix]
$m add rows 3
$m add columns 4
# Populate data
$m set rect 0 0 {
{1 2 3 4}
{5 6 7 8}
{9 10 11 12}
}
# Display it
puts "Print matrix, cell by cell:"
loop y 0 [$m rows] {
loop x 0 [$m columns] {
puts -nonewline [format "%4d" [$m get cell $x $y]]
}
puts ""
}
Output
Print matrix, cell by cell:
1 2 3 4
5 6 7 8
9 10 11 12
Discussion
In the first part of the script, I created a matrix, add 3 rows and 4 columns--a straight forward process.
Next, I called set rect to populate the matrix with data. Depend on your need, you might want to look into set cell, set column, or set row. For more information, please consult the reference for struct::matrix.
When it comes to displaying the matrix, instead of using the Tcl's for command, I prefer the loop command from the Tclx package, which is simpler to read and use.
Example 2 - Read from a CSV file
package require csv
package require struct::matrix
package require Tclx
# Read a matrix from a CSV file
set m [::struct::matrix]
set fileHandle [open data.csv]
::csv::read2matrix $fileHandle $m "," auto
close $fileHandle
# Displays the matrix
loop y 0 [$m rows] {
loop x 0 [$m columns] {
puts -nonewline [format "%4d" [$m get cell $x $y]]
}
puts ""
}
The data file, data.csv:
1,2,3,4
5,6,7,8
9,10,11,12
Output
1 2 3 4
5 6 7 8
9 10 11 12
Discussion
The csv package provides a simple way to read from a CSV file to a matrix.
The heart of the operation is in the ::csv::read2matrix command, but before that, I have to create an empty matrix and open the file for reading.
The code to display the matrix is the same as previous example.
Conclusion
While the struct::matrix package seems complicated at first; I only need to learn a couple of commands to get started.
Elegance is in the eye of the beholder :)
With basic core Tcl, I think you understand your options reasonably well. Either arrays or nested dictionaries have clunky edges when it comes to tabular oriented data.
If you are willing to explore extensions (and Tcl is all about the extension) then you might consider the matrix package from the standard Tcl library. It deals with rows and columns as key concepts. If you need to do transformations on tabular data then I would suggest TclRAL, a relational algebra library that defines a Relation data type and will handle all types of tabular data and provide a large number of operations on it. Alternatively, you could try something like SQLite which will also handle tabular data, provide for manipulating it and has robust persistent storage. The Tcl wiki will direct you to details of all of these extensions.
However, if these seem too heavyweight for your taste or if you don't want to suffer the learning curve, rolling up your sleeves and banging out an array or nested dictionary solution, while certainly being rather ad hoc, is probably not that difficult. Elegant? Well, that's for you to judge.
Nested lists work reasonably well for tabular data from 8.4 onwards (with multi-index lindex and lset) provided you've got compact numeric indices. 8.5's lrepeat is good for constructing an initial matrix too.
set mat [lrepeat 5 [lrepeat 5 0.0]]
lset mat 2 3 1.3
proc printMatrix {mat} {
set height [llength $mat]
set width [llength [lindex $mat 0]]
for {set j 0} {$j < $width} {incr j} {
puts -nonewline \t$j
}
puts ""
for {set i 0} {$i < $height} {incr i} {
puts -nonewline $i
for {set j 0} {$j < $width} {incr j} {
puts -nonewline \t[lindex $mat $i $j]
}
puts ""
}
}
printMatrix $mat
You should definitely consider using the struct::matrix and report packages from tcllib.
package require csv
package require struct::matrix
array set OPTS [subst {
csv_input_filename {input.csv}
}]
::struct::matrix indata
set chan [open $OPTS(csv_input_filename)]
csv::read2matrix $chan indata , auto
close $chan
# prints matrix as list format
puts [join [indata get rect 0 0 end end] \n];
# prints matrix as csv format
csv::joinmatrix indata
# cleanup
indata destroy
This is a one-liner way to print out a matrix as list or csv format respectively.
I have two lists that contain some data (numeric data or/and strings)?
How do I join these two lists, assuming that the lists do not contain sublists?
Which choice is preferred and why?
set first [concat $first $second]
lappend first $second
append first " $second"
It is fine to use concat and that is even highly efficient in some cases (it is the recommended technique in 8.4 and before, and not too bad in later versions). However, your second option with lappend will not work at all, and the version with append will work, but will also be horribly inefficient.
Other versions that will work:
# Strongly recommended from 8.6.1 on
set first [list {*}$first {*}$second]
lappend first {*}$second
The reason why the first of those is recommended from 8.6.1 onwards is that the compiler is able to optimise it to a direct "list-concatenate" operation.
Examples
% set first {a b c}
a b c
% set second {1 2 3}
1 2 3
% set first [concat $first $second]; # #1 is correct
a b c 1 2 3
% set first {a b c}
a b c
% lappend first $second; # #2 is wrong: appends the whole `second` list to `first
a b c {1 2 3}
Discussion
I looked up the documentation, also experiment with some lists and found out that:
Your first choice, concat is correct
lappend does not work because it treats $second as one element, not a list
append works, but you are treating your lists as string. I don't know what the implications are, but it does not communicate the intention that first and second are lists.
Maybe a bit old, but wanted to clarify;
As already stated, the standard way to merge 2 lists is via concat pre-v8.6. However please note that concat gets very inefficient when dealing with long lists, since it analyzes the lists as part of the merge. eg when merging lists, the larger they get the slower they merge.
Both appends do not merge "lists", they just add to an existing list (lappend) or variable (append). Both appends have no impact to speed, since they do not analyze anything when appending.
If merging single entry list elements, one could merge them via set first [join [lappend first $second]] but only if dealing with simple/single elements within each list (ie no spaces per element).
To add to the other answers, I ran a rough benchmark comparing the different versions (tclsh 8.6.13).
#! /usr/bin/env tclsh
set a {1}
for {set i 0} {$i < 25} {incr i} {
switch $argv {
list {
set a [list {*}$a {*}$a]
}
concat {
set a [concat $a $a]
}
lappend {
lappend a {*}$a
}
append {
append a " $a"
}
}
}
Results:
./test.tcl lappend 0.28s user 0.51s system 99% cpu 0.795 total
./test.tcl list 0.22s user 0.29s system 99% cpu 0.511 total
./test.tcl append 0.04s user 0.08s system 99% cpu 0.115 total
./test.tcl concat 0.04s user 0.08s system 99% cpu 0.112 total
Note that the semantics aren't quite the same between the different versions. For example, list will re-quote list elements.