I know that you can "hack" nesting associative arrays in tcl, and I also know that with dictionaries (which I have no experience with) you can nest them pretty easily. I'm trying to find a way to store the values of a function that has two variables, and then at the end I just want to print out a table of the two variables (column and row headers) with the function values in the cells. I can make this work, but it is neither succinct nor efficient.
Here's what should be printed. The rows are values of b and columns are values of a (1,2,3,4,5 for simplicity):
b
1 2 3 4 5
1 y(1,1) y(1,2) y(1,3) y(1,4) y(1,5)
2 y(2,1) y(2,2) y(2,3) y(2,4) y(2,5)
a 3 y(3,1) y(3,2) y(3,3) y(3,4) y(3,5)
4 y(4,1) y(4,2) y(4,3) y(4,4) y(4,5)
5 y(5,1) y(5,2) y(5,3) y(5,4) y(5,5)
To store this, I imagine I would simply do two nested for loops over a and b and somehow store the results in nested dictionaries. Like have one dictionary with 5 entries, 1 for each value of b, and each entry in this is another dictionary for each value of b.
To print it, the only way I can think of is to just explicitly print out each table line and call each dictionary entry. I'm not too versed in output formatting with tcl, but I can probably manage there.
Can anyone think of a more elegant way to do this?
Here are a couple of examples on how you might use the struct::matrix package.
Example 1 - Simple Create/Display
package require struct::matrix
package require Tclx
# Create a 3x4 matrix
set m [::struct::matrix]
$m add rows 3
$m add columns 4
# Populate data
$m set rect 0 0 {
{1 2 3 4}
{5 6 7 8}
{9 10 11 12}
}
# Display it
puts "Print matrix, cell by cell:"
loop y 0 [$m rows] {
loop x 0 [$m columns] {
puts -nonewline [format "%4d" [$m get cell $x $y]]
}
puts ""
}
Output
Print matrix, cell by cell:
1 2 3 4
5 6 7 8
9 10 11 12
Discussion
In the first part of the script, I created a matrix, add 3 rows and 4 columns--a straight forward process.
Next, I called set rect to populate the matrix with data. Depend on your need, you might want to look into set cell, set column, or set row. For more information, please consult the reference for struct::matrix.
When it comes to displaying the matrix, instead of using the Tcl's for command, I prefer the loop command from the Tclx package, which is simpler to read and use.
Example 2 - Read from a CSV file
package require csv
package require struct::matrix
package require Tclx
# Read a matrix from a CSV file
set m [::struct::matrix]
set fileHandle [open data.csv]
::csv::read2matrix $fileHandle $m "," auto
close $fileHandle
# Displays the matrix
loop y 0 [$m rows] {
loop x 0 [$m columns] {
puts -nonewline [format "%4d" [$m get cell $x $y]]
}
puts ""
}
The data file, data.csv:
1,2,3,4
5,6,7,8
9,10,11,12
Output
1 2 3 4
5 6 7 8
9 10 11 12
Discussion
The csv package provides a simple way to read from a CSV file to a matrix.
The heart of the operation is in the ::csv::read2matrix command, but before that, I have to create an empty matrix and open the file for reading.
The code to display the matrix is the same as previous example.
Conclusion
While the struct::matrix package seems complicated at first; I only need to learn a couple of commands to get started.
Elegance is in the eye of the beholder :)
With basic core Tcl, I think you understand your options reasonably well. Either arrays or nested dictionaries have clunky edges when it comes to tabular oriented data.
If you are willing to explore extensions (and Tcl is all about the extension) then you might consider the matrix package from the standard Tcl library. It deals with rows and columns as key concepts. If you need to do transformations on tabular data then I would suggest TclRAL, a relational algebra library that defines a Relation data type and will handle all types of tabular data and provide a large number of operations on it. Alternatively, you could try something like SQLite which will also handle tabular data, provide for manipulating it and has robust persistent storage. The Tcl wiki will direct you to details of all of these extensions.
However, if these seem too heavyweight for your taste or if you don't want to suffer the learning curve, rolling up your sleeves and banging out an array or nested dictionary solution, while certainly being rather ad hoc, is probably not that difficult. Elegant? Well, that's for you to judge.
Nested lists work reasonably well for tabular data from 8.4 onwards (with multi-index lindex and lset) provided you've got compact numeric indices. 8.5's lrepeat is good for constructing an initial matrix too.
set mat [lrepeat 5 [lrepeat 5 0.0]]
lset mat 2 3 1.3
proc printMatrix {mat} {
set height [llength $mat]
set width [llength [lindex $mat 0]]
for {set j 0} {$j < $width} {incr j} {
puts -nonewline \t$j
}
puts ""
for {set i 0} {$i < $height} {incr i} {
puts -nonewline $i
for {set j 0} {$j < $width} {incr j} {
puts -nonewline \t[lindex $mat $i $j]
}
puts ""
}
}
printMatrix $mat
You should definitely consider using the struct::matrix and report packages from tcllib.
package require csv
package require struct::matrix
array set OPTS [subst {
csv_input_filename {input.csv}
}]
::struct::matrix indata
set chan [open $OPTS(csv_input_filename)]
csv::read2matrix $chan indata , auto
close $chan
# prints matrix as list format
puts [join [indata get rect 0 0 end end] \n];
# prints matrix as csv format
csv::joinmatrix indata
# cleanup
indata destroy
This is a one-liner way to print out a matrix as list or csv format respectively.
Related
I am new to tcl and I need to read a binary file and compare first and last byte of data but I discovered that binary scan is not working for me (even the examples)
Here is the snippet
set fp [open $temp_file_name]
defer { close $fp }
fconfigure $fp -translation binary
# ...
seek $fp 0
set m [binary scan [read $fp 1] h* v]
puts $v
# can't read "v": no such variable
# m is 0
With the example
> puts [binary scan abcdefg s3s first second]
> puts $first
## can't read "first": no such variable
> puts $second
## can't read "second": no such variable
What am I doing wrong?
The result of binary scan is the number of format groups that have been satisfied. If we consider the case:
binary scan abcdefg s3s first second
The result of this is 1, because there are enough bytes to only satisfy the first format group, s3. That consumes six bytes (three little-endian 16-bit quantities, using up abcdef). There's insufficient data to satisfy the following s; there'd need to be another byte for that.
% info patchlevel
8.6.10
% puts [binary scan abcdefg s3s first second]
1
% puts $first
25185 25699 26213
% puts $second
can't read "second": no such variable
With the other code:
set m [binary scan [read $fp 1] h* v]
This will read a byte from the stream, convert into little-endian hex digits (I don't like the h conversion very much; H seems more sensible to me, especially for single bytes) and stores all those hex characters in the variable v. It should return 1 if anything is read from the stream at all.
I've no idea at all why binary scan is not working for you. It does not appear to be functioning in accordance with documented behaviour at all. To double-check, you haven't accidentally replaced the ::binary or ::tcl::binary::scan (which is the default implementation of binary scan) commands, have you?
Since things are behaving massively out of spec, it's hard to suggest how to fix that.
I have used some tcl code in a design tool to get the list of standard cells in a single list.
Tcl has a limitation of processing a large number of elements to read from any list. How do I split these list of standard cells in a single data structure for the tool to read?
If you've got a big list that you've got to split into small chunks for processing and don't want to just do all the pieces one by one with foreach, you can do this:
set big_list {lots and lots and lots...}
set index 0
set stride 10
while true {
set chunk [lrange $big_list $index [expr {$index + $stride - 1}]]
# Nothing left; got to the end
if {![llength $chunk]} break
incr index $stride
process_chunk $chunk
}
Tune the stride size for how much you can feed through. (Theoretically, you can do auto-tuning of the stride length if there's some complex limit involved. Practically, just pick something by hand that works and isn't too inefficient; auto-tuning algorithms are always quite a lot more complicated.)
I would like to divied a bigger ::math::linearalgebra::mkMatrix to multiple smaller ones and store them in a list or any other container. Unfortunatly I did not manage to find a way to do this with lists. Is it possible to store multiple ::math::linearalgebra::mkMatrices in a list, array or even in a dictionary.
I tried to store the names in a list and then whenever I needed the mkMtarix I used [lindex matrices 0]. However this didn't work.
Is there any good material around about this?
Most of the commands in the math::linearalgebra expect to take the name of a variable holding the matrix. That means that while their value can go nicely in a list for storage, you can't really manipulate them like that. You're much better off using a Tcl array for what you're doing. Then you can go:
# An all-zero 3x3 matrix
set collection(0) [math::linearalgebra::mkMatrix 3 3 0.0]
# Turn it into an identity matrix; notice the use of a variable to name the element
set matrixID 0
for {set i 0} {$i < 3} {incr i} {
math::linearalgebra::setelem collection($matrixID) $i $i 1.0
}
The only things to be careful of are that elements of a Tcl array are not ordered, but in compensation you can use non-trivial keys into the array as well as simple integers. This means you can use composite keys like 1,2 (or fred,wilma), which you can generate like $x,$y, i.e., as in:
set matrixX 1
set matrixY 2
for {set i 0} {$i < 3} {incr i} {
math::linearalgebra::setelem collection($matrixX,$matrixY) $i $i 1.0
}
A powerful technique that you might find very useful.
I have two lists that contain some data (numeric data or/and strings)?
How do I join these two lists, assuming that the lists do not contain sublists?
Which choice is preferred and why?
set first [concat $first $second]
lappend first $second
append first " $second"
It is fine to use concat and that is even highly efficient in some cases (it is the recommended technique in 8.4 and before, and not too bad in later versions). However, your second option with lappend will not work at all, and the version with append will work, but will also be horribly inefficient.
Other versions that will work:
# Strongly recommended from 8.6.1 on
set first [list {*}$first {*}$second]
lappend first {*}$second
The reason why the first of those is recommended from 8.6.1 onwards is that the compiler is able to optimise it to a direct "list-concatenate" operation.
Examples
% set first {a b c}
a b c
% set second {1 2 3}
1 2 3
% set first [concat $first $second]; # #1 is correct
a b c 1 2 3
% set first {a b c}
a b c
% lappend first $second; # #2 is wrong: appends the whole `second` list to `first
a b c {1 2 3}
Discussion
I looked up the documentation, also experiment with some lists and found out that:
Your first choice, concat is correct
lappend does not work because it treats $second as one element, not a list
append works, but you are treating your lists as string. I don't know what the implications are, but it does not communicate the intention that first and second are lists.
Maybe a bit old, but wanted to clarify;
As already stated, the standard way to merge 2 lists is via concat pre-v8.6. However please note that concat gets very inefficient when dealing with long lists, since it analyzes the lists as part of the merge. eg when merging lists, the larger they get the slower they merge.
Both appends do not merge "lists", they just add to an existing list (lappend) or variable (append). Both appends have no impact to speed, since they do not analyze anything when appending.
If merging single entry list elements, one could merge them via set first [join [lappend first $second]] but only if dealing with simple/single elements within each list (ie no spaces per element).
To add to the other answers, I ran a rough benchmark comparing the different versions (tclsh 8.6.13).
#! /usr/bin/env tclsh
set a {1}
for {set i 0} {$i < 25} {incr i} {
switch $argv {
list {
set a [list {*}$a {*}$a]
}
concat {
set a [concat $a $a]
}
lappend {
lappend a {*}$a
}
append {
append a " $a"
}
}
}
Results:
./test.tcl lappend 0.28s user 0.51s system 99% cpu 0.795 total
./test.tcl list 0.22s user 0.29s system 99% cpu 0.511 total
./test.tcl append 0.04s user 0.08s system 99% cpu 0.115 total
./test.tcl concat 0.04s user 0.08s system 99% cpu 0.112 total
Note that the semantics aren't quite the same between the different versions. For example, list will re-quote list elements.
I am looking for a clean way to list the (8 bit) integers whose binary representation is not the same as another integer up to rotation and reflection.
For example the list will probably start as
0
1
(2=10b is skipped because you can rotate the bits in 1, therefore all powers of 2 are skipped. Also every number except 0 will be odd)
3=11b
5=101b
7=111b
9=1001b
11=1011b (so 13=1101b will be skipped because 11010000b is a reflection of 1101b which can then be rotated to the right 4 times )
.
.
.
Also ideally how could this be generalized to numbers with different numbers of bits, (16, 32, or just n) and other bases beside 2.
Since #John Smith thought my comment was a good answer, here it is an answer.
The answers here may be illuminating.
Thanks to Jeffromi for explaining the problem better -- I've deleted my previous answer.
Here's another solution in Perl. Perl is a good language for this sort of problem because it makes it easy to treat numbers as text and text as numbers.
i: for $i (0..255) {
$n1 = sprintf "%08b", $i; # binary representation of $i
$n2 = $n1; # "unreflected" copy of $n1
$n3 = reverse $n1; # "reflection" of $n1
for $j (1..8) {
$n2 = chop($n2) . $n2; # "rotate" $n2
$n3 = chop($n3) . $n3; # "rotate" $n3
next i if $found{$n2} or $found{$n3};
}
# if we get here, we rotated $n2 and $n3 8 times
# and didn't get a nonsymmetric byte that we've
# seen before -- this is a nonsymmetric byte
$found{$n1}++;
print "$i $n1\n";
}
This isn't as simple as the previous solution, but the jist is to try out all 16 combinations (2 reflections x 8 rotations) and compare them with all of the nonsymmetric bytes you've seen before.
There's a operator for bit shifting with rotation in Perl, but the chop($num) . $num idiom I used generalizes better to problems with base n.
You can use a sieve, similar to the sieve of Eratosthenes for prime numbers.
Use a bit array (BitSet in Java) with one bit for each number.
Initially mark all bits.
Go sequentially through the bit array until you find the next bit that is set at index n, this is a number in your set. Then clear the bits of all other numbers that can be reached from n via rotation and mirroring.
On today's machines this is feasible up to 32 bits, which would use 512MB of memory.
An alternative solution to Eratosthenes' Sieve would be to construct a test T(k) that returns True or False for any given k.
It would be slower, but this way no storage would be needed, so it would extend more readily to arbitrary data length.
If you simplify the problem for a moment, and say we are simply looking to discard reflections, then it would be easy:
T_ref(k) returns true iff k <= Reflection(k).
As for rotating bits, exactly the same can be done:
T_rot(k) returns true iff k == MIN{the set of all rotations of k}
You can think of dividing your integers up into a bunch of equivalence classes E(k) where E(k) is the set of all reflection&rotation permutations of k.
You might want to take a moment to satisfy yourself that the set of natural numbers N partitions itself readily into such disjoint subsets.
Then the set
{k s.t. k == MIN(E(k)) }
will guarantee to contain exactly one element from each equivalence class.
This would make a really nice interview question.