I am new learner of tcl scripting language. I am using TCL version 8.5. I read text file through tcl script and count similar words frequency. I used for loop and dictionary to count similar words and their frequency but output of the program print like this: alpha 4 beta 2 gamma 1 delta 1
But I want to print it in one column each key, value pair of dictionary or we could say each key, value pair print line by line in output. Following is my script in tcl and its output at the end.
set f [open input.txt]
set text [read $f]
foreach word [split $text] {
dict incr words $word
}
puts $words
Output of the above script:
alpha 4 beta 2 gamma 1 delta 1
You would do:
dict for {key value} $words {
puts "$key $value"
}
When reading the dict documentation, take care about which subcommands require a dictionaryVariable (like dict incr) and which require a dictionaryValue (like dict for)
For nice formatting, as suggested by Donal, here's a very terse method:
set maxWid [tcl::mathfunc::max {*}[lmap w [dict keys $words] {string length $w}]]
dict for {word count} $words {puts [format "%-*s = %s" $maxWid $word $count]}
Or, look at the source code for the parray command for further inspiration:
parray tcl_platform ;# to load the proc
info body parray
Related
I am new to tcl scripting.
I have 2 columns in my CSV file.
Example: My data
A 12
D 18
B 33
A 11
D 49
I would like to extract column 2 values using Column 1 data.
Required output:
A: 12,11
B: 33
D: 18, 49
Can someone please help me to write a tcl script to perform this task?
Thank you in advance.
This is a very well known problem, and Tcl's dict lappend is exactly the tool you need to solve it (together with csv::split from tcllib to do the parsing of your input data).
package require csv
# Read split and collect
set data {}
while {[gets stdin line] >= 0} {
lassign [csv::split $line] key value; # You might need to configure this
dict lappend data $key $value
}
# Write out
puts ""; # Blank header line
dict for {key values} $data {
puts [format "%s: %s" $key [join $values ", "]]
}
The above is written as a stdin-to-stdout filter. Adapting to work with files is left as an exercise.
I have a csv file which has hostname and attached serial numbers. I want to create a key value pair with key being hostname and value being the list of serial numbers. The serial numbers can be one or many.
For example:
A, 1, 2, 3, 4
B, 5, 6
C, 7, 8, 9
D, 10
I need to access key A and get {1 2 3 4} as output. And if I access D i should get {10}
How should I do this? As the version of TCL i am using doesn't support any packages like CSV and I also won't be able to install it as it is in the server, So I am looking at a solution which doesn't include any packages.
For now, I am splitting the line with \n and then I process each element. Then I split the elements with "," and then I get the host name and serial numbers in a list. I then use the 0th index of the list as hostname and remaining values as serial numbers. Is there a cleaner solution?
I'd do something like:
#!/usr/bin/env tclsh
package require csv
package require struct::queue
set filename "file.csv"
set fh [open $filename r]
set q [struct::queue]
csv::read2queue $fh $q
close $fh
set data [dict create]
while {[$q size] > 0} {
set values [lassign [$q get] hostname]
dict set data $hostname [lmap elem $values {string trimleft $elem}]
}
dict for {key value} $data {
puts "$key => $value"
}
then
$ tclsh csv.tcl
A => 1 2 3 4
B => 5 6
C => 7 8 9
D => 10
The repeated recommendation given here is to use the CSV package for this purpose. See also the answer by #glenn-jackman. If unavailable, the time is better invested in obtaining it at your server side.
To get you started, however, you might want to adopt something along the lines of:
set dat {
A, 1, 2, 3, 4
B, 5, 6
C, 7, 8, 9
D, 10
}
set d [dict create]
foreach row [split [string trim $dat] \n] {
set row [lassign [split $row ,] key]
dict set d [string trim $key] [concat {*}$row]
}
dict get $d A
dict get $d D
Be warned, however, such hand-knitted solutions typically only serve their purpose when you have full control of the data being processed and its representation. Again, time is better invested by obtaining the CSV package.
I tried this way and got it working. Thanks again for your inputs. Yes, I know csv package would be easy but I cannot install it in server/product.
set multihost "host_slno.csv"
set fh1 [open $multihost r]
set data [read -nonewline $fh1]
close $fh1
set hostslnodata [ split $data "\n" ]
set hostslno [dict create];
foreach line $hostslnodata {
set line1 [join [split $line ", "] ]
puts "$line1"
if {[regexp {([A-Za-z0-9_\-]+)\s+(.*)} $line1 match hostname serial_numbers]} {
dict lappend hostslno $hostname $serial_numbers
}
}
puts [dict get $hostslno]
The sourcecode from the csv package is available. If you are unable to install the full csv package, you can include the code from here:
http://core.tcl.tk/tcllib/artifact/2898cd911697ecdb
If you still can't use that option, then stripping out all the whitespace and splitting on "," is required.
An alternative to the earlier answers is using string map:
set row [split [string map {" " ""} $row ] ,]
The string map will remove all spaces, and then split on ","
Once you have converted the lines of text into valid tcl lists:
A 1 2 3 4
B 5 6
C 7 8 9
D 10
Then you can use the lindex and lrange commands to pluck off all the pieces.
foreach row $data {
set server [lindex $row 0]
set serial_numbers [lrange $row 1 end]
dict set ...
One possibility:
set hostslno [dict create]
set multihost "host_slno.csv"
set fh1 [open $multihost]
while {[gets $fh line] >= 0} {
set numbers [lassign [regexp -inline -all {[^\s,]+} $line] hostname]
dict set hostslno $hostname $numbers
}
close $fh1
puts [dict get $hostslno A]
I have a file having following data
Anny : dancing
Sonny : reciting
Joel : dancing
Anny : reciting
Anny : singing
I want the following o/p in tcl:
Anny -
singing 1
dancing 1
reciting 1
Joel -
dancing 1
I want to print in this format along with their count.
Working with Donal's answer, but using a single dictionary instead of an array of dictionaries:
set data [dict create]
set f [open yourinputfile.txt r]
while {[gets $f line] != -1} {
if {[scan $line "%s : %s" who what] == 2} {
dict update data $who activities {
dict incr activities $what
}
}
}
close $f
dict for {who activities} $data {
puts "$who -"
dict for {what count} $activities {
puts "$what $count"
}
puts ""
}
This is really about counting words, so we're going to be dealing with dictionaries — dict incr is a perfect tool for this — but you also need to do a bit of parsing. Parsing is done in many ways, but in this case scan can do what we want easily. (Remember when reading my code that the result of scan is the number of fields that it managed to satisfy.)
set f [open "yourinputfile.txt"]
set data [split [read $f] "\n"]
close $f
# Peel apart that data
foreach line $data {
if {[scan $line "%s : %s" who what] == 2} {
dict incr activity($who) $what
}
}
# Now produce the output
foreach who [lsort [array names activity]] {
puts "$who -"
dict for {what count} $activity($who) {
puts "$what $count"
}
# And the extra blank line
puts ""
}
You could use an array to store the info while you're collecting in.
The regexp you're using is wrong.
Use a list of list to collect the matching in a pair way (i.e. word #n), and then assign all the collected matching to the proper key on the array.
Here is an example on how to do it:
set file_content {Anny : dancing
Sonny : reciting
Joel : dancing
Anny : reciting
Anny : singing
}
array set res {}
set anny {}
lappend anny [list dancing [regexp -all {Anny\s*:\s*dancing} $file_content] ]
lappend anny [list singing [regexp -all {Anny\s*:\s*singing} $file_content] ]
lappend anny [list reciting [regexp -all {Anny\s*:\s*reciting} $file_content] ]
set res(Anny) $anny
puts [array get res]
If I run this the output is:
Anny {{dancing 1} {singing 1} {reciting 1}}
Now you could use the array to format the output as you wish.
Of course you should do the same with other names, so the best is to put the code inside a function.
This is one way to do it.
Count the number of different lines. Get rid of the colon.
foreach line [split $data \n] {
dict incr d0 [string map {: {}} $line]
}
Convert the dictionary of lines and counts to a hierarchical dictionary with names on the highest level and activities on the next level. If line contains "Joel dancing", the invocation below will be, after expansion with {*}: dict set d1 Joel dancing 1, creating the dictionary item Joel {dancing 1}.
dict for {line count} $d0 {
dict set d1 {*}$line $count
}
Iterate over the dictionary and print the keys and values.
dict for {name activities} $d1 {
puts "$name -"
foreach {activity count} $activities {
puts "$activity $count"
}
puts {}
}
Documentation:
dict,
foreach,
puts,
split,
string,
{*} (syntax)
I am forced to use TCL for something and I need to create a json string like this:
{ "mainKey": "mainValue", "subKey": [{"key1":"value1"},{"key2":"value2"}]}
So I am trying to do this:
set subDict1 [dict create key1 value1]
set subDict2 [dict create key2 value2]
set subDictList [list $subDict1 $subDict2]
set finalDict [dict create mainKey mainValue subKey $subDictList]
When I convert this dict to json, I get:
{"mainKey":"mainValue", "subKey":{"key1 value1":{"key2":"value2"}}}
instead of the required:
{ "mainKey": "mainValue", "subKey": [{"key1":"value1"},{"key2":"value2"}]}
What am I doing wrong?
First you have to understand that TCL is a very typeless language. What exactly are list and dicts in tcl?
In Tcl a list is a string that is properly formatted where each member of the list is separated by spaces (space, tab or newline) and if the data contained by an item contains spaces they can be escaped either by:
using backslash escaping:
"this is a list\ of\ four\ items"
using "" grouping:
{this is a "list of four items"}
using {} grouping:
{this is a {list of four items}}
Note that internally, once a string has been parsed as a list, Tcl uses a different internal data structure to store the list for speed. But semantically it is still a string. Just like HTML is a specially formatted string or JSON is a specially formatted string Tcl takes the attitude that lists are nothing but specially formatted strings.
So, what are dicts? In Tcl dicts are lists with even number of elements. That's it. Nothing special. A dict is therefore also semantically a string (though as mentioned above, once tcl sees you using that string as a dict it will compile it to a different data structure for optimizing speed).
Note again the core philosophy in tcl: almost all data structures (with the exception of arrays) are merely strings that happens to be formatted in a way that has special meaning.
This is the reason you can't auto-convert tcl data structures to JSON - if you ask Tcl to guess what the data structure is you end up with whatever the programmer who wrote the guessing function want it to be. In your case it looks like it defaults to always detecting lists with even number of elements as dicts.
So how can you generate JSON correctly?
There are several ways to do this. You can of course use custom dedicated for loops or functions to convert your data structure (which again, is just a specially formatted string) to JSON.
Several years ago I've written this JSON compiler:
# data is plain old tcl values
# spec is defined as follows:
# {string} - data is simply a string, "quote" it if it's not a number
# {list} - data is a tcl list of strings, convert to JSON arrays
# {list list} - data is a tcl list of lists
# {list dict} - data is a tcl list of dicts
# {dict} - data is a tcl dict of strings
# {dict xx list} - data is a tcl dict where the value of key xx is a tcl list
# {dict * list} - data is a tcl dict of lists
# etc..
proc compile_json {spec data} {
while [llength $spec] {
set type [lindex $spec 0]
set spec [lrange $spec 1 end]
switch -- $type {
dict {
lappend spec * string
set json {}
foreach {key val} $data {
foreach {keymatch valtype} $spec {
if {[string match $keymatch $key]} {
lappend json [subst {"$key":[
compile_json $valtype $val]}]
break
}
}
}
return "{[join $json ,]}"
}
list {
if {![llength $spec]} {
set spec string
} else {
set spec [lindex $spec 0]
}
set json {}
foreach {val} $data {
lappend json [compile_json $spec $val]
}
return "\[[join $json ,]\]"
}
string {
if {[string is double -strict $data]} {
return $data
} else {
return "\"$data\""
}
}
default {error "Invalid type"}
}
}
}
(See http://wiki.tcl.tk/JSON for the original implementation and discussion of JSON parsing)
Because tcl can never correctly guess what your "string" is I've opted to supply a format string to the function in order to correctly interpret tcl data structures. For example, using the function above to compile your dict you'd call it like this:
compile_json {dict subKey list} finalDict
I've begged the tcllib maintainers to steal my code because I still believe it's the correct way to handle JSON in tcl but so far it's still not in tcllib.
BTW: I license the code above as public domain and you or anyone may claim full authorship of it if you wish.
It's not completely wrong to say that Tcl is a typeless language, because the types of the data objects in a Tcl program aren't expressed fully in the code, and not always even in the Tcl_Obj structures that represent data objects internally. Still, types are certainly not absent from a Tcl program, it's just that the type system is a lot less intrusive in Tcl than in most other programming languages.
The complete type definition in a Tcl program emerges from a dynamic combination of code and data objects as the program executes. The interpreter trusts you to tell it how you want your data objects to behave.
As an example, consider the following string:
set s {title: Mr. name: Peter surname: Lewerin}
Is this a string, an array, or a dictionary? All of the above, actually. (At least it's not an integer, a double or a boolean, other possible Tcl types.)
Using this string, I can answer a number of questions:
Tell me about your name
puts $s
# => title: Mr. name: Peter surname: Lewerin
What do polite people call you?
puts [dict values $s]
# => Mr. Peter Lewerin
What was your last name again?
puts [lindex $s end]
# => Lewerin
Here, I used the same string as a string, as a dictionary, and as an array. The same string representation was used for all three types of object, and it was the operations I used on it that determined the type of the object in that precise moment.
Similarly, the literal 1 can mean the integer 1, the single-character string 1, or boolean truth. There is no way to specify which kind of 1 you mean, but there is no need either, since the interpreter won't complain about the ambiguity.
Because Tcl doesn't store complete type information, it's quite hard to serialize arbitrary collections of data objects. That doesn't mean Tcl can't play well with serialization, though: you just need to add annotations to your data.
This string:
di [dm [st mainKey] [st mainValue]] [dm [st subKey] [ar [di [dm [st key1] [st value1]]] [di [dm [st key2] [st value2]]]]]
can be fed into the Tcl interpreter, and given the proper definitions of di, dm, st, and ar (which I intend to signify "dictionary", "dictionary member", "string", and "array", respectively), I can have the string construct a dictionary structure equivalent to the one in the question, or the string representation of such an object, just a bare list of keys and values, or XML, or JSON, etc. By using namespaces and/or slave interpreters, I can even dynamically switch between various forms. I won't provide examples for all forms, just JSON:
proc di args {return "{[join $args {, }]}"}
proc st val {return "\"$val\""}
proc ar args {return "\[[join $args {, }]]"}
proc dm {k v} {return "$k: $v"}
The output becomes:
{"mainKey": "mainValue", "subKey": [{"key1": "value1"}, {"key2": "value2"}]}
This example used the command nesting of the Tcl interpreter to define the structure of the data. Tcl doesn't need even that: a list of token classes and tokens such as a scanner would emit will suffice:
< : ' mainKey ' mainValue : ' subKey ( < : ' key1 ' value1 > < : ' key2 ' value2 > ) >
Using these simple commands:
proc jsonparseseq {endtok args} {
set seq [list]
while {[lsearch $args $endtok] > 0} {
lassign [jsonparseexpr {*}$args] args expr
lappend seq $expr
}
list [lassign $args -] $seq
}
proc jsonparseexpr args {
set args [lassign $args token]
switch -- $token {
' {
set args [lassign $args str]
set json \"$str\"
}
: {
lassign [jsonparseexpr {*}$args] args key
lassign [jsonparseexpr {*}$args] args val
set json "$key: $val"
}
< {
lassign [jsonparseseq > {*}$args] args dict
set json "{[join $dict {, }]}"
}
( {
lassign [jsonparseseq ) {*}$args] args arr
set json "\[[join $arr {, }]]"
}
}
list $args $json
}
proc jsonparse args {
lindex [jsonparseexpr {*}$args] end
}
I can parse that stream of token classes (<, (, ', :, ), >) and tokens into the same JSON string as above:
jsonparse < : ' mainKey ' mainValue : ' subKey ( < : ' key1 ' value1 > < : ' key2 ' value2 > ) >
# -> {"mainKey": "mainValue", "subKey": [{"key1": "value1"}, {"key2": "value2"}]}
Tcl offers quite a lot of flexibility; few languages will be as responsive to the programmer's whim as Tcl.
For completeness I will also demonstrate using the Tcllib huddle package mentioned by slebetman to create a the kind of structure mentioned in the question, and serialize that into JSON:
package require huddle
# -> 0.1.5
set subDict1 [huddle create key1 value1]
# -> HUDDLE {D {key1 {s value1}}}
set subDict2 [huddle create key2 value2]
# -> HUDDLE {D {key2 {s value2}}}
set subDictList [huddle list $subDict1 $subDict2]
# -> HUDDLE {L {{D {key1 {s value1}}} {D {key2 {s value2}}}}}
set finalDict [huddle create mainKey mainValue subKey $subDictList]
# -> HUDDLE {D {mainKey {s mainValue} subKey {L {{D {key1 {s value1}}} {D {key2 {s value2}}}}}}}
huddle jsondump $finalDict {} {}
# -> {"mainKey":"mainValue","subKey":[{"key1":"value1"},{"key2":"value2"}]}
Another approach is to create regular Tcl structures and convert ("compile") them to huddle data according to a type specification:
set subDict1 [dict create key1 value1]
set subDict2 [dict create key2 value2]
set subDictList [list $subDict1 $subDict2]
set finalDict [dict create mainKey mainValue subKey $subDictList]
huddle compile {dict mainKey string subKey {list {dict * string}}} $finalDict
The result of the last command is the same as of the last huddle create command in the previous example.
Documentation: dict, join, lappend, lassign, lindex, list, lsearch, proc, puts, return, set, switch, while
file1.txt
dut1Loop1Net = [::ip::contract [::ip::prefix 1.1.1.1/24]]/24
My script is
set in [open file1.txt r]
set line [gets $in]
if {[string trim [string range $line1 0 0]] != "#"} {
set devicePort [string trim [lindex $line1 0]]
set mark [expr [string first "=" $line1] + 1]
set val [string trim [string range $line1 $mark end]]
global [set t $devicePort]
set [set t $devicePort] $val
}
close $in
Problem
I am getting output as
% set dut1Loop1Net
[::ip::contract [::ip::prefix 1.1.1.1/24]]/24
Here i am getting the string without evaluating.
I am expecting the output as 1.1.1.0/24. Because TCL does not evaluate code here, it is printing like a string.
I am interesting to know how TCL stores the data and in which form it will retreive the data.
How Tcl stores values.
The short story:
Everything is a string
The long strory
Tcl stores the data in the last used datatype, calculate the string representation only when nessecary, uses copy on write, a simple refcount memory managment.
The answer how you evaluate it is with eval or subst. In your case probably subst.
Edit:
If your config file looks like this:
# This is a comment
variable = value
othervar = [doStuff]
you can use some tricks to get Tcl parsing it for you:
rename ::unknown ::_confp_unknown_orig
proc unknown args {
if {[llength $args] == 3 && [lindex $args 1] eq "="} {
# varname = value
uplevel 1 [list set [lindex $args 0] [lindex $args 2]
return [lindex $args 2]
}
# otherwise fallback to the original unknown
uplevel 1 [linsert $args 0 ::_confp_unknown_orig]
# if you are on 8.6, replace the line above with
# tailcall ::_confp_unknown_orig {*}$args
}
# Now just source the file:
source file1.txt
# cleanup - if you like
rename ::unknown {}
rename ::_confp_unknown_orig ::unknown
An other way to do that is to use a safe interp, but in this case using your main interp looks fine.
The problem is that the code you store inside val is never executed.
You access it using $val, but this way you get the code itself, and not the result of its execution.
To solve it, you must be sure [::ip::contract [::ip::prefix 1.1.1.1/24]]/24 is executed, and you can do that by replacing this line
set val [string trim [string range $line1 $mark end]]
with this one
eval "set val [string trim [string range $line1 $mark end]]"
Why? Here's my simple explaination:
The parser sees the "..." part, so it performs substitutions inside it
The first substitution is the execution of the string range $line1 $mark end command
The second substitution is the execution of the string trim ... command
So, when substitutions are complete and the eval command is ready to run, its like your line has become
eval {set val [::ip::contract [::ip::prefix 1.1.1.1/24]]/24}
Now the eval command is executed, it calls recursively the interpreter, so the string set val [::ip::contract [::ip::prefix 1.1.1.1/24]]/24 goes to another substitution phase, which finally runs what you want and puts the string 1.1.1/24 into the variable val.
I hope this helps.