Getting the next key in an array - tcl

Question

My goal is to find the next key in an array... below my data :
# Index increment may change, there is not necessarily continuity like this example.
# My $index can be 1,2,3,4,8,12,25,32...
# but the size of my array is about 100,000 elements.
for {set index 1} {$index < 100000} {incr index} {
set refdata($index,Pt,X) [expr {10 + $index}]
}
I need to know the next key to be able to build a geometric line... I did not find in the help a command that allows me to find the next key of my array so I created my own function below :
proc SearchNextKeyArrayElement {dataarray mykey} {
upvar $dataarray myarray
set mydata [lsort -dictionary [array names myarray]]
set index [lsearch $mydata $mykey]
if {$index > -1} {
return [lindex $mydata [expr {$index + 1}]]
}
return ""
}
foreach k [lsort -dictionary [array names refdata]] {
if {[string match "*,Pt,*" $k]} {
set nextkey [SearchNextKeyArrayElement refdata $k]
}
}
And it takes a long time...array nextelement command is maybe the solution...But I do not understand how to use it ?

Answer 1

Here's an example:
start a search with array startsearch
loop while array anymore is true
get the next key with array nextelement
tidy up with array donesearch
use try {} catch {} finally for safety
# array foreach
# to be subsumed in Tcl 8.7 by `array for`
# https://core.tcl.tk/tips/doc/trunk/tip/421.md
#
# example:
# array set A {foo bar baz qux}
# array foreach {key val} A {puts "name=$key, value=$val"}
#
# A note on performance: we're not saving any time with this approach.
# This is essentially `foreach name [array names ary] {...}
# We are saving memory: iterating over the names versus extracting
# them all at the beginning.
#
proc array_foreach {vars arrayName body} {
if {[llength $vars] != 2} {
error {array foreach: "vars" must be a 2 element list}
}
lassign $vars keyVar valueVar
# Using the complicated `upvar 1 $arrayName $arrayName` so that any
# error messages propagate up with the user's array name
upvar 1 $arrayName $arrayName \
$keyVar key \
$valueVar value
set sid [array startsearch $arrayName]
# If the array is modified while a search is ongoing, the searchID will
# be invalidated: wrap the commands that use $sid in a try block.
try {
while {[array anymore $arrayName $sid]} {
set key [array nextelement $arrayName $sid]
set value [set "${arrayName}($key)"]
uplevel 1 $body
}
} trap {TCL LOOKUP ARRAYSEARCH} {"" e} {
puts stderr [list $e]
dict set e -errorinfo "detected attempt to add/delete array keys while iterating"
return -options $e
} finally {
array donesearch $arrayName $sid
}
return
}

Answer 2

Generally speaking, Tcl arrays have no order at all; they can change their order on any modification to the array or any of its elements. The commands that iterate over the array (array for, array get, array names, and the iteration commands array startsearch/array nextelement/array anymore) only work with the current order. However, you can use array names to get the element names into a Tcl list (which is order preserving), sort those to get the order that you're going to iterate over, and then use foreach over that. As long as you're not adding or removing elements, it'll be fine. (Adding elements is sort-of OK too; you'll just not see them in your iteration.)
foreach key [lsort -dictionary [array names myarray]] {
ProcessElement $key $myarray($key)
}
By contrast, trying to just go from one element to the next will hurt a lot; that operation is not exposed.
Using the iteration commands is done like this:
set s [array startsearch myarray]
while {[array anymore myarray $s]} {
set key [array nextelement myarray $s]
ProcessElement $key $myarray($key)
}
Note that you don't get an option to sort the search. You won't see these used much in production code; doing array names or array get is usually better. And now (well, 8.7 is still in alpha) you've also got array for:
array for {key value} myarray {
ProcessElement $key $value
}
Efficient for large arrays, but still doesn't permit sorting; supporting direct sorting would require a different sort of storage engine on the back of the array.

Answer 3

This is why it's slow: You're sorting the array names once for the foreach command and then again for each element. Sort once and cache it, then you can iterate over it much more efficiently
set sorted_names [lsort -dictionary [array names refdata -glob {*,Pt,*}]]
set len [llength $sorted_names]
for {set i 0; set j 1} {$i < $len} {incr i; incr j} {
set this_name [lindex $sorted_names $i]
set next_name [lindex $sorted_names $j]
# ...
}