file writing operation in tcl takes lot of time - tcl

Question

set filePointer [open "fileName" "r"]
set fileWritePointer [open "fileNameWrite" "w"]
set lines [split [read $filePointer] "\n"]
close $filePointer
set length [llength $lines]
for {set i 0} {$i<$length} {incr i} {
if {[regexp "Matching1" $line]} {
puts $fileWritePointer $line
}
if {[regexp "Matching" $line]} {
puts $fileWritePointer $line
}
}
close $fileWritePointer
I am reading all the lines of the file at a time and splitting it by new line character and reading each line at a time inside the for loop.
After some syntax checks using regexp for the lines I am dumping only selected lines into a new file using the below syntax.
puts $filePointer $line
My file has around 2 million lines of code.
Like this many regexp matching is present roughly around 1.5.

Answer 1

Without knowing why the code is slow (or what exactly you're using a baseline for measurement against) it's hard to be sure what to do to accelerate it. However, you can try switching to streaming processing:
set fin [open "fileName"]
set fout [open "fileNameWrite" "w"]
while {[gets $fin line] >= 0} {
if {[regexp "Matching1" $line]} {
puts $fout $line
}
if {[regexp "Matching" $line]} {
puts $fout $line
}
}
close $fout
close $fin
You should make sure that your regular expressions are constant values for the duration of the processing to avoid recompiling them for every line (which would be very slow!) though those constant values can be stored in variables, so long as those variables are used without anything being added to them:
set RE1 "Matching1"
set RE2 "Matching"
# Note: these variables are NOT assigned to below! They are just used!
set fin [open "fileName"]
set fout [open "fileNameWrite" "w"]
while {[gets $fin line] >= 0} {
# Added “--” to make sure that the REs are never interpreted as anything else
if {[regexp -- $RE1 $line]} {
puts $fout $line
}
if {[regexp -- $RE2 $line]} {
puts $fout $line
}
}
close $fout
close $fin
You might also get extra speed by choosing the right encodings, putting all this code in a procedure, etc. As noted, it's hard to be sure what is the best thing to try without knowing why the code is actually slow, and that in part depends on the system on which it is being run.

Answer 2

Do you actually need regular expression matching? String matching is likely to be faster.
Can more than one match be made against the same line, and in that case do you really need the line to be written once for each match? If not, you can speed things up by skipping the rest of the matching attempts once one has succeeded:
if {[regexp -- $RE1 $line]} {
puts $fout $line
} elseif {[regexp -- $RE2 $line]} {
puts $fout $line
} elseif { ... } {
or
if {
[regexp -- $RE1 $line] ||
[regexp -- $RE2 $line] ||
...
} then {
puts $fout $line
}
or
switch -regexp -- $line \
$RE1 - \
$RE2 - \
... - \
default {
puts $fout $line
}