Expect : error can't read "ip": no such variable - tcl

I am a newbie in expect / TCL and trying to parse an HTML page that has output some thing like below:
<li><p>Timestamp: Wed, 14 Nov 2012 16:37:50 -0800
<li><p>Your IP address: 202.76.243.10</p></li>
<li><p class="XXX_no_wrap_overflow_hidden">Requested URL: /</p></li>
<li><p>Error reference number: 1003</p></li>
<li><p>Server ID: FL_23F7</p></li>
<li><p>Process ID: PID_1352939870.809-1-428432242</p></li>
<li><p>User-Agent: </p></li>
My script is below. I am able to get the web page which I am not able to parse the line "Your IP address:" which is giving me errors:
#!/usr/bin/expect -f
set timeout -1
spawn telnet www.whatismyip.com 80
send "GET /\r\n"
expect
set output $expect_out(buffer)
foreach line [split $output \n] {
regexp {.*<li><p>Your IP Address Is:.*?(\d+\.\d+\.\d+\.\d+)} $line ip
if {[string length ${ip}]} {
puts $ip
}
}
The error is:
Connection closed by foreign host.
can't read "ip": no such variable
while executing
"string length ${ip}"
("foreach" body line 3)
invoked from within
"foreach line [split $output \n] {
regexp {.*<li><p>Your IP Address Is:.*?(\d+\.\d+\.\d+\.\d+)} $line ip
if {[string length ${ip}]} {
..."
(file "./t4" line 7)
Any pointers where I am doing wrong?

The regular expression did not match, so the variable was not assigned. You should check the result of regexp to see if the match succeeded; when not using the -all option to regexp, you can treat it like a boolean. Try this:
foreach line [split $output \n] {
if {[regexp {<li><p>Your IP Address Is:.*?(\d+\.\d+\.\d+\.\d+)(?!\d)} $line -> ip]} {
puts $ip
}
}
The -> is really a (weird!) variable name which will hold the whole matched string; we're not interested in it (just the parenthetical part) so we use the non-alphabetic to mnemonically say “this is going to there” (the submatch to the ip variable).

Your line contains "address" (lowercase) but you're trying to match "Address" (uppercase). Add the
-nocase option to the regexp command. Also, Tcl regular expressions cannot have mixed greediness -- the first quantifier determines if the whole expression is greedy or non-greedy (I can't find where this is documented right now).
regexp -nocase {IP Address.*(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})} $line -> ip

If your ultimate goal is to get your host's external IP, then go with an API solution, such as one from exip.org:
#!/usr/bin/env tclsh
set api http://api-nyc01.exip.org/?call=ip
if {[catch {exec curl --silent $api} output]} {
puts "Failed to acquire external IP"
} else {
puts "My external IP is $output"
}
Please visit their API site for more information, especially if you live outside the USA. This solution requires curl, which you might need to install.

Related

How to check the string value of an argument passed to expect script?

I'm writing a expect script that takes command line arguments. I would like to be able to detect whether the first argument is "--help" and print a Usage string then. Otherwise use the argument as a port number with a specific default (let's say 1818).
I tried this code that fails:
#!/usr/bin/expect
if {[llength $argv] != 1} {
puts "No Port number specified, defaulting to port 1818."
set port 1818
} else {
if {[lindex $argv 0] eq "--help"} {
puts "Usage: testit [--help] [port]"
exit
} else {
set port [lindex $argv 0]
}
}
The error is:
invalid command name "--help"
while executing
"--help"
invoked from within
"if {[llength $argv] != 1} {
puts "No Port number specified, defaulting to port 1818."
set port 1818
} else {
if {[lindex $argv 0] eq "--he..."
Obviously it is trying to interpret the content of the "--help" string while I'm trying to make the script compare the value of argument 0 to "--help".
What is wrong in the above logic or syntax?
I tried using other strings, like "help" instead of "--help" but the outcome is the same.
I'm not that familiar with expect and tcl, but I tried the expression in tclsh and the same thing happens there. So this issue has to do with invalid tcl code. The following tcsh session shows that the syntax if {$variable=="--help"} {...} is OK, but removing the white space in my string comparison attempt above does not solve the problem.
Here's the tcsh session:
% set v1 "--help"
--help
% if [v1 == "--help'] { puts "allo"}
extra characters after close-quote
% if [v1 == "--help"] { puts "allo"}
invalid command name "v1"
% if [$v1 == "--help"] { puts "allo"}
invalid command name "--help"
% if $v1 == help {puts "allo"}
invalid bareword "help"
in expression "--help";
should be "$help" or "{help}" or "help(...)" or ...
% if $v1 == "--help" {puts "allo"}
invalid bareword "help"
in expression "--help";
should be "$help" or "{help}" or "help(...)" or ...
% if {$v1=="--help"} {puts "allo"}
allo
%
The problem is this line:
puts "Usage: testit [--help] [port]"
And the problem with it is that [...] does command substitution in that situation. You need to add a couple of backslashes in there to prevent that, like this:
puts "Usage: testit \[--help] \[port]"
Or you can enclose the string in braces to inhibit all substitutions:
puts {Usage: testit [--help] [port]}
Either will work (and they'll get compiled to exactly the same thing so use whichever you prefer).

Expect - avoid sending escape prompt sequences via ssh

The script is intended to retrieve the contents of some directory when it is getting full.
For development, the 'full' was set at 15%, the directory is /var/crash.
expect "#*" {
foreach part $full {
puts "part: $part"
set dir [split $part]
puts "dir: $dir [llength $dir]"
set d [lindex $dir 0]
puts "d: $d"
send -s -- "ls -lhS $d\n"
expect "#*" { puts "for $dir :: $expect_out(buffer)"}
}
}
send "exit\r"
The output of the script is:
part: /var/crash 15%
dir: {/var/crash} 15% 2
d: /var/crash
send: sending "ls -lhS \u001b[01;31m\u001b[K/var\u001b[m\u001b[K/crash\n" to { exp7 }
expect: does "" (spawn_id exp7) match glob pattern "#*"? no
expect: does "ls -lhS \u00071;31m\u0007/var\u0007\u0007/" (spawn_id exp7) match glob pattern "#*"? no
expect: does "ls -lhS \u00071;31m\u0007/var\u0007\u0007/crash\r\n" (spawn_id exp7) match glob pattern "#*"? no
As can be seen, although $d is /var/crash, when it is sent via ssh it becomes something like \u001b[01;31m\u001b[K/var\u001b[m\u001b[K/crash.
I cannot change the remote machine definitions for the command prompt.
How to get rid of these escape sequences that are sent?
Edit: Info about $full as requested
The proc analyze just tries to filter meaningful data.
proc analyze_df {cmd txt} {
set full [list]
set lines [split $txt \n]
foreach l $lines {
if {[string match $cmd* $l]} { continue }
set lcompact [regsub -all {\s+} $l " "]
set data [split $lcompact]
if {[string match 8?% [lindex $data 4]] \
|| [string match 9?% [lindex $data 4]] \
|| [string match 1??% [lindex $data 4]] \
|| [string match 5?% [lindex $data 4]] \
|| [string match 1?% [lindex $data 4]] } {
lappend full "[lindex $data 5] [lindex $data 4]"
}
}
return $full
}
The extract about the $full that was missing.
set command0 "df -h | grep /var"
send -- "$pass\r"
expect {
-nocase "denied*" {puts "$host denied"; continue}
-nocase "Authentication failed*" {puts "$host authentication failed"; continue}
"$*" {send -s -- "$command0\n"}
timeout {puts "$host TIMEOUT"; continue}
}
expect "$*" {puts "$host -> $expect_out(buffer)" }
set full [analyze_df $command0 $expect_out(buffer)]
Taking the suggestion received, perhaps it's grep that is adding the escape sequences, no?
You don't show how $full gets its value. But it must already have the escape codes. When printing $d those escape codes are interpreted by the terminal, so they may not be obvious. But Expect/Tcl definitely doesn't insert them. This is also confirmed by the braces around the first element when you print $dir. If this element was plain /var/crash, there would be no braces.
Your remark about the command prompt would suggest that $full may be taken from there. Maybe you cannot permanently change the remote machine's command prompt, but you should be able to change it for your session by setting the PS1 environment variable.
Another trick that may help in such situations is to do set env(TERM) dumb before spawning the ssh command. If the prompt (or other tools) correctly use the tput command to generate their escape codes, a dumb terminal will result in empty strings. This won't work if the escape codes are hard-coded for one specific TERM. But that's a bug on the remote side.
If you're absolutely stuck with that input data (and can't tell things to not mangle it with those ANSI terminal colour escape codes) then you can strip them out with:
set dir [split [regsub -all {\u001b[^a-zA-z]*[a-zA-Z]} $part ""]]
This makes use of the fact that the escape sequences start with the escape character (encoded as \u001b) and continue to the first ASCII letter. Replacing them all with the empty string should de-fang them cleanly.
You are recommended to try things like altering the TERM environment variable before calling spawn so that you don't have to do such cleaning. That tends to be easier than attempting to "clean up" the data after the fact.

TCL regsub uses RegEx match as index in associate array

I'd like to automatically convert URLs, i.e
"https://sc-uat.ct.example.com/sc/" into "https://invbeta.example.com/sc/"
"https://sc-dev.ct.example.com/sc/" into "https://invtest.example.com/sc/"
"https://sc-qa.ct.example.com/sc/" into "https://invdemo.example.com/sc/"
I've tried following code snippet in TCL
set loc "https://sc-uat.ct.example.com/sc/"
set envs(dev) "test"
set envs(uat) "beta"
set envs(qa) "demo"
puts $envs(uat)
regsub -nocase {://.+-(.+).ct.example.com} $loc {://inv[$envs(\1)].example.com} hostname
puts "new location = $hostname"
But the result is: new location = https://inv[$envs(uat)].example.com/sc/
It seems that [$envs(uat)] is NOT evaluated and substituted further with the real value. Any hints will be appreciated. Thanks in advance
But the result is: new location =
https://inv[$envs(uat)].example.com/sc/ It seems that [$envs(uat)] is eval-ed further.
You meant to say: [$envs(uat)] is not evaluated further?
This is because due to the curly braces in {://inv[$envs(\1)].example.com}, the drop-in string is taken literally, and not subjected to variable or command substitution. Besides, you don't want command and variable substitution ([$envs(\1)]), just one of them: $envs(\1) or [set envs(\1)].
To overcome this, you must treat the regsub-processed string further via subst:
set hostname [subst -nocommands -nobackslashes [regsub -nocase {://.+-(.+).ct.example.com} $loc {://inv$envs(\1).example.com}]]
Suggestions for improvement
I advise to avoid the use of subst in this context, because even when restricted, you might run into conflicts with characters special to Tcl in your hostnames (e.g., brackets in the IPv6 authority parts). Either you have to sanitize the loc string before, or, better work on string ranges like so:
if {[regexp -indices {://(.+-(.+)).ct.example.com} $loc _ replaceRange keyRange]} {
set key [string range $loc {*}$keyRange]
set sub [string cat "inv" $envs($key)]
set hostname [string replace $loc {*}$replaceRange $sub]
}

Expect Script - How can the first and last line of a file be validated

I'm using Expect to create a CSR file on a remote system. I'm capturing the output from the system and placing it into a file on my local PC (where it is needed).
I need to validate the first and last line of this file to make sure the file looks like the following:
-----BEGIN CERTIFICATE REQUEST-----
.
.
.
-----END CERTIFICATE REQUEST-----
Originally I was only looking for the last line (or so I thought) by looping through the lines of the file looking for -----END CERTIFICATE REQUEST-----
set fp [ open $csrname ]
while {[gets $fp line] != -1} {
if { $line == "-----END CERTIFICATE REQUEST-----" } {
puts "The Certificate Signing Request file \"$csrname\" has been succesfully created"
} else {
puts "The certificate file is invalid."
puts $line
exit 41
}
}
I have a flaw in my logic because I end up in the error leg of that if statement and exit.
How can I validate just the first and last line of the file?
A CSR isn't really all that long; just a few kilobytes at most. We can validate the lot in one go!
# Load *everything* from a file at once
set f [open $csrname]
set contents [read $f]
close $f
# Validate it using this regular expression:
set RE {^-----BEGIN CERTIFICATE REQUEST-----\n.*\n-----END CERTIFICATE REQUEST-----\n*$}
if {![regexp $RE $contents]} {
puts "The certificate file is invalid."
exit 1
# This spot is unreachable, of course...
}
puts "The Certificate Signing Request file \"$csrname\" has been succesfully created"
I'd recommend checking that the bit between the separators is only using valid characters too (it's base64-encoded PKCS#10), but that's rather more complicated once you go beyond the basics. Probably best to just confirm that you've not got truncation or something that just isn't a CSR at all.
Donal has a nice answer. Here are a couple of alternatives:
set first [exec sed {1q} $csrname]
set last [exec sed -n {$p} $csrname]
or
set f [open $csrname]
set lines [split [read -nonewline $f] \n]
close $f
set first [lindex $lines 0]
set last [lindex $lines end]
In either case, you can
if {$first eq "-----BEGIN CERTIFICATE REQUEST-----" &&
$last eq "-----END CERTIFICATE REQUEST-----"} {...}
Also you can refer to http://wiki.tcl.tk/1466
[eof] is used to determine whether the channel has reached the end of input.

Get line number using grep

I would like to get the line number using grep command, but I am getting the error message when search pattern is not a single word:
couldn't read file "Pattern": no such file or directory
How should be the proper usage of the grep? The code is here:
set status [catch {eval exec grep -n '$textToGrep' $fileName} lineNumber]
if { $status != 0 } {
#error
} else {
puts "lineNumber = $lineNumber"
}
Also if the search pattern is not matched at all, the returned value is : "child process exited abnormally"
Here is the simple test case:
set textToGrep "<BBB name=\"BBBRM\""
file contents:
<?xml version="1.0"?>
<!DOCTYPE AAA>
<AAA>
<BBB name="BBBRM" />
</AAA>
Well, I also get problems with your code and a single word pattern!
First of all, I don't think you need the eval command, because catch itself does an evaluation of its first argument.
Then, the problem is that you put the $textToGrep variable in exec inside single quotes ', which have no meaning to Tcl.
Therefore, if the content of textToGrep is foo, you are asking grep to search for the string 'foo'. If that string, including the single quotes, is not found in the file, you get the error.
Try to rewrite your first line with
set status [catch {exec grep -n $textToGrep $fileName} lineNumber]
and see if it works. Also, read the exec man page, which explains well these problems.
If your system has tcllib install, you can use the fileutil::grep command from the fileutil package:
package require fileutil
set fileName data.xml
set textToGrep {<BBB +name="BBBRM"}; # Update: Add + for multi-space match
set grepResult [::fileutil::grep $textToGrep $fileName]
foreach result $grepResult {
# Example result:
# data.xml:4: <BBB name="BBBRM" />
set lineNumber [lindex [split $result ":"] 1]
puts $lineNumber
# Update: Get the line, squeeze the spaces before name=
set line [lindex [split $result ":"] 2]
regsub { +name=} $line " name=" line
puts $line
}
Discussion
When assigning value to textToGrep, I used the curly braces, thus allowing double quote inside without having to escape them.
the result of the ::fileutil::grep command is a lits of strings. Each string contains the file name, line number, and the line itself; separated by colon.
One way to extract the line number is to first split the string (result) into pieces, using the colon as a separator. Next, I use lindex to grab the second item (index=1, since list is zero-base).
I have updated the code to account for case where there are multiple spaces before name=
There are two problems here:
Pattern matching does not work.
grep exits with error child process
exited abnormally when pattern is not found
The first problem is because you are not enclosing the textToGrep within double quotes(instead of single quotes). So your code should be:
[catch {exec grep -n "$textToGrep" $fileName} lineNumber]
Second problem is because of the exit status of grep command. grep exits with error when the pattern is not found. Here is the try on a shell:
# cat file
pattern
pattern with multiple spaces
# grep pattern file
pattern
pattern with multiple spaces
# echo $?
0
# grep nopattern file
# echo $?
1
EDIT:
In your case you have special characters such as < and > (which have special meaning on a shell).
set textToGrep "<BBB name=\"BBBRM\""
regsub -all -- {<} "$textToGrep" "\\\<" textToGrep
regsub -all -- {>} "$textToGrep" "\\\>" textToGrep
set textToGrep {\<BBB name="BBBRM"}
catch {exec grep -n $textToGrep $fileName} status
if {![regexp "child process" $status]} {
puts $status
} else {
puts "no word found"
}
I think you should do regular expression with child process. Just check above code if it works. In if statement you can process the status command as you like.
With the given example (in your post) the above code works only you need to use backslash for the "<" in the textToGrep variable