I have a program read from stdin and process it. ( like "tee /some/file" )
This program wait stdin end to exit itself.
If I spawn it from Expect, after I send many content, how to send an "EOF" to the program?
there is a close command in Expect, but it will also send a SIGHUP, and can not expect program output anymore.
Expect works (on non-Windows) by using a virtual terminal which the spawned program runs within. This means that you can do things by sending character sequences to simulate keys. In particular, the EOF control sequence is done with Ctrl+D, which becomes the character U+000004. The terminal processes this to turn it into a true EOF.
There's a few ways to write it, depending on which escape sequence you prefer, but one of these will work:
# Hexadecimal-encoded escape
send \x04
# Octal-encoded escape
send \004
# UNICODE escape (also hexadecimal)
send \u0004
# Generate by a command
send [format "%c" 4]
When Expect is using Tcl 8.6, these all generate the same bytecode so use whichever you prefer.
Related
I would like to read from stdin on a per character basis without the stdin being flushed. I could not find how to do that after tweaking for hours. Tcl always seems to wait for the channel to be flushed even in fconfigure stdin -blocking 0 -buffering none. Is that true? How would I otherwise approach this?
More explanation:
Imagine a Tcl program that makes its own prompt with some threads running code in the background. I would like this prompt to react to single keystrokes, for example: when you press 'p' (without pressing enter) the prompt reads that character and pauses the threads, when you press 'q' the prompt kills the threads and stops the program. The cleanest and closest solution is shortly demonstrated in the following code snippet.
proc readPrompt { } {
set in [ read stdin 1 ]
if { $in eq "q" } {
puts "Quitting..."
set ::x 1
} {
puts "Given unknown command $in"
}
}
fconfigure stdin -blocking 0 -buffering none
fileevent stdin readable { readPrompt }
vwait x
The result from running this is:
a
Given unknown command a
Given unknown command
After pressing the 'a', nothing happens. My guess is that the stdin is not flushed or something. Pressing enter or CTRL-d triggers the fileevent and the prompt then reads both the characters 'a' and 'enter'.
Ideally, I want the enter-press not to be needed. How could I accomplish this?
EDIT: I found this question and solution about a related use in Python: Determine the terminal cursor position with an ANSI sequence in Python 3 This is approximately the behaviour I'm looking for, but in Tcl.
If you have 8.7 (currently in alpha) then this is “trivial”:
fconfigure stdin -inputmode raw
That delivers all characters to you, without echoing them. (There's also modes normal and password, both of which preprocess the data before delivery and only one of which echoes.) You'll have to look after giving visual feedback to the user yourself, and be aware that all includes all characters usually only used for line editing purposes.
Otherwise, on Unixes (Linux, macOS) you do:
exec stty raw -echo <#stdin >#stdout
to switch the mode to the same config, and:
exec stty -raw echo <#stdin >#stdout
to switch back. (Not all Unixes need the input and output redirects, but some definitely do.)
Windows consoles have something similar in 8.7, but not in previous versions; a workaround might be possible using the TWAPI console support but that's a very low level API (and I don't know the details).
Want to use some custom function (written in tcl) in Unix pipeline ie grep patt file.rpt | tclsh summary.tcl. How to make tcl script to take output from the pipeline, process and out on the commandline as if a normal unix command?
This is very easy! The script should read input from stdin (probably with gets) and write output to stdout (with puts; this is the default destination).
Here's a very simple by-line filter script:
set lineCount 0
while {[gets stdin line] >= 0} {
incr lineCount
puts stdout "$lineCount >> $line <<"
}
puts "Processed $lineCount lines in total"
You probably want to do something more sophisticated!
The gets command has two ways of working. The most useful one here is the one where it takes two arguments (channel name, variable name), writes the line it has read from the channel into the variable, and returns the number of characters read or -1 when it has an EOF (or would block in non-blocking mode, or has certain kinds of problems; you can ignore these cases). That works very well with the style of working described above in the sample script. (You can distinguish the non-success cases with eof stdin and fblocked stdin, but don't need to for this use case.)
I am navigating a Java-based CLI menu on a remote machine with expect inside a bash script and I am trying to extract something from the output without leaving the expect session.
Expect command in my script is:
expect -c "
spawn ssh user#host
expect \"#\"
send \"java cli menu command here\r\"
expect \"java cli prompt\"
send \"java menu command\"
"
###I want to extract a specific string from the above output###
Expect output is:
Id Name
-------------------
abcd 12 John Smith
I want to extract abcd 12 from the above output into another expect variable for further use within the expect script. So that's the 3rd line, first field by using a double-space delimiter. The awk equivalent would be: awk -F ' ' 'NR==3 {$1}'
The big issue is that the environment through which I am navigating with Expect is, as I stated above, a Java CLI based menu so I can't just use awk or anything else that would be available from a bash shell.
Getting out from the Java menu, processing the output and then getting in again is not an option as the login process lasts for 15 seconds so I need to remain inside and extract what I need from the output using expect internal commands only.
You can use regexp in expect itself directly with the use of -re flag. Thanks to Donal on pointing out the single quote and double quote issues. I have given solution using both ways.
I have created a file with the content as follows,
Id Name
-------------------
abcd 12 John Smith
This is nothing but your java program's console output. I have tested this in my system with this. i.e. I just simulated your program's output with cat. You just replace the cat code with your program commands. Simple. :)
Double Quotes :
#!/bin/bash
expect -c "
spawn ssh user#domain
expect \"password\"
send \"mypassword\r\"
expect {\\\$} { puts matched_literal_dollar_sign}
send \"cat input_file\r\"; # Replace this code with your java program commands
expect -re {-\r\n(.*?)\s\s}
set output \$expect_out(1,string)
#puts \$expect_out(1,string)
puts \"Result : \$output\"
"
Single Quotes :
#!/bin/bash
expect -c '
spawn ssh user#domain
expect "password"
send "mypasswordhere\r"
expect "\\\$" { puts matched_literal_dollar_sign}
send "cat input_file\r"; # Replace this code with your java program commands
expect -re {-\r\n(.*?)\s\s}
set output $expect_out(1,string)
#puts $expect_out(1,string)
puts "Result : $output"
'
As you can see, I have used {-\r\n(.*?)\s\s}. Here the braces prevent any variable substitutions. In your output, we have a 2nd line with full of hyphens. Then a newline. Then your 3rd line content. Let's decode the regex used.
-\r\n is to match one literal hyphen and a new line together. This will match the last hyphen in the 2nd line and the newline which in turn make it to 3rd line now. So, .*? will match the required output (i.e. abcd 12) till it encounters double space which is matched by \s\s.
You might be wondering why I need parenthesis which is used to get the sub-match patterns.
In general, expect will save the expect's whole match string in expect_out(0,string) and buffer all the matched/unmatched input to expect_out(buffer). Each sub match will be saved in subsequent numbering of string such as expect_out(1,string), expect_out(2,string) and so on.
As Donal pointed out, it is better to use single quote's approach since it looks less messy. :)
It is not required to escape the \r with the backslash in case of double quotes.
Update :
I have changed the regexp from -\r\n(\w+\s+\w+)\s\s to -\r\n(.*?)\s\s.
With this way - your requirement - such as match any number of letters and single spaces until you encounter first occurrence of double spaces in the output
Now, let's come to your question. You have mentioned that you have tried -\r\n(\w+)\s\s. But, there is a problem here with \w+. Remember \w+ will not match space character. Your output has some spaces in it till double spaces.
The use of regexp will matter based on your requirements on the input string which is going to get matched. You can customize the regular expressions based on your needs.
Update version 2 :
What is the significance of .*?. If you ask separately, I am going to repeat what you commented. In regular expressions, * is a greedy operator and ? is our life saver. Let us consider the string as
Stackoverflow is already overflowing with number of users.
Now, see the effect of the regular expression .*flow as below.
* matches any number of characters. More precisely, it matches the longest string possible while still allowing the pattern itself to match. So, due to this, .* in the pattern matched the characters Stackoverflow is already over and flow in pattern matched the text flow in the string.
Now, in order to prevent the .* to match only up to the first occurrence of the string flow, we are adding the ? to it. It will help the pattern to behave as non-greedy manner.
Now, again coming back to your question. If we have used .*\s\s, then it will match the whole line since it is trying to match as much as possible. This is common behavior of regular expressions.
Update version 3:
Have your code in the following way.
x=$(expect -c "
spawn ssh user#host
expect \"password\"
send \"password\r\"
expect {\\\$} { puts matched_literal_dollar_sign}
send \"cat input\r\"
expect -re {-\r\n(.*?)\s\s}
if {![info exists expect_out(1,string)]} {
puts \"Match did not happen :(\"
exit 1
}
set output \$expect_out(1,string)
#puts \$expect_out(1,string)
puts \"Result : \$output\"
")
y=$?
# $x now contains the output from the 'expect' command, and $y contains the
# exit status
echo $x
echo $y;
If the flow happened properly, then exit code will have value as 0. Else, it will have 1. With this way, you can check the return value in bash script.
Have a look at here to know about the info exists command.
How do you process a Chrome Native Messaging API-call with a bash script?
I succeeded in doing it with python with this example
Sure I can call bash from the python code with subprocess, but is it possible to skip python and process the message in bash directly?
The problematic part is reading the JSON serialized message into a variable. The message is serialized using JSON, UTF-8 encoded and is preceded with 32-bit message length in native byte order through stdin.
echo $* only outputs:
chrome-extension://knldjmfmopnpolahpmmgbagdohdnhkik/
Also something like
read
echo $REPLY
doesn't output anything. No sign of the JSON message. Python uses struct.unpack for this. Can that be done in bash?
I suggest to not use (bash) shell scripts as a native messaging host, because bash is too limited to be useful.
read without any parameters reads a whole line before terminating, while the native messaging protocol specifies that the first four bytes specify the length of the following message (in native byte order).
Bash is a terrible tool for processing binary data. An improved version of your read command would specify the -n N parameter to stop reading after N characters (note: not bytes) and -r to remove some processing. E.g. the following would store the first four characters in a variable called var_prefix:
IFS= read -rn 4 var_prefix
Even if you assume that this stores the first four bytes in the variable (it does not!), then you have to convert the bytes to an integer. Did I already mention that bash automatically drops all NUL bytes? This characteristics makes Bash utterly worthless for being a fully capable native messaging host.
You could cope with this shortcoming by ignoring the first few bytes, and start parsing the result when you spot a { character, the beginning of the JSON-formatted request. After this, you have to read all input until the end of the input is found. You need a JSON parser that stops reading input when it encounters the end of the JSON string. Good luck with writing that.
Generating output is a easier, just use echo -n or printf.
Here is a minimal example that assumes that the input ends with a }, reads it (without processing) and replies with a result. Although this demo works, I strongly recommend to not use bash, but a richer (scripting) language such as Python or C++.
#!/bin/bash
# Loop forever, to deal with chrome.runtime.connectNative
while IFS= read -r -n1 c; do
# Read the first message
# Assuming that the message ALWAYS ends with a },
# with no }s in the string. Adopt this piece of code if needed.
if [ "$c" != '}' ] ; then
continue
fi
message='{"message": "Hello world!"}'
# Calculate the byte size of the string.
# NOTE: This assumes that byte length is identical to the string length!
# Do not use multibyte (unicode) characters, escape them instead, e.g.
# message='"Some unicode character:\u1234"'
messagelen=${#message}
# Convert to an integer in native byte order.
# If you see an error message in Chrome's stdout with
# "Native Messaging host tried sending a message that is ... bytes long.",
# then just swap the order, i.e. messagelen1 <-> messagelen4 and
# messagelen2 <-> messagelen3
messagelen1=$(( ($messagelen ) & 0xFF ))
messagelen2=$(( ($messagelen >> 8) & 0xFF ))
messagelen3=$(( ($messagelen >> 16) & 0xFF ))
messagelen4=$(( ($messagelen >> 24) & 0xFF ))
# Print the message byte length followed by the actual message.
printf "$(printf '\\x%x\\x%x\\x%x\\x%x' \
$messagelen1 $messagelen2 $messagelen3 $messagelen4)%s" "$message"
done
How to execute set command without printing output on the screen? I want to read a file without displaying the contents on the screen.
set a [open "giri.txt" r]
set b [read $ifile]
What you're observing is just the standard behaviour of an interactive Tcl shell: each Tcl command returns a result value, and a return code. If the Tcl shell is interactive (that is, its input and output streams are connected to a terminal), after executing each command, the string representation of the result value the command returned is printed, and then the prompt is shown again. If the shell is not interactive, no results are printed and no prompt is shown.
(On a side note, such behaviour is ubiquitous with interpreters — various Unix shells, Python and Ruby interpreters do the same thing.)
If you want to inhibit such printouts in an interactive session (comes in handy from time to time), a simple hack to achieve that is to chain the command you want to "silence" with a "silent" command (producing a value whose string representation is an empty string), for instance:
set a [open "giri.txt" r]; list
Here, the list returned by the list command having no arguments is an empty list whose string representation is an empty string. In an interactive shell, this chain of commands will output literally nothing.
It bears repeating that such a hack might only ever be needed in an interactive session — do not use it in scripts.
In Mentor ModelSim Tcl it is possible to do:
quietly set answer 42
Also in Mentor Questa:
help quietly
The quietly command turns off transcript echoing for the specified command.
You can turn this off in an interactive tclsh
set tcl_interactive false
but that will also turn off the prompt.