rsync error: when using source which contain "*" character - tcl

In my TCL script I use rsync command and unfortunately it is refusing to sync path contain "*" character.
Code Example:
set srs "/users/home/username/common/*"
catch {exec rsync -av $src /tmp} res
puts $res
Output:
building file list ... done
sent 29 bytes received 20 bytes 98.00 bytes/sec
total size is 0 speedup is 0.00
rsync: link_stat "/users/home/username/common/*" failed: No such file or directory (2)
rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
while executing
"exec rsync -rLptgov $src /tmp "

The canonical way of writing that is:
set srs "/users/home/username/common/*"
catch {exec rsync -av {*}[glob $src] /tmp} res
puts $res
This is because Tcl doesn't expand glob metacharacters by default; it's safer that way and easier to write correct code (it's hard to write good shell code that is resistant to problems in this area) but it does mean that you need to do a little extra work. The extra work is:
Ask for glob expansion of $srs with the glob command. That returns a list.
Ask for expansion of the list out of the glob by using the {*} pseudo-operator (it's not an operator — it's technically a kind of syntax — but it works a lot like one).
If you're using Tcl 8.4 (or before!) then you do it a bit different way:
set srs "/users/home/username/common/*"
catch {eval [list exec rsync -av] [glob $src] [list /tmp]} res
puts $res
OK, that's sometimes often shortened by leaving out the list bits (and there are many more obscure ways of writing it!) but that's a bad habit as it can cause huge problems when dealing with variables with values that aren't “nice”, e.g., with spaces in pathnames. If you have 8.5 or later, use {*}. Really.

To elaborate on what GrAnd said, you have to build the list of source files explicitly, probably using glob, so you'd do it like this
set cmd [concat [list exec rsync -av] \
[glob -nocomplain /users/home/username/common/*]]
lappend cmd /tmp
catch {eval $cmd} res
puts $res
or, alternatively, use {*} to expand the agument list for exec (this is for Tcl >= 8.5), like this:
set cmd [concat [list rsync -av] \
[glob -nocomplain /users/home/username/common/*]]
lappend cmd /tmp
catch {exec {*}$cmd} res
puts $res
Update: fixed example so that it really works.

That's because tcl interpreter does not expand the '*' symbol as shell does.
For example:
$ tclsh
% cd /usr
% exec ls -d *
ls: *: No such file or directory
% exec sh -c "ls -d *"
X11R6
bin
docs
etc
...

Related

How can I run a csh script from a tcl script, csh script contain bsub command?

I am trying to run a csh script within tcl script. The csh script contains bsub command in it. When i am trying to run script it is erroring out saying bsub: Command not found.
Script specimen :
exec cp -r x ../
cd ../../my_area
exec cp -r y ../../
cd ../../my_scripts
exec /bin/csh/ aim.csh >#stdout 2>#stderr
# also tried set hd aim.csh , exec /bin/csh $hd >#stdout 2>#stderr}
details of aim.csh
setenv IDC $d
setenv LGH $f
set error $e
bsub -q normal -db <path>
The correct way to invoke the script is like this:
exec /bin/csh aim.csh >#stdout 2>#stderr
You might want to make sure that the aim.csh file exists prior to running. It's important to do this after any cd call; that changes where the code will look for aim.csh.
If you have a problem with bsub not being on the executable search path, that's properly neither Tcl's nor csh's fault. (I've no idea where it would be, and can't remember which package supplies it.) You can extend the executable search path from your Tcl code if you need to:
# Before calling exec; after is too late
append env(PATH) :/path/to/directory
For example, if bsub were actually /opt/local/somewhere/bin/bsub, then you'd do:
append env(PATH) :/opt/local/somewhere/bin
A truly proper way to do this is with this procedure:
proc addToPath {directory} {
global env tcl_platform
if {[file type $directory] ne "directory"} {
error "must pass a directory name"
}
append env(PATH) $tcl_platform(pathSeparator) [file nativename $directory]
return
}
addToPath /opt/local/somewhere/bin
But the simple append is good enough for non-portable code.

Opening gnome-terminal and running script in tcl

When I try to execute below command it is showing Error like "extra characters after close-quote" but I gave it properly & when i try to it in unix command line terminal is opening properly.
exec gnome-terminal -e 'sh -c "bsub -Ip -n 1 -M <Memory> -q <queue_name> make"'
Can any one help me to resolve this issue or is there any way to do the same thing ??
Edited -> changed " from before sh to before bsub
Tcl's quoting is not the shell's quoting. Tcl uses {…} like the shell uses single quotes, except that braces nest nicely. Nesting single quotes is a recipe for shell headaches.
exec gnome-terminal -e {sh -c "bsub -Ip -n 1 -M <Memory> -q <queue_name> make"}
However, in this case I'd instead be tempted to go with this:
set memory "<Memory>"
set queue "<queue_name>"
set command "make"
set bsubcmd "bsub -Ip -n 1 -M $memory -q $queue $command"
# It's much more convenient to build this command like this here.
# Otherwise you're doing lots of backslashes and so on and it's horrible and very easy to make bugs
exec gnome-terminal -e [format {sh -c "%s"} $bsubcmd]
The only really messy thing is that command and bsubcmd have to be built using shell syntax if you're passing spaces around. “Fortunately” you're dealing with make anyway, so you probably really want to avoid having spaces in names passed there.

How to create temporary shellscript files and directories through Tcl language?

There are numerous ways to do this and the one I present here is one of them, using the “mktemp” tool that is already installed by default in many Linux distros.
Temporary Files and Directories
The "mktemp" tool comes by default in the "GNU coreutils" package and its purpose is solely to create temporary files/directories.
Creating tempfiles and tempdir
Its basic use is quite simple. Calling 'mktemp' from the command line without any parameters will create a file on your disk inside /tmp and whose name will be displayed on the screen. Look:
$ mktemp
/tmp/tmp.JVeOizVlqk
Creating directories is as simple as adding the “-d” parameter to the command line.
$ mktemp -d
/tmp/tmp.sBihXbcmKa
Using in practice
In practice there is no use in the temporary file or directory name being displayed on the screen. It must be stored in a variable that can be accessed at all times that can be read or written during processing.
In the example below there is a small script, useless by the way, but which shows us a suitable step-by-step. Look:
#!/bin/sh
#
#Create a temporary file
TEMP = `mktemp`
#Save all the content of the entered path
ls -l "$1"> $TEMP
#Read temporary file and list only directories
grep ^d $TEMP
sudo rm -f $TEMP
Note that the command "mktemp" was invoked in a subshell (between "- crase) and its output stored in the variable "TEMP".
Then, so that the file can be read or written just use the variable, where there would be the file name, as done in the ls, grep and rm commands.
If you needed to create directories, as already said, the process is the same, just adding a -d to the mktemp command line.
#!/bin/sh
#
TEMP = `mktemp -d`
cd $TEMP
.
.
.
sudo rm -rf $TEMP
If you want to create many temporary files, we use the “-p” parameter that specifies the path where the file should be created.
#!/bin/sh
#
TEMP = `mktemp -d`
cd $TEMP
FILE1 = `mktemp -p $TEMP`
.
.
.
sudo rm -f $FILE1
sudo rm -rf $TEMP
Only that. Your scripts will now be able to use temporary files more professionally.
However, I want to do this with tclsh instead of sh [bourn shell .. but I made a few attempts and nothing worked. Here's an example of what I tried:
# Create a temporary file
exec sh -c "TEMP =`mktemp -d`"
set dir [cd $TEMP]
# Save all content from the entered path
exec ls -l "$1"> $TEMP
set entry [glob -type f $env(dir) *.jpg]
# Read temporary file and list only directories
puts $entry
My biggest problem was and is in creating the variable
# Create a temporary file
exec sh -c "TEMP =`mktemp -d`"
This is not working!
Can someone give me a free sample ?!
Creating temporary files can be done using file tempfile. For directories file tempdir will be available in Tcl 8.7.
On Tcl versions before 8.7, you can use file tempfile to obtain a path to a temporary location and then create a directory by that name:
set fd [file tempfile temp]
close $fd
file delete $temp
file mkdir $temp
The file tempfile command also allows you to specify a template, similar to the -p option of mktemp
To answer your updated question, you can do something like:
# Create a temporary file
set temp [exec mktemp]
# Save all content from the entered path
exec ls -l [lindex $argv 0] > $temp
# Read temporary file
set f [open $temp]
set lines [split [read $f] \n]
close $f
# List only directories
puts [join [lsearch -all -inline $lines {d*}] \n]
I ignored your mixing up of directories and regular files and whatever the *.jpg is supposed to be.
Your attempts to create shell variables from inside Tcl and then use those in the next exec command will always fail because those variables are gone when the first subshell terminates. Keep the results in Tcl variables instead, like I did above.
Of course you could more easily find the directories using glob -type d, but I kept the shell command to serve as an example.
With example, the creating of directory temporary It would be like this:
# Create a temporary directory
set dir [exec mktemp -d] ;
# Now, files insert in directory
# (In this example I am decompressing a ZIP file and only JPEG format images)
exec unzip -x $env(HOME)/file.zip *.jpg -d $dir ;
# Look the directory now with this command:
puts [glob -nocomplain -type f -directory $dir -tails *.jpg] ;

Using eval exec with an executable name containing spaces in the path

I came across this scenario, where I have to execute tshark for decoding the pcap file.
Location of of tshark is C:\Program Files\Wireshark\tshark.exe
% set fileName TestTShark.pcap
TestTShark.pcap
% set sTsharkCmd "-r $fileName -Tfields -e ip.src"
-r TestTShark.pcap -Tfields -e ip.src
% set tsharkPath "C:/Program Files/Wireshark/tshark.exe"
C:/Program Files/Wireshark/tshark.exe
% eval exec $tsharkPath $sTsharkCmd > packet.log
couldn't execute "C:\Program": no such file or directory
% exec $tsharkPath $sTsharkCmd > packet.log
tshark: The file " TestTShark.pcap -Tfields -e ip.src" doesn't exist.
% set tsharkPath "C:\\Program Files\\Wireshark\\tshark.exe"
C:\Program Files\Wireshark\tshark.exe
% exec $tsharkPath $sTsharkCmd > packet.log
tshark: The file " TestTShark.pcap -Tfields -e ip.src" doesn't exist.
% eval exec $tsharkPath $sTsharkCmd > packet.log
couldn't execute "C:Program": no such file or directory
Since the path containing a space, while evaluating the code, Tcl treating C:\Program as a program name and throwing the error. Then I managed to escape the space with backslash as follows, which worked then.
% set tsharkPath "C:/Program\\ Files/Wireshark/tshark.exe"
C:/Program\ Files/Wireshark/tshark.exe
% eval exec $tsharkPath $sTsharkCmd > packet.log
% type packet.log
20.0.0.5
%
I am just curious. Is there any other option to handle this scenario ? Same problem observed with {*} also.
Your problem is that you're wanting space to be a part of an argument taken from variable, but to separate arguments coming from another argument. Tcl by default will try to apply the same uniform rules to all arguments: you need to do something extra to change that.
You want to use expansion of an argument. Your command
% eval exec $tsharkPath $sTsharkCmd > packet.log
Should be written as:
% exec $tsharkPath {*}$sTsharkCmd > packet.log
See how it is shorter? It's safer too. Try to avoid eval unless you really are doing something very complicated. (I hardly ever need it in my code…)
If you're stuck with an ancient version of Tcl without {*}, you need to do this instead:
% eval [linsert $sTsharkCmd 0 exec $tsharkPath > packet.log]
Which is horribly non-obvious, or this:
% eval {exec $tsharkPath} $sTsharkCmd {> packet.log}
Which isn't too nice either (and it's easy to forget to get it quoted right).
eval [list exec $tsharkPath {*}$sTsharkCmd]
Make the new command to evaluate with the sTsharkCmd attributes separated...

could not able to spawn(ssh) using expect

while executing
$expect filename.exp user pwd ip
I got the error
could not execute "ssh -l user ip": no such file or directory
The contents of filename.exp are this:
#!/opt/sfw/bin/expect -D
set OMC_Usr [lindex $argv 0]
set OMC_Pwd [lindex $argv 1]
set OMC_IP [lindex $argv 2]
set cmd "ssh -l $OMC_Usr $OMC_IP"
spawn $cmd
expect "Password:"
send "$OMC_Pwd\r"
interact
help me to proceed
Thanks...
The problem is that spawn needs the command and its arguments passed to it as multiple Tcl arguments, just like the standard Tcl exec command. The fix is to do this directly:
spawn ssh -l $OMC_Usr $OMC_IP
Or, if you prefer (and you've got Tcl 8.5):
set cmd "ssh -l $OMC_Usr $OMC_IP"
spawn {*}$cmd
Or, if you've got 8.4 or before:
eval spawn [lrange $cmd 0 end]
But don't do this:
eval spawn $cmd
because that will break unexpectedly if you have a Tcl metacharacter in the username (or IP address, but that's very unlikely).
Of course, the real fix is to set up an RSA keypair and use ssh-agent to manage it. Like that, you won't need to pass passwords on any command line; that's important because the command line of a process is public information about the process. Really. You can find it out with something trivial like ps -efww (or the equivalent for your OS). Environment variables are just as insecure too; there's an option to ps to show them too.