How can I edit remove excessive exact lines from expect script? - tcl

I have created an expect script using expect, and I realize that the expect -exact are constructed to match lines contain lots of unnecesary output. For example when I execute a postgresql restore command all the psql output gets included in the expect -exact line.
What other syntax can be used to trim the expect -exact lines? Can they be removed in some cases?

Tools that generate scripts by watching activity in a session are definitely prone to thinking far too much is significant. They simply do not understand what matters. Typically, the way to get rid of the unnecessarily-expected pieces is to just remove them with a text editor. The places to watch out for are where you're coming up to a send (or exp_send; there's a few variations); it's a good idea to always have an expect of something before a send, but what that should be needs some smarts on your part.
Later on, you may want to write scripts that can handle multiple conditions under which to take action by expecting several things at once. That's when you write things like this:
expect {
"abc" {
send "foo\r"
}
"def" {
send "bar\r"
}
}
There's a lot more complexity to automating access to an application than there appears to be at first because applications often are more complex than they appear to be at first.

Related

TCL stack trace not showing desired error line number

The -errorline element of the return options dictionary for the following TCL script is "2":
puts [info patchlevel]
try {
error "this is an error"
} trap {} {result ropts} {
puts $result
puts $ropts
}
How do I get the stacktrace to display the line number in the source file where the error was actually raised (ie. line 4 instead of 2) ?
Example screenshot:
Tcl often has that information available, but doesn't use it.
It has the information available because you have a chance to retrieve it with info frame and getbytecode (which is in the tcl::unsupported namespace, mostly because we reserve the right to change how the bytecodes themselves work at any time). I'm not quite sure if that would work in your specific case, but if you put your test code in a procedure then it definitely would. (There are complexities here with fragility that I don't fully understand.)
It doesn't use it because, for backward-compatibility with existing tooling, it uses the line numbers it was using prior to the creation of the machinery to support info frame. Those line numbers are relative to the local script fragment (which is whatever reports the line number in the error info trace first); in this case, that is the body of the try.
I don't like that it works like that at all. However, changing things is a bit tricky because we'd need to also figure out what else to report and what to do in the cases where the information genuinely isn't available (such as for automatically-generated code where things are assembled from many strings from many lines).

Trying to build a regex for if then statement in vscode extension syntax

Long time lurker, first time poster.
I'm trying to write a vscode syntax for TeraTerm (ttl), of which has been quite the learning experience, and have hit a few walls. Biggest problem I have is writing a regex for ttl's if/then/endif. Normally I enjoy the satisfaction of figuring things out for myself, but every time I mess up on this I end up locking things up pretty hard. ttl's format for if then is:
if expression then
statement
elseif expression then
statement
endif
A single if/then is easy to work out in tmlanguage, but putting a regex together for nested if/thens has resulted in several ctrl-alt-deletes. I've gotten pretty good at lookaheads/behinds within a single line, however multiple lines have been a royal pain. This is kinda how I've been doing it:
"if-endif":{
"patterns": [
{
"begin":"(?im)(?<ifthen>(^\\s*if\\s+(\\w+|\\s+|=)+\\s+then$))(?=(g<ifthen>).*^\\s*endif$)*.*^\\s*endif$)",
"beginCaptures":{"1":{"name":"keyword.control.ifthen.ttl"}},
"end":"(?i)(?<=(ditto above lookahead)*)(^\\s*endif$))",
"endCaptures":{"1":{"name":"keyword.control.endif.ttl"}}
"patterns":[{"include":"#if-endif"},{"include":"#alltheotherthings"}]
}
]
}
Somehow I need to get the if/then and its matching endif while not accidently grabbing an endif that belongs do a different if. My biggest issue is I really have no idea how to deal with multiple lines in a regex apart from including (?m).
My other problem I'm not sure is even possible within the syntax. I'd like a way to see if a variable has already been defined in the above script so I can markup appropriately if I'm trying to use a string/integer in the wrong place, or if it hasn't been defined. However I'm not aware of any way to compare one capture group to another inside a regex. If anyone knows of a good example or documentation I'd be overjoyed. My primary docs I've been using for this are MS's vscode stuff and macromates.
Any pointers/help/links would be greatly appreciated. This is pretty much the last regex holding me up as everything else are single line affairs.

TCL man page: it is better to place comment section way ahead

I know I am really picky here, but like to throw it out in case I am off in my interpreting the TCL man page, actually, I wish I was wrong here, as you see the below story.
So for every new TCL developer, we recommend reading the famous "11 rules" (now it is 12 rules).
Yesterday I was asked this question: why does the following script fail?
# puts "hello
world!"
Of course it fails, I said, the first line is taken as comment, that leaves world!" as a command.
But, the newbie said, the manpage indicates that the script is parsed in certain order:
As #2 Evaluation states, the command is parsed to words first.
As #4 Double quotes states, newline is taken as is in parsing double quotes. This makes hello and world! into one word, with a newline in between.
Comments at #10 does states everything up till the next newline is ignored, but after the above processing, the newline should be the 2nd newline, the one after world!.
I see he had a point.
It makes more sense to move the comment section way ahead in the man page, maybe at the second section. With this order change, it indicates that comment recognition is preceding the word-tokenizing process.
How do you think?
Again, I have no intention to ask for change of the manpage, just want to make sure if I miss anything in interpreting the bible.
[UPDATE]
To the people suggesting to close this question as not-a-technical question, it is the same as if my colleague came here asking why that script fails even though his understanding of TCL man page indicates it is a good script.
Again, I am not asking to change the man page.
Let me re-phrase my question - when you are asked this same question, what flaw do you see in his reasoning?
[UPDATE2]
Thanks Donal. I think this is what I learnt, TCL parser goes one char by another, there is no look-ahead.
This is another example:
puts [#haha]
Such script fails at tclsh for the same reason, TCL parser does not break down the script first and only parses the string embedded inside the matching brackets, instead it recognizes "#" as the start of comment and ignores everything after it.
The rules in the Tcl(n) manual page describe pretty precisely the parser that Tcl uses. Requests to change it substantively are usually denied as they tend to have far-reaching consequences and interact with each other trickily. Verifying that a reordering of the rules is not substantive is a non-trivial task, as they correspond to quite a bit of code (our parser and a chunk of our bytecode compiler).
Adding non-normative sections (e.g., EXAMPLES) is easier.
Update based on your updated question
The problem with the reasoning is that the rules are a whole, not really a layered set of parts. They do interact with each other. (The one that usually trips people up is the interaction between the brace rule and the comment rule when inside a braced string such as a procedure body.) Comments really are true comments, and extend up to the end of the line (allowing for backslash-newline sequences) but not beyond, but they only start at places where commands start, not at other places with a # character, and that's the genuinely tricky bit.
Unfortunately, the way that the Tcl parser works is a bit different to the way that programmers think, but most of the time it's pretty good at pretending to work in a “reasonable fashion”. The tricky edge cases don't actually come up too often other than when dealing with the brace-comment interaction mentioned above. The other cases which I hit tend to be either with a switch (resolvable by just putting the comment in the arm) or with long literal lists of things where I want to comment some sections of the list; in that latter case, I actually post-process the string before using it as a list.
set exampleList {
a b c
d e f
# Not really a comment but I want to use it like one!
g h i
j k l
}
# Convert “comment” lines to empty lines
regsub -all -line "^\\s*#.*$" $exampleList "" exampleList
The general advantage of Tcl's rules is that it is actually pretty easy to embed other languages within Tcl, precisely because Tcl only treats # (and other character) as special in well-defined contexts. As long as you can have the embedded language be one that uses balanced braces — and that's almost all of them in practice — then embedding it is utterly trivial. The other cases have to use backslashes and/or double quotes and are pretty ugly, but are also a minuscule fraction of all the embedding cases.
Your colleague's problem is that he's looking at the whole script in one go, whereas the Tcl parser handles one character at a time and doesn't do meaningful amounts of lookahead. It's just some dumb code.

Iterating over a string in Vimscript or Parse a JSON file

So I'm creating a vim script that needs to load and parse a JSON file into a local object graph. I searched and I couldn't find any native way to process a JSON file, and I don't want to add any dependencies to the script. So I wrote my own function to parse the JSON string (gotten from the file), but it's really slow. At the moment, I iterate through each character in the file like so:
let len = strlen(jsonString) - 1
let i = 0
while i < len
let c = strpart(jsonString, i, 1)
let i += 1
" A lot of code to process file....
" Note: I've tried short cutting the process by searching for enclosing double-quotes when I come across the initial double quotes (also taking into account escaping '\' character. It doesn't help
endwhile
I've also tried this method:
for c in split(jsonString, '\zs')
" Do a lot of parsing ....
endfor
For reference, a file with ~29,000 characters takes about 4 seconds to process, which is unacceptable.
Is there a better way to iterate over a string in vim script?
Or better yet, have I missed a native function to parse JSON?
Update:
I asked for no dependencies because I:
Didn't want to deal with them
Genuinely wanted some ideas for best way to do this without someone else's work.
Sometimes I just like to do things manually even though the problem has already been solved.
I'm not against plugins or dependencies at all, it's just that I'm curious. Thus the question.
I ended up creating my own function to parse the JSON file. I was creating a script that could parse the package.json file associated with node.js modules. Because of this, I could rely on a fairly consistent format and quit the processing whenever I'd retrieved the information I needed. This usually cut out large chunks of the file since most developers put the largest chunk of the file, their "readme" section, at the end. Because the package.json file is strictly defined, I left the process somewhat fragile. It assumed a root dictionary { } and actively looks for certain entries. You can find the script here: https://github.com/ahayman/vim-nodejs-complete/blob/master/after/ftplugin/javascript.vim#L33.
Of course, this doesn't answer my own question. It's only the solution to my unique problem. I'll wait a few days for new answers and pick the best one before the bounty ends (already set an alarm on my phone).
The simplest solution with the least dependencies is just using the json_decode vim function.
let dict = json_decode(jsonString)
Even though Vim's origin dates back a lot it happens that its internal string() eval() representation is that close to JSON that its likely to work unless you need special characters.
You can lookup the implementation here which even supports true/false/null if you want:
https://github.com/MarcWeber/vim-addon-json-encoding
Better use that library (vim-addon-manager allows to install dependencies easily).
Now it depends on your data whether this is good enough.
Now Benjamin Klein posted your question to vim_use which is why I'm replying.
Best and fast replies happen if you subscribe to the Vim mailinglist.
Goto vim.sf.net and follow the community link.
You cannot expect the Vim community to scrape stackoverflow.
I've added the keyword "json" and "parsing" to that little code that it can be found easier.
If this solution does not work for you you can try the many :h if_* bindings or write an external script which extracts the information you're looking for, or turns JSON into Vim's dictionary representation which can be read by eval() escaping special characters you care about correctly.
If you seek for completely correct solution omitting dependencies is one of the worst thing you can do. The eval() variant mentioned by #MarcWeber is one of the fastest, but it has its disadvantages:
Using solution for securing eval I mentioned in comment makes it no longer the fastest. In fact after you use this it makes eval() slower by more then an order of magnitude (0.02s vs 0.53s in my test).
It does not respect surrogate pairs.
It cannot be used to verify that you have correct JSON: it accepts some strings (e.g. "\<C-o>") that are not JSON strings and it allows trailing commas.
It fails to give normal error messages. It fails badly if you use vam#VerifyIsJSON I mentioned in p.1.
It fails to load floating point values like 1e10 (vim requires numbers to look like 1.0e10, but numbers like 1e10 are allowed: note “and/or” in the first paragraph).
. All of the above (except for the first) statements also apply to vim-addon-json-encoding mentioned by #MarcWeber because it uses eval. There are some other possibilities:
Fastest and the most correct is using python: pyeval('json.loads(vim.eval("varname"))'). Not faster then eval, but fastest among other possibilities. (0.04 in my test: approximately two times slower then eval())
Note that I use pyeval() here. If you want solution for vim version that lacks this functionality it will no longer be one of the fastest.
Use my json.vim plugin. It has an advantages of slightly better error reporting compared to failed vam#VerifyIsJSON, slightly worse compared to eval() and it correctly loads floating-point numbers. It can be used for verification of strings (it does not accept "\<C-a>"), but it loads lists with trailing comma just fine. It does not support surrogate pairs. It is also very slow: in the test I used (it uses 279702 character long strings) it takes 11.59s to load. Json.vim tries to use python if possible though.
For the best error reporting you can take yaml.vim and purge YAML support out of it leaving only JSON (I once have done the same thing for pyyaml, though in python: see markedjson library used in powerline: it is pyyaml minus YAML stuff plus classes with marks). But this variant is even slower then json.vim and should only be used if the main thing you need is error reporting: 207 seconds for loading the same 279702 character long string.
Note that the only variant mentioned that satisfies both requirements “no dependencies” and “no python” is eval(). If you are not fine with its disadvantages you have to throw away one or both of these requirements. Or copy-paste code. Though if you take speed into account only two candidates are left: eval() and python: if you want to parse json fast you really must use C and only these solutions spend most time in functions written in C.
Most other interpreters (ruby/perl/TCL) do not have pyeval() equivalent so they will be slower even if their JSON implementation is written in C. Some other (lua/racket (mzscheme)) have pyeval() equivalent, but e.g. luaeval('{}') is zero meaning that you will have to add additional step explicitly and recursively converting objects into vim dictionaries and lists (e.g. luaeval('vim.dict({})')) which will impact performance. Cannot say anything about mzeval(), but I have never heard about anybody actually using racket (mzscheme) with vim.

Is stdout necessary in tcl?

I'm very new to tcl and I hope to become proficient in this language, so I thought I should ask if there was a reason why some example codes have stdout in the code and some just use puts. The book doesn't seem to explain this. Was the IDE updated to automatically assume stdout?
The puts command writes to stdout if you don't provide it with a channel name (and has worked this way since… well, since forever). If you want, you can put it in or leave it out; it doesn't matter, but might make things clearer if done one way or the other. Let it be your choice. The only mandatory argument to puts is the string to write.
In performance terms, it's slightly faster to omit the value, but you should disregard that as the additional cost of a lookup is absolutely measly by comparison with the cost of doing IO at all. Clarity of expression is the only valid reason for picking one over the other.
There is a case where it matters though. If you want to write code that writes to standard output by default, but where you can override it to write to a file (or a socket or a pipe or …) then it's much easier to have the channel name stored in a variable (that is set to stdout by default, of course). Like that, all your code is the same except for one bit: it's not a good idea to close stdout normally. That's easy to avoid with code like this:
# Also don't want to close stdin or stderr...
if {![string match std* $channel]} {
close $channel
}