TCL API coverage : check if a TCL command have been called and tested exaustively in a test suite - tcl

Supposing I have a TCL API like :
namespaceXY::apiXY <value> -opt1 <value1> -opt2 <value2> -opt3 <value3>
This API is used (or maybe not) in a test suite (i.e thousands of tests).
How I can check if my API have been called + tested exhaustively (all options have been called/tested).
Many thanks

You can set an execution trace on the command. That way the signature of your command won't change. So you still get the same results if any code does info args namespaceXY::apiXY. Also error messages are not affected.
proc cmdtracer {cmd op} {
global cmdtracer
dict incr cmdtracer $cmd
}
trace add execution namespaceXY::apiXY enter cmdtracer
In the end you'll have a cmdtracer dict that contains the counts of each way the command was called. You will have to figure out yourself how to check if all options have been tested. There is not enough information in your question to provide suggestions for that part.

See #SchelteBron's answer for covering commands.
Exhaustively testing all options is going to be tricky, since they could potentially all interact in complex ways and some may be mutually-exclusive (think about the standard Tcl lsearch command for example). However, auditing that all options are at least called in your own commands can be done by additional audit-only probes. Checking all the sensible combinations of them is a manual task; you probably need a coverage tool for that.
Auditing Options in C Commands
Assuming that you're dealing with the case where you've got a C command that uses Tcl_GetIndexFromObj() to parse the option name (this is common and recommended) and where you don't mind having a threading hazard (also pretty common) the idea is simple. Make an integer variable (probably with file scope) in your C code, bind it to a Tcl variable with Tcl_LinkVar(), then use the resulting index from your (successful) Tcl_GetIndexFromObj() call to set a bit in that integer variable that says that the option was parsed.
#ifdef AUDIT_OPTIONS
static int foobar_optionTracker;
#endif
// in the implementation function, called FoobarImpl here for sake of argument
int index;
if (Tcl_GetIndexFromObj(interp, objPtr, optionNameTable, "option", 0, &index) != TCL_OK) {
return TCL_ERROR;
}
#ifdef AUDIT_OPTIONS
foobar_optionTracker |= 1 << index;
// Theoretically should call Tcl_UpdateLinkedVar() here, but for audit-only its not important
#endif
switch (index) {
// ...
}
// In your command registration function
Tcl_CreateObjCommand(interp, "foobar", FoobarImpl, NULL, NULL);
#ifdef AUDIT_OPTIONS
Tcl_LinkVar(interp, "optionTracker(foobar)", (void*) &foobar_optionTracker, TCL_LINK_INT);
#endif
With that in place, you can just read the array element optionTracker(foobar) from your Tcl test control code to see what options have been parsed (assuming you're happy with a bit-mask) in the foobar command since the last time the mask was reset. You reset the mask by just writing 0 to it.
Note that there's also Tcl_GetIndexFromObjStruct() in the C API, but auditing coverage of that is not significantly different from above.
Auditing Options in Tcl Commands
The equivalent of Tcl_GetIndexFromObj() in pure Tcl code is tcl::prefix match, but that doesn't return an index. Instead it returns the full option name that you can use with switch. Auditing that is most easily done with a full array. (This is morally the same as what the version for the C code does, but adapted to work with the optimal tools in a particular language.)
proc foobar {mandatoryArgument1 mandatoryArgument2 args} {
# Parse other things here, set up the TABLE of option descriptors, etc.
foreach option $args {
set option [tcl::prefix match $TABLE $option]
if {$::DoAudit} {
set ::foobarAudit($option) 1
}
switch -- $option {
# etc...
}
}
You can use things like array size foobarAudit to count the number of options actually used, or parray foobarAudit to print out what was actually used.

Related

Chisel randomly initialize register value when simulating with verilator

I'm using Chisel and blackbox to run my chisel logic against a verilog register file.
The registerfile does not have reset signal so I expect the register to be randomly initialized.
I passed the --x-initial unique to verilator,
Basically this is how I launch the test:
private val backendName = "verilator"
"NOCDMA" should s" do blkwrite and blkread correctly (with $backendName)" in {
Driver.execute(Array("--fint-write-vcd","--backend-name",s"$backendName",
"--more-vcs-flags","--trace-depth 1 --x-initial unique"),
()=>new DMANetworkWithMem(memAddrWidth,memDataWidth)(nocDataWidth)(nNodesX,nNodesY)){
c => new DMANetworkRWTest(c)
}
}
But The data I read from the register file is all zero before I wrote anything to it.
The read data is correct after I wrote to it.
So, is there anything inside chisel that I need to tune or I did not do everything properly ?
Any suggestions?
I'm not certain, but I found the following issue on Verilator with a similar issue: https://github.com/verilator/verilator/issues/1399.
From skimming the above issue, I think you also need to pass +verilator+seed+<value> and +verilator+rand+reset+<value> at runtime. I am not an expert in the iotesters, but I believe you can add these runtime values through the iotesters argument: --more-vcs-c-flags.
Side note, I would also set --x-assign unique in Verilator if there are cases in the Verilog where runtime would otherwise inject an X (eg. out-of-bounds index).
I hope this helps!

Adding help to the user defined functions in TCL

how can i add some kind of help to the user defined functions in TCL
Supposing if i have a function called runtest {ip_address test_time},
How to describe what the test or procedure is about in the TCL_shell?
How can i specify the information to the user, if he types in function_name --help in the TCL shell the user should be able to know what the function does and what exactly are the parameters.
how can i do this?
While it is true that it is not part of the language, it is fairly easy to implement something that adds this functionality to pre-existing functions. The only caveat being that the function then cannot take the string --help as a valid argument since that argument will trigger the feature.
Here's one simple implementation:
# Lets call the feature "document". As in, add documentation:
proc document {procname text} {
rename $procname __$procname
proc $procname args [string map [list %TEXT% $text %PROC% $procname] {
if {$args == "--help"} {
puts {%TEXT%}
} else {
set script [linsert $args 0 __%PROC%]
return [uplevel 1 $script]
}
}]
}
What this does is to override the function (by renaming and then declaring another function of the same name) and see if the function is called with the argument --help. If it is it prints the documentation otherwise it executes the original function. Just be careful not to call this twice on the same function (it can be modified for it to work though).
So you can do things like:
proc foo {} {puts 2}
document foo {Prints the number 2.}
Now if you call:
foo --help
and it would output:
Prints the number 2.
You don't have to touch the existing procedures:
proc help {procname} {
puts $::helptext($procname)
}
proc addhelp {procname text} {
set ::helptext($procname) $text
}
addhelp foo "this is the help text for procedure foo"
help foo
Without redefining the proc command, you cannot. Ie, that functionality is not built into the language, but would be possible to add if you so wished.
I will note that adding the capability yourself, while possible, is probably beyond the difficulty where it's worth doing, especially for someone that isn't intimately familiar with the language. That being said, you can check the tclers wiki for a preexisting implementation.
I tend to prefer to put help in a separate file (e.g., as HTML) which lets me browse it in another window. There are so many ways you can do that while still keeping the docs with the code, such as through doxygen.
There are also a number of ways of doing interactive documentation, several of which are described in the Tcler's Wiki; it would seem to me that some of the techniques mentioned on that page (together with #slebetman's answer) would give you what you wanted. (I think it would be easier to have a separate help command as that would avoid having the help syntax interfere with the command, but that's your call.)

Tcl info exists

I have a curious case of Tcl that perhaps I just don't understand.
The following code is done at the top level (not inside of any procedure):
if {![info exists g_log_file_name]} {
set g_log_file_name "default.txt"
}
What I hope it would do is to declare a global variable with some value if it wasn't declared yet (which can be done at some other script or C application). However, the if statement always false. I ran on Tcl 7.4.
What may be the problem?
Thank you so much.
% info level
0
% info exists g_log_file_name
0
% set g_log_file_name whatever
whatever
% info exists g_log_file_name
1
Hence the reason you observe is probably because the variable is really always unset at the time your if command is executed.
Possible reasons for this I can imagine are:
It's just not set: literally, no code attempt to do this;
The external code sets some other variable: name mismatch;
The external code sets a variable in some other interpreter: in a C code embedding Tcl, there can be any number of Tcl interpreters active at any moment (and those are free to create child interpreters as well);
I'm not sure abot the long forgotten version of Tcl you have at hand, but 8.x has the trace command which can be used to log accesses to a particular variable—you could try to use it to see what happens.

Tcl and records (structs)

Package struct::record from TCLLIB provides means for emulating record types. But record instances are commands in the current namespace and not variables in the current scope. This means there is no garbage collection for record instances. Passing name of the record instance to a procedure means passing it by reference not by value, it is possible to pass string representation of the record as parameter but it requires to create another instance in the procedure, configure it and delete by hand, it's annoying. I wonder about the rationale behind this design. A simple alternative is provide a lisp-style records - a set of construction, access and modification procedures and represent records as lists.
The struct::record implementation is, from my viewpoint, an oo-style implementation. If you're searching for a data-style implementation (like lisp) where the commands are totally separate from the data, you might want to look at the dict command.
I'll note that oo-style and data-style are really not good descriptions, but they were the best I could think of offhand.
You most certainly can do it “the Lisp way”.
proc mkFooBarRecord {foo bar} {
# Keep index #0 for a "type" for easier debugging
return [list "fooBarRecord" $foo $bar]
}
proc getFoo {fooBarRecord} {
if {[lindex $fooBarRecord 0] ne "fooBarRecord"} {error "not fooBarRecord"}
return [lindex $fooBarRecord 1]
}
# Etc.
That works quite well. (Write it in C and you can make it more efficient too.) Mind you, as a generic data structure, it seems that many people prefer Tcl 8.5's dictionaries. There are many ways to use them; here's one:
proc mkFooBarRecord {foo bar} {
return [dict create "type" fooBarRecord "foo" $foo "bar" $bar]
}
proc getFoo {fooBarRecord} {
dict with fooBarRecord {
if {$type ne "fooBarRecord"} {error "not fooBarRecord"}
return $foo
}
}
As for the whole structures versus objects debate, Tcl tends to regard objects as state with operations (leading to a natural presentation as a command, a fairly heavyweight concept) whereas structures are pure values (and so lightweight). Having written a fair chunk on this, I really don't know what's best in general; I work on a case-by-case basis. If you are going with “structures”, also consider whether you should have collections that represent fields across many structures (equivalent to using column-wise storage instead of row-wise storage in a database) as that can lead to more efficient handling in some cases.
Also consider using a database; SQLite integrates extremely well with Tcl, is reasonably efficient, and supports in-memory databases if you don't want to futz around with disk files.
I will not answer your question, because I was not been using Tcl for many years and I never use this kind of struct, but I can give you the path to two possible places that are very plausible to provide a good answer for you:
The Tcl'ers Wiki http://wiki.tcl.tk
The Frenode's Tcl IRC channel
At the time I used Tcl they proved to be invaluable resources.

Accessing the Body of a Function with Lua

I'm going back to the basics here but in Lua, you can define a table like so:
myTable = {}
myTable [1] = 12
Printing the table reference itself brings back a pointer to it. To access its elements you need to specify an index (i.e. exactly like you would an array)
print(myTable ) --prints pointer
print(myTable[1]) --prints 12
Now functions are a different story. You can define and print a function like so:
myFunc = function() local x = 14 end --Defined function
print(myFunc) --Printed pointer to function
Is there a way to access the body of a defined function. I am trying to put together a small code visualizer and would like to 'seed' a given function with special functions/variables to allow a visualizer to 'hook' itself into the code, I would need to be able to redefine the function either from a variable or a string.
There is no way to get access to body source code of given function in plain Lua. Source code is thrown away after compilation to byte-code.
Note BTW that function may be defined in run-time with loadstring-like facility.
Partial solutions are possible — depending on what you actually want to achieve.
You may get source code position from the debug library — if debug library is enabled and debug symbols are not stripped from the bytecode. After that you may load actual source file and extract code from there.
You may decorate functions you're interested in manually with required metadata. Note that functions in Lua are valid table keys, so you may create a function-to-metadata table. You would want to make this table weak-keyed, so it would not prevent functions from being collected by GC.
If you would need a solution for analyzing Lua code, take a look at Metalua.
Check out Lua Introspective Facilities in the debugging library.
The main introspective function in the
debug library is the debug.getinfo
function. Its first parameter may be a
function or a stack level. When you
call debug.getinfo(foo) for some
function foo, you get a table with
some data about that function. The
table may have the following fields:
The field you would want is func I think.
Using the debug library is your only bet. Using that, you can get either the string (if the function is defined in a chunk that was loaded with 'loadstring') or the name of the file in which the function was defined; together with the line-numbers at which the function definition starts and ends. See the documentation.
Here at my current job we have patched Lua so that it even gives you the column numbers for the start and end of the function, so you can get the function source using that. The patch is not very difficult to reproduce, but I don't think I'll be allowed to post it here :-(
You could accomplish this by creating an environment for each function (see setfenv) and using global (versus local) variables. Variables created in the function would then appear in the environment table after the function is executed.
env = {}
myFunc = function() x = 14 end
setfenv(myFunc, env)
myFunc()
print(myFunc) -- prints pointer
print(env.x) -- prints 14
Alternatively, you could make use of the Debug Library:
> myFunc = function() local x = 14 ; debug.debug() end
> myFunc()
> lua_debug> _, x = debug.getlocal(3, 1)
> lua_debug> print(x) -- prints 14
It would probably be more useful to you to retrieve the local variables with a hook function instead of explicitly entering debug mode (i.e. adding the debug.debug() call)
There is also a Debug Interface in the Lua C API.