Robot framework: Retrieving keys that contain dashes from a dict - json

I am new to Robot Framework and am trying to validate the contents of some JSON that is returned from a web service. The problem is that some attributes of the json objects have dashes in them and Robot doesn't seem to like this. I have something like the following
&{deployment} = list deployment ${deployment_name}
&{changeSets} = Set Variable ${deployment.ChangeSets}
&{myChangeSet} = Set Variable ${changeSets.my-change-set}
Should Be True ${myChangeSet.UseLocal}
Should Be Equal As Strings ${myChangeSet.Version} ${update_version}
But Robot fails on the 3rd line with the following error:
Resolving variable '${changeSets.my-change-set}' failed: AttributeError: my
I tried to escape the dashes but that still doesn't seem to work:
Resolving variable '${changeSets.my\-change\-set}' failed: SyntaxError: unexpected character after line continuation character (<string>, line 1)
I can't seem to find any information in the Robot docs with other ways to retrieve dict keys outside of the dot-notation. Any suggestions?

The use of dot notation is just a convenience. You can still access them the normal way (documented in the dictionary variables section of the user guide as &{NAME}[key]):
&{changeSets}[my-change-set]
Or, with extended variable syntax, which treats everything inside {} as a python expression:
${changeSets['my-change-set']}
Here is a working example illustrating these two methods:
*** Variables ***
&{changeSets} my-change-set=foo
*** Test Cases ***
Test 1
should be equal ${changeSets['my-change-set']} foo
Test 2
should be equal &{changeSets}[my-change-set] foo

Related

As far as I can tell, there is no way to parameterize character strings in an AllenNLP config file --- only ints or floats

So the issue is that, for using autotuning (like optuna) with AllenNLP, the suggested practice is to use, in jsonnet scripts, references to environment variables, and then to set up a study to modify those parameters.
That works fine, when the values are integers or floating points. For integers, you use std.parseInt(std.extVar(varname)), for floating point numbers, you use std.parseJson(std.extVar(varname)).
But if I want to change, say the optimization technique in my tests between "adam", "sparseadam", "adamax", adamw", etc. or change the type of RNN I am using, there does not appear to be an easy way to do that.
It would seem that you should be able to do std.extVar(varname) in that case without wrapping it inside a parseJson() or parseInt(), but that returns an error. Has anybody else had that problem and how did you get around it?
Just to add to this, I am trying this with three different string parameters. Here is the jsonnet for the first one, "bert_vocab":
local bert_vocab=std.extvar('bert_vocab');
Error message:
486 ext_vars = {**_environment_variables(), **ext_vars}
487
--> 488 file_dict = json.loads(evaluate_file(params_file, ext_vars=ext_vars))
489
490 if isinstance(params_overrides, dict):
RuntimeError: RUNTIME ERROR: field does not exist: extvar
/bigdisk/lax/cox/jupyter/bert_config.jsonnet:28:18-28 thunk <bert_vocab>
/bigdisk/lax/cox/jupyter/bert_config.jsonnet:61:22-32 object <anonymous>
/bigdisk/lax/cox/jupyter/bert_config.jsonnet:(59:16)-(63:12) object <anonymous>
/bigdisk/lax/cox/jupyter/bert_config.jsonnet:(58:21)-(64:10) object <anonymous>
/bigdisk/lax/cox/jupyter/bert_config.jsonnet:(56:19)-(65:8) object <anonymous>
During manifestation
I also tried various "string escaping functions" like here (but none of the string escaping functions work either:
local bert_vocab=std.escapeStringBash(std.extvar("bert_vocab"));
I can do the following to verify that the os environment variable is set:
os.environ['bert_vocab'] returns 'bert-base-uncased'
(I don't know AllenNLP, only Jsonnet.)
ExtVars can be arbitrary Jsonnet values (numbers, floats, strings, arrays, objects, functions or nulls).
Judging from the examples you brought up, AllenNLP passes the parameters as strings and you need to do the parsing. In such case it should be possible to just use "naked" std.ExtVar to get a string.

Why can't we run Get method of requests vs RequestsLibrary Library in Robot Framework together

Here is code:
*** Settings ***
Library Collections
Library RequestsLibrary
Library requests
# Declare Test case
*** Test Cases ***
Get Requests2
${resp2}= Get http://services.groupkt.com/country/get/iso2code/IN
${convert_to_json2}= set variable ${resp2.json()}
log to console ${convert_to_json2}
it displays the follwing error Multiple keywords with name 'Get' found. Give the full name of the keyword you want to use: RequestsLibrary.Get requests.Get`
The libraries likely both contain the same keyword, or rather they both have a keyword that use the same name. In order to identify which keyword you want to call, Robot Framework needs to have only unique names in it's keywords namespace.
As the error suggests, you can prefix the library's name before the keyword name, to allow Robot Framework to differentiate the two keywords:
*** Test Cases ***
Get Requests2
${resp2}= RequestsLibrary.Get http://services.groupkt.com/country/get/iso2code/IN
${convert_to_json2}= set variable ${resp2.json()}
log to console ${convert_to_json2}
Or,
*** Test Cases ***
Get Requests2
${resp2}= requests.Get http://services.groupkt.com/country/get/iso2code/IN
${convert_to_json2}= set variable ${resp2.json()}
log to console ${convert_to_json2}

Badly Formed hexadecimal uuid string error in Django fixture; json uuid conversion fails issue

File "/home/malikarumi/Projects/cannon/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 2390, in get_db_prep_value
value = uuid.UUID(value)
File "/usr/lib/python2.7/uuid.py", line 134, in __init__
raise ValueError('badly formed hexadecimal UUID string')
ValueError: Problem installing fixture '/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json': badly formed hexadecimal UUID string
I've found the following links so far:
https://github.com/dcramer/django-uuidfield/issues/40
https://github.com/dcramer/django-uuidfield/commit/caae1bc4e45445a06dd11bb22da6a9f07395f78a
Django UUIDField modelfield causes error in Django admin: badly formed hexadecimal UUID string
Django Primary Key: badly formed hexadecimal UUID string
I counted my uuidfield value. It is len=36, because it has dashes in it. At least the string representation I can see is that way. So I replaced it with the same alphanumeric without dashes, as suggested as a test by the bugfix, but I still got the same result.
I checked the model, but there is no max length on any uuid field, nor on the fk link back to the uuid. There's nothing on the fk to suggest it is, or should be limited to, chars, ints, uuids, etc.
Then I found this: http://arthurpemberton.com/2015/04/fixing-uuid-is-not-json-serializable which I hacked into /python2.7/site-packages/django/core/serializers/python.py. The blogger had put it into models.py. But I got the same error, before realizing it was NOT coming from serializers/python.py, as it was yesterday, but from /usr/lib/python2.7/uuid.py, line 134, in init. the relevant portions of that code are:
if hex is not None:
hex = hex.replace('urn:', '').replace('uuid:', '')
hex = hex.strip('{}').replace('-', '')
if len(hex) != 32:
raise ValueError('badly formed hexadecimal UUID string')
int = long(hex, 16)
Rather than try to hack more core code, given that the indication is the problem is json, not Python, I left this alone for now.
Finally, I looked at this:
https://code.djangoproject.com/ticket/24012
It is stated a couple of times here that Django's "UUIDField generates UUIDs in Python". Now here is some history. I created one row, a single instance of Model A into Django with a fixture that had no uuid and no datefield and had no issues. (The uuidfield is on an abstract model, so it is created when the object is created). I did that because I needed the uuid of that Model A instance for a fk field in Model B, which is the one I am struggling with now. I did that by copy pasting the Model A uuid into the fk field on Model B in a csv file which I then converted to json in order to use it as a fixture.
Is it possible that the uuid ran into problems in this copy paste maneuver, before the conversion to json?
If not, that means even though it was an acceptable Python object when it was created, going thru the json conversion messed it up, correct?
If that's the case, what is a workaround?
Can the Arthur Pemberton code be made to work somewhere else in this process?
If I leave the uuid off, I can probably make this work, but then I have to go back and put the all the fk uuid's in manually. Is there a better solution? Maybe a bulk insert of that field alone?
This may be a recurring issue for me, because I am also using Scrapy, which supports but does not require json. None of my scraped items will come with uuid, but how do I automate adding their fk's into my process in order to get them into Django?
Or is all of this a good reason to forget uuids altogether?
Thanks.
EDIT/UPDATE per #rolf:
Since I just discovered that the django shell differs more than I realized (the shell can find settings, the regular interpreter can't) I decided to run this once in each one, but the results were the same.
(cannon)malikarumi#Tetuoan2:~/Projects/cannon/jamf$ python manage.py shell
Python 2.7.10 (default, Oct 14 2015, 16:09:02)
IPython 4.0.3 -- An enhanced Interactive Python.
In [1]: uuid.UUID(a82857b6-e336-4c6c-8499-47601770b39d)
File "<ipython-input-1-e282858da374>", line 1
uuid.UUID(a82857b6-e336-4c6c-8499-47601770b39d)
^
SyntaxError: invalid syntax
In [2]: uuid.UUID(a0a69415-6627-43db-8c7a-b57d0c4cefe2)
File "<ipython-input-2-befebf1573ba>", line 1
uuid.UUID(a0a69415-6627-43db-8c7a-b57d0c4cefe2)
^
SyntaxError: invalid syntax
In [3]: uuid.UUID(e6e11b06-ea3b-4e98-a31f-9a83447ad884)
File "<ipython-input-3-a59ea095e61a>", line 1
uuid.UUID(e6e11b06-ea3b-4e98-a31f-9a83447ad884)
^
SyntaxError: invalid syntax
In [4]: uuid.UUID(bd116432-65d7-4612-abfe-9a99dcaf5cad)
File "<ipython-input-4-c4a04434aa3c>", line 1
uuid.UUID(bd116432-65d7-4612-abfe-9a99dcaf5cad)
^
SyntaxError: invalid syntax
Now that I have posted this, I notice that even Stack Overflow treats these uuid differently, i.e., the way they are colored, if that's relevant and meaningful here.
But now that we know this, what do we do with / about it?
2nd Update
This morning I thought, what about a uuid that had never been anywhere but in Django? So here's what I did:
In [5]: e.uuid
Out[5]: UUID('61877565-5fe5-4175-9f2b-d24704df0b74')
In [6]: uuid.UUID(61877565-5fe5-4175-9f2b-d24704df0b74)
File "<ipython-input-6-56137f5f4eb6>", line 1
uuid.UUID(61877565-5fe5-4175-9f2b-d24704df0b74)
^
SyntaxError: invalid syntax
In [7]: uuid.UUID('61877565-5fe5-4175-9f2b-d24704df0b74')
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-7-3b4d3e5bd156> in <module>()
----> 1 uuid.UUID('61877565-5fe5-4175-9f2b-d24704df0b74')
NameError: name 'uuid' is not defined
This is apparently because I left the quote around the alphanumeric, but why that would generate a uuid not defined error, instead of 'string type' or some such error is beyond me.
In [8]: uuid.UUID(61877565-5fe5-4175-9f2b-d24704df0b74)
File "<ipython-input-8-56137f5f4eb6>", line 1
uuid.UUID(61877565-5fe5-4175-9f2b-d24704df0b74)
^
SyntaxError: invalid syntax
The first time I keyed in the characters by hand. I decided to repeat the test by copying and pasting, but as you can see, it made no difference. If there was something weird about the way only the 5 that the caret is pointing to was generated, we might be on to something, but if so, why do I get the same error in the same place when I typed it in by hand myself?
This no longer seems like a json issue to me, since – as far as I know – json has never touched this uuid, unless it did somehow in the internal workings of Django.
Instead, there is either
1. something wrong with the way uuid.UUID generates uuids, or
2. the way it generates them on my system, (Ubuntu 15.10, Django 1.9.1, Python 2.7.10) or
3. the way it reads and evaluates them when they come back, like in uuid.UUID() or being input outside the internal, automatic uuid generation process.
But that also means people using uuid.UUID() to generate uuids will never know there is an issue unless they do what I did, which is try to bring them in from outside. I remember reading somewhere that all uuids are supposed to be compatible. So, unless someone here has a better insight, I think we might be up for a bug report. But is it a Python bug, a Django bug, or both?
Your syntax is wrong:
uuid.UUID('61877565-5fe5-4175-9f2b-d24704df0b74') # note the quotes

How long is a line after a readline(fh, line) call?

I have written a JSON parser in VHDL. The parser core uses two nested loops:
1. loop over all lines until EOF
2. loop over every char until line of end
For clearance: Its not a hardware parser. the parser used to read synthesis settings at synthesis time to configure instantiated entities like a baudrate in a UART module.)
The inner loop looks like this: loopj : for j in CurrentLine.all'range loop
Source: JSON.pkg.vhdl
This code works in XST 14.7, iSim 14.7 and GHDL, but not in Vivado. Vivado does not support .all. The error message is this one:
ERROR: [Synth 8-27] access type dereference not supported [D:/git/GitHub/JSON-for-VHDL/vhdl/JSON.pkg.vhdl:293]
Updated code, due to the hint from kraigher:
#Paebbles Have you tried foo'range instead of foo.all'range? I think I remember that it should implicitly work. - kraigher
I tried it before, but got an error. Maybe this error was related to another one. Now its working. So my current loopj line looks like this:
loopj : for j in CurrentLine'range loop
This line works fine in XST, iSim, GHDL and QuestaSim, but Vivado still has problems:
INFO: [Synth 8-638] synthesizing module 'Boards2' [.../Boards2.vhdl:16]
ERROR: [Synth 8-278] expression 0 out of range [.../JSON.pkg.vhdl:293]
ERROR: [Synth 8-421] mismatched array sizes in rhs and lhs of assignment [.../Boards2.vhdl:20]
ERROR: [Synth 8-285] failed synthesizing module 'Boards2' [.../Boards2.vhdl:16]
How can a expression be out of range? This message is very strange.
Is there another way to get a range for a loop, depending on how long the current line is?

How can I assign sequences to constants in the CONSTANTS section of a TLA+ configuration file?

I've tried
CONSTANTS seq = <<5,6,7>>
but TLC gives me a syntax error:
Error: TLC found an error in the
configuration file at line 1. It was
expecting = or <- and didn't find it.
I've also tried to include the Sequences module in the configuration file, to no avail.
So... what do I have to do to assign a sequence?
I don't remember ever facing this problem and my TLC is too rusty to try and give you a first hand answer (I just restarted using the TLA+ toolbox).
From the post linked bellow, however, I figure that you can't instantiate constants with sequences through the TLC config files.
http://bbpress.tlaplus.net/topic/creating-a-sequence-from-a-set
Even though the question is old, leaving an answer may help future TLAers.
You can't directly assign to a constant in the TLA+ file. If you're using the toolbox, write CONSTANTS seq, then in the model add the tuple you want as an ordinary assignment.
So it turns out that you need a separate .tla file for that. Suppose you have "Main.tla" as you source file. You want MyConst to have value <<1,2,3>>. TLA+ toolbox generates MC.tla where it puts the constants:
---- MODULE MC ----
EXTENDS Main, TLC
const_uniq12345 = <<1,2,3>>
====
in MC.cfg, there will be the line
CONSTANT MyConst <- const_uniq12345
Note that MyConst = const_uniq12345 won't work -- if you use = instead of <-, MyConst will contain model value const_uniq12345 instead of <<1, 2, 3>>
I used https://github.com/hwayne/tlacli to be able to run TLC from command line (TLA toolbox has awful UX), and I was able to supply a tuple constant with help of extra .tla file like this. I used meaningful name instead of const_uniq12345 too, of course. Had to call java -cp /path/to/tla2tools.jar ... manually, though (got the full command using --show-script option of tlacli), because currently tlacli doesn't handle <- in the config.