Elixir - Capitalized keys in structs - json

I am trying to write a CLI client in Elixir for an API so that I can login to the API system, fetch the data I need for my calculation and then logout. I have defined a Packet.Login struct that supposed to be my internal data structure that I end up with after parsing the JSON I receive.
I am using Poison to parse the JSON. The problem is that it seems like, because of the API returning capitalised properties, I can't match them when printing or parsing, as Poison will return a map with these capitalized keys. The problem is that it seems impossible for me to use the alias like this. If I try to use another syntax,
packet[:Token]
it still does not work and instead gives me an error. But this time about Packet.Login not implementing the Access behaviour. I can understand that part, but not the first issue. And I'm trying to keep the code stupid simple.
defmodule Packet.Login do
defstruct [:Data, :Token]
end
defimpl String.Chars, for: Packet.Login do
def to_string(packet) do
"Packet:\n---Token:\t\t#{packet.Token}\n---Data:\t#{packet.Data}"
end
end
loginPacket = Poison.decode!(json, as: %Packet.Login{})
IO.puts "#{loginPacket}"
When trying to compile the above I get this:
** (CompileError) lib/packet.ex:31: invalid alias: "packet.Token". If you wanted to define an alias, an alias must expand to an atom at compile time but it did not, you may use Module.concat/2 to build it at runtime. If instead you wanted to invoke a function or access a field, wrap the function or field name in double quotes
(elixir) expanding macro: Kernel.to_string/1
Is there a way for me to fix this somehow? I have thought of parsing the map and de-capitalizing all fields first, but I would rather not.
Why can't I have capitalized keys for a struct? It seems like I can though, as long as I don't try to use them.

In order to access a field of a map which is an atom starting with an uppercase letter, you need to either put the key in quotes, e.g. foo."Bar" or use the bracket syntax, e.g. foo[:Bar]. foo.Bar in Elixir is parsed as an alias. With structs, you cannot use the bracket syntax, so the easiest way is to use quotes around the field name. In your code, you'll therefore need to change:
"Packet:\n---Token:\t\t#{packet.Token}\n---Data:\t#{packet.Data}"
to:
"Packet:\n---Token:\t\t#{packet."Token"}\n---Data:\t#{packet."Data"}"
I could not find this documented clearly anywhere but Elixir's source mentions this in some places and also uses this syntax to access some functions in :erlang which have names that are not valid identifiers in Elixir, e.g. :erlang."=<".
Fun fact: you can define functions in Elixir that can only be called with this quote syntax as well:
iex(1)> defmodule Foo do
...(1)> def unquote(:"!##")(), do: :ok
...(1)> end
iex(2)> Foo."!##"()
:ok

Related

How to marshal a predicate from JSON in Prolog?

In Python it is common to marshal objects from JSON. I am seeking similar functionality in Prolog, either swi-prolog or scryer.
For instance, if we have JSON stating
{'predicate':
{'mortal(X)', ':-', 'human(X)'}
}
I'm hoping to find something like load_predicates(j) and have that data immediately consulted. A version of json.dumps() and loads() would also be extremely useful.
EDIT: For clarity, this will allow interoperability with client applications which will be collecting rules from users. That application is probably not in Prolog, but something like React.js.
I agree with the commenters that it would be easier to convert the JSON data to a .pl file in the proper format first and then load that.
However, you can load the predicates from JSON directly, convert them to a representation that Prolog understands, and use assertz to add them to the knowledge base.
If indeed the data contains all the syntax needed for a predicate (as is the case in the example data in the question) then converting the representation is fairly simple as you just need to concatenate the elements of the list into a string and then create a term out of the string. Note that this assumption skips step 2 in the first comment by Guy Coder.
Note that the Prolog JSON library is rather strict in which format it accepts: only double quotes are valid as string delimiters, and lists with singleton values (i.e., not key-value pairs) need to use the notation [a,b,c] instead of {a,b,c}. So first the example data needs to be rewritten:
{"predicate":
["mortal(X)", ":-", "human(X)"]
}
Then you can load it in SWI-Prolog. Minimal working example:
:- use_module(library(http/json)).
% example fact for testing
human(aristotle).
load_predicate(J) :-
% open the file
open(J, read, JSONstream, []),
% parse the JSON data
json_read(JSONstream, json(L)),
% check for an occurrence of the predicate key with value L2
member(predicate=L2, L),
% concatenate the list into a string
atomics_to_string(L2, S),
% create a term from the string
term_string(T, S),
% add to knowledge base
assertz(T).
Example run:
?- consult('mwe.pl').
true.
?- load_predicate('example_predicate.json').
true.
?- mortal(X).
X = aristotle.
Detailed explanation:
The predicate json_read stores the data in the following form:
json([predicate=['mortal(X)', :-, 'human(X)']])
This is a list inside a json term with one element for each key-value pair. The element has the syntax key=value. In the call to json_read you can already strip the json() term and store the list directly in the variable L.
Then member/2 is used to search for the compound term predicate=L2. If you have more than one predicate in the JSON file then you should turn this into a foreach or in a recursive call to process all predicates in the list.
Since the list L2 already contains a syntactically well-formed Prolog predicate it can just be concatenated, turned into a term using term_string/2 and asserted. Note that in case the predicate is not yet in the required format, you can construct a predicate out of the various pieces using built-in predicate manipulation functionality, see https://www.swi-prolog.org/pldoc/doc_for?object=copy_predicate_clauses/2 for some pointers.

Regex for replacing unnecessary quotation marks within a JSON object containing an array

I am currently trying to format a JSON object using LabVIEW and have ran into the issue where it adds additional quotation marks invalidating my JSON formatting. I have not found a way around this so I thought just formatting the string manually would be enough.
Here is the JSON object that I have:
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":"["cat","dog","bird"]",
"count":3
}
}
Here is the JSON object I want with the quotation marks removed.
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":["cat","dog","bird"],
"count":3
}
}
I am still not an expert with regex and using a regex tester I was only able to grab the "objects" and "count" fields but I would still feel I would have to utilize substrings to remove the quotation marks.
Example I am using (would use a "count" to find the start of the next field and work backwards from there)
"([objects]*)"
Additionally, all the other Regex I have been looking at removes all instances of quotation marks whereas I only need a specific area trimmed. Thus, I feel that a specific regex replace would be a much more elegant solution.
If there is a better way to go about this I am happy to hear any suggestions!
Your question suggests that the built-in LabVIEW JSON tools are insufficient for your use case.
The built-in library converts LabVIEW clusters to JSON in a one-shot approach. Bundle all your data into a cluster and then convert it to JSON.
When it comes to parsing JSON, you use the path input terminal and the default type terminals to control what data is parsed from a JSON string.
If you need to handle JSON in a manner similar to say JavaScript, I would recommend something like the JSONText Toolkit which is free to use (and distribute) under the BSD licence. This allows more complex and iterative building of JSON strings from LabVIEW types and has text-path style element access along with many more features.
The Output controls from both my examples are identical - although JSONText provides a handy Pretty Print vi.
After using a regex from one of the comments, I ended up with this regex which allowed me to match the array itself.
(\[(?:"[^"]*"|[^"])+\])
I was able to split the the JSON string into before match, match and after match and removed the quotation marks from the end of 'before match' and start of 'after match' and concatenated the strings again to form a new output.

QuickSight parseJson with dots in key

In AWS QuickSight, I'm trying to create a Calculated Field based on some JSON data. The data has dots in some of the keys, e.g:
{"foo.bar": "baz"}
In order to parse this, I'd need something like the bracket notation, since the dot notation won't work (unless there's a way to escape the dot?):
parseJson({the_column}, '$["foo.bar"]')
Trying this out, I'm getting errors saying that the syntax is incorrect.
So, my question: What is the correct syntax to achieve this?
Note:
I can use replace to at least be able to parse it, e.g parseJson(replace({the_field}, "foo.bar", "whatever"), "$.whatever"). But this does not seem optimal, and it is prone to errors like accidentally replacing the wrong string.

Regex: Is it possible to do a substitution within a capture group?

I have this one line JSON text:
{"schemaText":{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"},"description":"Autogenerated by NiFi"}
As can be seen there is a property called "schemaText" that contains an object, I want to convert it to a string, so the 'only' thing I need to do is add quotes at the beginning and end of the property and escape the quotes inside.
Using the regular expression bellow (not that my regex knowledge is really low), I am able to do the first step:
({"schemaText":)(\{"fields":\[.*)(,"description.*)
Using the substitution
$1"$2"$3
gives the result:
{"schemaText":"{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"}","description":"Autogenerated by NiFi"}
But still remains to escape the quotes to get this:
{"schemaText":"{\"fields\":[{\"name\":\"AX_SND_TYPE\",\"type\":\"string\"},{\"name\":\"BWORK\",\"type\":\"int\"}],"name":"XXXSchema","type":"record"}","description":"Autogenerated by NiFi"}
That is have valid JSON format.
The question is: is there a way to escape the quotes inside $2 capture group in the same regular expression?
Thanks in advance.
The answer to your question is no, it's not possible. You're really trying to do two different, unrelated substitutions in a single regular expression. This is a feature that no regular expression engine supports.
Think about it: Your first requirement is for the engine to perform a substitution on the whole text (the quotes), and then, for your second requirement, the engine has to somehow backtrack and perform more substitutions on text which may or may not have already changed: e.g.: It would need to perform a new match on the already substituted text, which, depending on what the first substitution did, may not even exist anymore!
If, as you say, you already have an aproach that works, keep that. A single regular expression is simply not a good fit for what you are trying to do.
I'd recommend tackling this problem using code e.g. with vanilla JavaScript:
let json = '{"schemaText":{"fields":[{"name":"AX_SND_TYPE","type":"string"},{"name":"BWORK","type":"int"}],"name":"XXXSchema","type":"record"},"description":"Autogenerated by NiFi"}';
let obj = JSON.parse(json);
let schemaTextAsString = JSON.stringify(obj.schemaText)
obj.schemaText = schemaTextAsString
var result = JSON.stringify(obj)
You can see this working here.
Note that in your desired output you were not escaping the quotes in schemaText's name field, but this code does.
Finally whenever I use regular expressions I always think of this classic article "Regular Expressions: Now You Have Two Problems"!
Just for your information, you can actually match at every position where a substitution should occur, using an expression such as the following:
/({"schemaText":)|}(,"description")(.*)|([^"]*)"/g
The only issue, as others have mentioned, is that you want to do more than match; you want to perform a "conditional replacement" because there does not exist a single catch-all substitution that will cover all 3 cases you're dealing with (insert starting ", insert \ before quotes, and insert ending ").
You can in fact accomplish this with a single replace() call:
var test = "{\"schemaText\":{\"fields\":[{\"name\":\"AX_SND_TYPE\",\"type\":\"string\"},{\"name\":\"BWORK\",\"type\":\"int\"}],\"name\":\"XXXSchema\",\"type\":\"record\"},\"description\":\"Autogenerated by NiFi\"}";
window.alert(test.replace(/({"schemaText":)|}(,"description")(.*)|([^"]*)"/g, function(a,b,c,d,e){ return (b=="{\"schemaText\":"?b+"\"":(c==",\"description\""?"}\""+c+d:e+"\\\"")) })));
So it's technically "the same regex", but the substitution parameter uses an inline function as replacement rather than a static string.

Accessing nested list items in an interpolated string using dot notation in Scala

I'm trying to pass a value via JSON that I am having trouble accessing. We have a data structure (that was obviously not built by me otherwise I would likely understand it) that looks something like this when sent to the browser:
{Foo(Bar(List(Baz(List(G3),w))),G3,None)}
This is sent via a JSON write method, but the originating Scala line looks like:
val hint = Some(s"{$question}") where $question is of type Foo.
I've tried using dot notation to access the list items in ways that I thought would work:
val hint = Some(s"{$question.Bar.Baz})"
val hint = Some(s"{$question.Bar(0).Baz(0)"})
It's the deepest G3 I wanted to strip out and send, but instead the JSON object comes through looking like:
{Foo(Bar(List(Baz(List(G3),w))),G3,None)}.Bar.Baz or
{Foo(Bar(List(Baz(List(G3),w))),G3,None)}.Bar(0).Baz(0)
I must be fundamentally missing something about the data structures involved here.
I think you're just using the wrong syntax. The $ needs to come before the {} and the {} is necessary for any expression more complicated than just a variable name:
s"${question.bar(0).baz(0)}"