I have a list of JSON objects (received from a nosql db) and want to remove or rename some keys. And then I want to return data as a list of JSON objects once again.
This Stackoverflow post provides a good sense of how to use mochijson2. And I'm thinking I could use a list comprehension to go through the list of JSON objects.
The part I am stuck with is how to remove the key for each JSON object (or proplist if mochijson2 is used) within a list comprehension. I can use the delete function of proplists. But I am unsuccessful when trying to do that within a list comprehension.
Here is a bit code for context :
A = <<"[{\"id\": \"0129\", \"name\": \"joe\", \"photo\": \"joe.jpg\" }, {\"id\": \"0759\", \"name\": \"jane\", \"photo\": \"jane.jpg\" }, {\"id\": \"0929\", \"name\": \"john\", \"photo\": \"john.jpg\" }]">>.
Struct = mochijson2:decode(A).
{struct, JsonData} = Struct,
{struct, Id} = proplists:get_value(<<"id">>, JsonData),
Any suggestions illustrated with code much appreciated.
You can use lists:keydelete(Key, N, TupleList) to return a new tuple list with certain tuples removed. So in the list comprehension, for each entry extract the tuple lists (or proplists), and create a new struct with the key removed:
B = [{struct, lists:keydelete(<<"name">>, 1, Props)} || {struct, Props} <- Struct].
gives:
[{struct,[{<<"id">>,<<"0129">>},
{<<"photo">>,<<"joe.jpg">>}]},
{struct,[{<<"id">>,<<"0759">>},
{<<"photo">>,<<"jane.jpg">>}]},
{struct,[{<<"id">>,<<"0929">>},
{<<"photo">>,<<"john.jpg">>}]}]
and
iolist_to_binary(mochijson2:encode(B)).
gives:
<<"[{\"id\":\"0129\",\"photo\":\"joe.jpg\"},{\"id\":\"0759\",\"photo\":\"jane.jpg\"},{\"id\":\"0929\",\"photo\":\"john.jpg\"}]">>
By the way, using the lists/* tuple lists functions are much faster, but sometimes slightly less convenient than the proplists/* functions: http://sergioveiga.com/index.php/2010/05/14/erlang-listskeyfind-vs-proplistsget_value/
Related
I am trying to parse out the language for different profiles that are stored in a json field named "data". They are stored in their own array in the json field like:
"languages": ["EN", "BN", "HI"]
I can call the whole array by using:
data->>'languages' as languages
but I would like to split it out into
language1 = "EN"
language2 = "BN"
language3 = "HI"
I think if possible the best solution would be to return the whole language array but exclude "EN" from it, but I'm not sure if that is possible. ex.
"languages": ["BN", "HI"]
You can use the - operator to remove the EN key:
select (data -> 'languages') - 'EN' as languages
from the_table;
Online example
How do I convert a list with Tuples & Atoms & Binary strings in a list into JSON?
I see Erlang : Tuple List into JSON
and I found https://github.com/rustyio/BERT-JS
I want an API I can call like
erlang_json:convert([{a, b, {{c, d}}, 1}, {"a", "b", {{cat, dog}}, 2}
where the atoms would be converted to strings or some other standard way to process on the Javascript side.
I have complicated Erlang lists I need to send to my webpage.
It's unclear what [{a, b, {{c, d}}, 1}, {"a", "b", {{cat, dog}}, 2}... would turn into as JSON, but you might take a look at jiffy or jsx. Both of them work on simple key/value structures. For instance:
> Term = #{a => b, c => 1, <<"x">> => <<"y">>}.
#{a => b,c => 1,<<"x">> => <<"y">>}
> jiffy:encode(Term).
<<"{\"x\":\"y\",\"c\":1,\"a\":\"b\"}">>
> jsx:encode(Term).
<<"{\"a\":\"b\",\"c\":1,\"x\":\"y\"}">>
If you can say what JSON you want your example input to turn into, I might be able to give you a better suggestion.
Just for you
https://github.com/romanr321/t2j
You don't need to wrap it in a list though, it takes one tuple argument and returnes a json formated string.
>Tuple = {{key, value}, { key2, {key3, [value1, 2,3]}}}.
>t2j:t2jp(Tuple).
{"key":"value", "key2, {"key3":["value1", 2,3]}}
The library jsone is pretty good. It can translate between maps or tuples:
https://github.com/sile/jsone
I've used it extensively and it's lightning fast.
The only problem I've found is that a map that contains a list of maps throws an error. I hope this is fixed, but maybe I'm the only tart trying to do that.
I am trying to use JSX to convert a list of tuples to a JSON object.
The list items are based on a record definition:
-record(player, {index, name, description}).
and looks like this:
[
{player,1,"John Doe","Hey there"},
{player,2,"Max Payne","I am here"}
]
The query function looks like this:
select_all() ->
SelectAllFunction =
fun() ->
qlc:eval(qlc:q(
[Player ||
Player <- mnesia:table(player)
]
))
end,
mnesia:transaction(SelectAllFunction).
What's the proper way to make it convertable to a JSON knowing that I have a schema of the record used and knowing the structure of tuples?
You'll have to convert the record into a term that jsx can encode to JSON correctly. Assuming you want an array of objects in the JSON for the list of player records, you'll have to either convert each player to a map or list of tuples. You'll also have to convert the strings to binaries or else jsx will encode it to a list of integers. Here's some sample code:
-record(player, {index, name, description}).
player_to_json_encodable(#player{index = Index, name = Name, description = Description}) ->
[{index, Index}, {name, list_to_binary(Name)}, {description, list_to_binary(Description)}].
go() ->
Players = [
{player, 1, "John Doe", "Hey there"},
% the following is just some sugar for a tuple like above
#player{index = 2, name = "Max Payne", description = "I am here"}
],
JSON = jsx:encode(lists:map(fun player_to_json_encodable/1, Players)),
io:format("~s~n", [JSON]).
Test:
1> r:go().
[{"index":1,"name":"John Doe","description":"Hey there"},{"index":2,"name":"Max Payne","description":"I am here"}]
I want to parse a string of complex JSON in Pig. Specifically, I want Pig to understand my JSON array as a bag instead of as a single chararray. I found that complex JSON can be parsed by using Twitter's Elephant Bird or Mozilla's Akela library. (I found some additional libraries, but I cannot use 'Loader' based approach since I use HCatalog Loader to load data from Hive.)
But, the problem is the structure of my data; each value of Map structure contains value part of complex JSON. For example,
1. My table looks like (WARNING: type of 'complex_data' is not STRING, a MAP of <STRING, STRING>!)
TABLE temp_table
(
user_id BIGINT COMMENT 'user ID.',
complex_data MAP <STRING, STRING> COMMENT 'complex json data'
)
COMMENT 'temp data.'
PARTITIONED BY(created_date STRING)
STORED AS RCFILE;
2. And 'complex_data' contains (a value that I want to get is marked with two *s, so basically #'d'#'f' from each PARSED_STRING(complex_data#'c') )
{ "a": "[]",
"b": "\"sdf\"",
"**c**":"[{\"**d**\":{\"e\":\"sdfsdf\"
,\"**f**\":\"sdfs\"
,\"g\":\"qweqweqwe\"},
\"c\":[{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"}]
},
{\"**d**\":{\"e\":\"sdfsdf\"
,\"**f**\":\"sdfs\"
,\"g\":\"qweqweqwe\"},
\"c\":[{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"},
{\"d\":21321,\"e\":\"ewrwer\"}]
},]"
}
3. So, I tried... (same approach for Elephant Bird)
REGISTER '/path/to/akela-0.6-SNAPSHOT.jar';
DEFINE JsonTupleMap com.mozilla.pig.eval.json.JsonTupleMap();
data = LOAD temp_table USING org.apache.hive.hcatalog.pig.HCatLoader();
values_of_map = FOREACH data GENERATE complex_data#'c' AS attr:chararray; -- IT WORKS
-- dump values_of_map shows correct chararray data per each row
-- eg) ([{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... }])
([{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... },
{"d":{"e":"sdfsdf","f":"sdfs","g":"sdf"},... }]) ...
attempt1 = FOREACH data GENERATE JsonTupleMap(complex_data#'c'); -- THIS LINE CAUSE AN ERROR
attempt2 = FOREACH data GENERATE JsonTupleMap(CONCAT(CONCAT('{\\"key\\":', complex_data#'c'), '}'); -- IT ALSO DOSE NOT WORK
I guessed that "attempt1" was failed because the value doesn't contain full JSON. However, when I CONCAT like "attempt2", I generate additional \ mark with. (so each line starts with {\"key\": ) I'm not sure that this additional marks breaks the parsing rule or not. In any case, I want to parse the given JSON string so that Pig can understand. If you have any method or solution, please Feel free to let me know.
I finally solved my problem by using jyson library with jython UDF.
I know that I can solve it by using JAVA or other languages.
But, I think that jython with jyson is the most simplist answer to this issue.
I'm using Postgrex in Elixir, and when it returns query results, it returns them in the following struct format:
%{columns: ["id", "email", "name"], command: :select, num_rows: 2, rows: [{1, "me#me.com", "Bobbly Long"}, {6, "email#tts.me", "Woll Smoth"}]}
It should be noted I am using Postgrex directly WITHOUT Ecto.
The columns (table headers) are returned as a collection, but the results (rows) are returned as a list of tuples. (which seems odd, as they could get very large).
I'm trying to find the best way to programmatically create JSON objects for each result in which the JSON key is the column title and the JSON value the corresponding value from the tuple.
I've tried creating maps from both, merging and then serialising to JSON objects but it seems there should be an easier/better way of doing this.
Has anyone dealt with this before? What is the best way of creating a JSON object from a separate collection and tuple?
Something like this should work:
result = Postgrex.query!(...)
Enum.map(result.rows, fn row ->
Enum.zip(result.columns, Tuple.to_list(row))
|> Enum.into(%{})
|> JSON.encode
end)
This will result in a list of json objects where each row in the resultset is a json object.