Elixir decode with Poison - json

I'm getting this string as query result from my database:
"%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
Is there any way to convert this one back to map?
I'm getting this error decoding it with poison
** (Poison.SyntaxError) Unexpected token: %
(poison) lib/poison/parser.ex:56: Poison.Parser.parse!/2
(poison) lib/poison.ex:83: Poison.decode!/2
I can't fix the way data is being added to database, i must find a proper way for a key/value route to easily retrive data from that. (this is just a sample for a more complex result)

As it was mentioned in comments, you should not use Code.eval_string. But, there is a way to safely convert your code to Elixir struct, using Code module:
ex(1)> encoded = "%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
"%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
First, get the AST from the string, but use the pattern matching to ensure it is a struct you are looking for ({:__aliases__, _, [:Sample, :Struct]}). All other (potentially malicious) code will fail this match:
iex(2)> {:ok, {:%, _, [{:__aliases__, _, [:Sample, :Struct]}, {:%{}, _, keymap}]} = ast} = Code.string_to_quoted(encoded)
{:ok,
{:%, [line: 1],
[{:__aliases__, [line: 1], [:Sample, :Struct]},
{:%{}, [line: 1], [list: [], total: "0.00", day: 6, id: "8vfts6"]}]}}
Here you have the full ast for you struct, and the keymap. You may now be tempted to use eval_quoted with the AST, to get the struct you needed:
iex(3)> {struct, _} = Code.eval_quoted(ast)
{%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}, []}
iex(4)> struct
%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}
But it is still not safe! Someone may put a function causing side effect into the string, like "%Sample.Struct{list: IO.puts \"Something\"}", which will be executed during the evaluation. So you will need to check the keymap firsts, if it contain safe data.
Or you may just use keymap directly, without evaluating anyting:
iex(5)> struct(Sample.Struct, keymap)
%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}

Related

Elixir - JasonHelpers - How can I send a keyword list to json_map?

I have a data structure that I want to convert to json and preserve the key order.
For example:
%{ x: 1, a: 5} should be converted to "{\"x\": 1, \"a\": 5}"
Poison does it without any problem. But when I upgrade to Jason, it changes to "{\"a\": 5, \"x\": 1}".
So I use JasonHelpers json_map to preserve the order like this:
Jason.Helpers.json_map([x: 1, a: 5])
It creates a fragment with correct order.
However, when I use a variable to do this:
list = [x: 1, a: 5]
Jason.Helpers.json_map(list)
I have an error:
** (Protocol.UndefinedError) protocol Enumerable not implemented for {:list, [line: 15], nil} of type Tuple.
....
QUESTION: How can I pass a pre-calculated list into Jason.Helpers.json_map ?
The calculation is complicated, so I don't want to repeat the code just to use json_map, but use the function that returns a list.
json_map/1 is a macro, from its docs:
Encodes a JSON map from a compile-time keyword.
It is designed for compiling JSON at compile-time, which is why it doesn't work with your runtime variable.
Support for encoding keyword lists was added to the Jason library a year ago, but it looks like it hasn't been pushed to hex yet. I managed to get it work by pulling the latest code from github:
defp deps do
[{:jason, git: "https://github.com/michalmuskala/jason.git"}]
end
Then by creating a struct that implements Jason.Encoder (adapted from this solution by the Jason author):
defmodule OrderedObject do
defstruct [:value]
def new(value), do: %__MODULE__{value: value}
defimpl Jason.Encoder do
def encode(%{value: value}, opts) do
Jason.Encode.keyword(value, opts)
end
end
end
Now we can encode objects with ordered keys:
iex(1)> Jason.encode!(OrderedObject.new([x: 1, a: 5]))
"{\"x\":1,\"a\":5}"
I don't know if this is part of the public API or just an implementation detail, but it appears you have some control of the order when implementing the Jason.Encoder protocol for a struct.
Let's say you've defined an Ordered struct:
defmodule Ordered do
#derive {Jason.Encoder, only: [:a, :x]}
defstruct [:a, :x]
end
If you encode the struct, the "a" key will be before the "x" key:
iex> Jason.encode!(%Ordered{a: 5, x: 1})
"{\"a\":5,\"x\":1}"
Let's reorder the keys we pass in to the :only option:
defmodule Ordered do
#derive {Jason.Encoder, only: [:x, :a]}
defstruct [:a, :x]
end
If we now encode the struct, the "x" key will be before the "a" key:
iex> Jason.encode!(%Ordered{a: 5, x: 1})
"{\"x\":1,\"a\":5}"

Pretty-print using the new Ruby syntax for hashes

There's a ton of questions about Ruby pretty-printing of recursive structures à la JSON (i.e, just scalars, arrays and hashes), and the answers refer to the json, pp, awesome_printer, etc. However, I have not seen a way to pretty-print a hash in Ruby syntax, that in addition would please classical Ruby linters. Something like
> pretty({a: [1, 2, {b: 3, c: 4}], d: {e: {'f g': 42}}})
=> "{a: [1, 2, {b: 3, c: 4}], d: {e: {'f g': 42}}}"
awesome_print comes close:
> ({a: [1, 2, {b: 3, c: 4}], d: {e: {'f g': 42}}}).
ai(plain: true, multiline: false, ruby19_syntax: true)
=> "{ a: [ 1, 2, { b: 3, c: 4 } ], d: { e: { \"f g\": 42 } } }"
but I didn't find a means to get rid of the inner spaces for braces and brackets, and it chose to use double-quotes for a constant string, which is disliked by Rubocop.
I can write my pretty-printer myself, but I'm surprised there's no COTS^h^h^h^hgem that does that. Did I miss something?

Write ecto schema to csv file in elixir?

Hi I have Person schema in elixir like this
[
%Texting.Contact.Person{
__meta__: #Ecto.Schema.Metadata<:loaded, "people">,
email: nil,
name: "John",
phone_number: "13334445555",
phonebook: #Ecto.Association.NotLoaded<association :phonebook is not loaded>,
phonebook_id: 60,
previous_phonebook_id: 60,
subscribed: true,
updated_at: ~N[2018-09-22 14:36:04.788163],
user: #Ecto.Association.NotLoaded<association :user is not loaded>,
user_id: 54
},
%Texting.Contact.Person{
__meta__: #Ecto.Schema.Metadata<:loaded, "people">,
email: nil,
name: "Rhee",
phone_number: "14443335555",
phonebook: #Ecto.Association.NotLoaded<association :phonebook is not loaded>,
phonebook_id: 60,
previous_phonebook_id: 60,
subscribed: true,
updated_at: ~N[2018-09-22 14:36:13.671479],
user: #Ecto.Association.NotLoaded<association :user is not loaded>,
user_id: 54
}
]
And I want to save this to csv file format.So I did this
def write!(people) do
file = File.open("contacts.csv", [:write, :utf8])
people
|> Enum.map(&Map.from_struct(&1))
|> Enum.map(&CSV.encode(&1, headers: [:name, :phone_number]))
|> Enum.map(&IO.write(file, &1))
end
but I got an error like this
** (Protocol.UndefinedError) protocol String.Chars not implemented for #Function<62.51129937/2 in Stream.transform/3>. This protocol is implemented for: Atom, BitString, Date, DateTime, Decimal, Ecto.Date, Ecto.DateTime, Ecto.Time, Float, Floki.Selector, Floki.Selector.AttributeSelector, Floki.Selector.Combinator, Floki.Selector.Functional, Floki.Selector.PseudoClass, Integer, List, NaiveDateTime, Postgrex.Copy, Postgrex.Query, Postgrex.Stream, Time, URI, Version, Version.Requirement
(elixir) /home/ubuntu/bob/tmp/0a92cc555e2418d1b56e3b10e5321a85/elixir/lib/elixir/lib/string/chars.ex:3: String.Chars.impl_for!/1
(elixir) /home/ubuntu/bob/tmp/0a92cc555e2418d1b56e3b10e5321a85/elixir/lib/elixir/lib/string/chars.ex:22: String.Chars.to_string/1
(elixir) lib/io.ex:553: IO.write/2
(elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
What I want to do is save only name and phone_number field to csv file format.
How can I do this?
According to the documentation, CSV.encode/2 takes a list of list of strings and doesn't have the option you tried to use.
Here's how I'd do it:
def write!(people) do
people
|> Stream.map(&[&1.name, &1.phone_number])
|> CSV.encode()
|> Enum.into(File.stream!("contacts.csv"))
end
The Enum.map creates a list of list of strings which is then encoded and streamed into contacts.csv.
To add a header line at the top, you can use Stream.concat/2 to prepend the header line:
def write!(people) do
[["name", "phone_number"]]
|> Stream.concat(people |> Stream.map(&[&1.name, &1.phone_number]))
|> CSV.encode()
|> Enum.into(File.stream!("contacts.csv"))
end

how to use a jsonfiy object in assertions?

I am trying to unit test some code, and want to assert that the jsonify output of the code is correct. here is what I have so far.
def test_get_ticket(self):
with self.app.test_request_context('/?main_id=14522&user_id=82'):
methodOutput = brain_get_ticket.get_ticket({'main_id': {1: 0}, 'status': {'Closed': 0},
'available': {'False': 0}}, "main_id, status, available",
['main_id', 'status', 'available'])
correct_return_output = json.dumps(dict(
to_be_working_on_last_id=0,
to_be_working_on_id=6,
information={'status': {'Closed': 1}, 'available': {'False': 1}, 'main_id': {1: 1}}
))
self.assertEquals(json.loads(methodOutput.data()), correct_return_output, "output was: " + str(methodOutput) + " it should be: " + str(correct_return_output))
the output i'm getting is :
self.assertEquals(json.loads(methodOutput.data()), correct_return_output)
TypeError: 'str' object is not callable
any suggestions????
Solved:
the main problem was that I was using data as if it was a method, not a descriptor, like Martijn said. Also changing the correct_return_output to a dictionary instead of a jsonify object to compare to the actual method output worked. THANKS!
Response.data is a descriptor and does not need to be called; you are trying to call the returned JSON string here.
Your better bet is to decode that JSON response; dictionaries are unordered and you should not count on what order the resulting JSON data is listed in. You already do so, but then you should compare that against a dictionary, not a new JSON string!
def test_get_ticket(self):
with self.app.test_request_context('/?main_id=14522&user_id=82'):
methodOutput = brain_get_ticket.get_ticket(
{'main_id': {1: 0}, 'status': {'Closed': 0},
'available': {'False': 0}},
"main_id, status, available", ['main_id', 'status', 'available'])
correct_return_output = dict(
to_be_working_on_last_id=0,
to_be_working_on_id=6,
information={'status': {'Closed': 1},
'available': {'False': 1},
'main_id': {1: 1}})
self.assertEquals(
json.loads(methodOutput.data),
correct_return_output,
"output was: {!r}, it should be {!r}".format(
methodOutput.data, correct_return_output))

Argument error in collect object Rails 3

Guys I'm using a select() tag in .html.erb file as follows
<%= select(:hfi_id, b.beneficiaryloans.collect { |h| [User.find(h.hfi_id).firstname, h.hfi_id] }) %>
what's wrong in this statement? actually it is giving an error called
wrong number of arguments (2 for 3) - error for above line
But same thing I executed in irb console, it's working fine like
irb(main):012:0> me=Beneficiary.find(1)
=> #<Beneficiary id: 1, firstname: "Mohan", lastname: "Bairwa", address: "1399 m.k.b jagatpira", age: 24, sex: "Male", total_members: 1, cso_id: 123, project_id: 17, remarks: nil, status_id: 4, created_at: "2011-11-07 09:39:24", updated_at: "2011-11-07 09:55:07">
irb(main):018:0> me.beneficiaryloans.collect {|h|User.find(h.hfi_id).firstname,h.hfi_id]}
=> [["Gruh", 117]]
using irb console I'm getting correct result
=> [["Gruh", 117]]
but when I put it in .html.erb file, It's giving argument error. How to solve this?
Look at this. select method has 3 obligatory parameters and you provide only two..