I have successfully produced the json representation below for one custom object. However, when I use json_decode(jsonObjectBelow) I am not successfully in converting it to an array. I have finally found out why this has happened. If I delete characters with accents e.g "Muñoz-Gómez", then the json_decode(jsonObjectBelow) works, and I am able to generate my array and extract the attributes. For practical purposes, I cannot scour all the attributes and removing these accents...Is there a solution?
json_object received: [{"abstractText":"Fevers of unknown origin (FUOs) may be due to any of over 200 different disorders. We present a most unusual case of an FUO in a returning traveler from the Dominican Republic. Work-up for Q fever, Brucellosis, Bartonella, malaria and HIV were negative, but very highly elevated ESRs and ferritin levels suggested possible Legionnaire\u0027s disease. This is the third reported case of Legionnaire\u0027s disease presenting as an FUO.","authorString":"Muñoz-Gómez S, Cunha BA.","issue":"1","journalTitle":"Heart Lung","pageInfo":"72-74","pmid":"25444558","pubYear":"2015","title":"Fever of unknown origin (FUO) due to Legionnaire\u0027s disease.","volume":"44"}]
This code works for text in attributes that have not got accents, but does not work for text with accents.
My php code:
//Get JSON POST
$json = $_POST["jsonObj"];
//Decode JSON into an Array
$data = json_decode($json);
for($i = 0; $i<count($data); $i++){
echo "[".$i."]" . " " .$data[$i]->authorString . "/ " . $data[$i]->journalTitle . "<br>";
}
if possible try to encode the values and the send the json object. Try the steps in the following link:
UTF-8 character encoding battles json_encode()
Related
This is strange, I've used json encoding arrays and the output is something like [{" etc, but in another code this time the output is "{"1":{".. causing multiple errors. I don't understand what is going on.
this is the part of the code:
$json_arr = json_decode($json_str, true);
$fecha = date("Y-m-d H:i:s");
foreach (array_column($json_arr, 'f') AS $k => $fecha) {
if($fecha < $ahora){
unset($json_arr[$k]);
}
}
$json_str = json_encode($json_arr,true);//this will be inserted in the DB table
but the $json_str is in the form of "{"1":{".. but I need in the form of [{".
Here some of the images when debugging:
in orange, the json_str is readed from the BD table
after decoding, you see the json_arr is an array of three elements
after deleting some row, you see json_arr is still an array
after encoding the json_arr, I get the "{"1":{" format, in other cases of encoding arrays I had the [{" format, which is what I need.
You are starting with an array of 3 arrays, with the indices of 0, 1, and 2.
Then you are deleting the first one. If you compare your 2nd and 3rd screenshots, specifically the popup portion, you'll see that what you have lost is index 0. Your array now starts with index 1.
But in JavaScript, an array can't start with index 1. It has to start with index 0, so PHP is encoding it as an object, instead of an array.
If you use the PHP function array_values() it will re-index your array and you should be good to go.
Haskell has become useful as a web language (thanks Servant!), and yet JSON is still so painful for me so I must be doing something wrong (?)
I hear JSON mentioned as a pain point enough, and the responses I've heard revolve around "use PureScript", "wait for Sub/Row Typing", "use esoterica, like Vinyl", "Aeson + just deal with the explosion of boiler plate data types".
As an (unfair) reference point, I really enjoy the ease of Clojure's JSON "story" (of course, it's a dynamic language, and has it's tradeoffs for which I still prefer Haskell).
Here's an example I've been staring at for an hour.
{
"access_token": "xxx",
"batch": [
{"method":"GET", "name":"oldmsg", "relative_url": "<MESSAGE-ID>?fields=from,message,id"},
{"method":"GET", "name":"imp", "relative_url": "{result=oldmsg:$.from.id}?fields=impersonate_token"},
{"method":"POST", "name":"newmsg", "relative_url": "<GROUP-ID>/feed?access_token={result=imp:$.impersonate_token}", "body":"message={result=oldmsg:$.message}"},
{"method":"POST", "name":"oldcomment", "relative_url": "{result=oldmsg:$.id}/comments", "body":"message=Post moved to https://workplace.facebook.com/{result=newmsg:$.id}"},
{"method":"POST", "name":"newcomment", "relative_url": "{result=newmsg:$.id}/comments", "body":"message=Post moved from https://workplace.facebook.com/{result=oldmsg:$.id}"},
]
}
I need to POST this to FB workplace, which will copy a message to a new group, and comment a link on both, linking to each other.
My first attempt looked something like:
data BatchReq = BatchReq {
method :: Text
, name :: Text
, relativeUrl :: Text
, body :: Maybe Text
}
data BatchReqs = BatchReqs {
accessToken :: Text
, batch :: [BatchReq]
}
softMove tok msgId= BatchReqs tok [
BatchReq "GET" "oldmsg" (msgId `append` "?fields=from,message,id") Nothing
...
]
That's painfully rigid, and dealing with Maybes all over is uncomfortable. Is Nothing a JSON null? Or should the field be absent? Then I worried about deriving the Aeson instances, and had to figure out how to convert eg relativeUrl to relative_url. Then I added an endpoint, and now I have name clashes. DuplicateRecordFields! But wait, that causes so many problems elsewhere. So update the data type to use eg batchReqRelativeUrl, and peel that off when deriving instances using Typeables and Proxys. Then I needed to add endpoints, and or massage the shape of that rigid data type for which I added more datapoints, trying to not let the "tyranny of small differences" bloat my data types too much.
At this point, I was largely consuming JSON, so decided a "dynamic" thing would be to use lenses. So, to drill into a JSON field holding a group id I did:
filteredBy :: (Choice p, Applicative f) => (a -> Bool) -> Getting (Data.Monoid.First a) s a -> Optic' p f s s
filteredBy cond lens = filtered (\x -> maybe False cond (x ^? lens))
-- the group to which to move the message
groupId :: AsValue s => s -> AppM Text
groupId json = maybe (error500 "couldn't find group id in json.")
pure (json ^? l)
where l = changeValue . key "message_tags" . values . filteredBy (== "group") (key "type") . key "id" . _String
That's rather heavy to access fields. But I also need to generate payloads, and I'm not skilled enough to see how lenses will be nice for that. Circling around to the motivating batch request, I've come up with a "dynamic" way of writing these payloads. It could be simplified with helper fns, but, I'm not even sure how much nicer it'll get with that.
softMove :: Text -> Text -> Text -> Value
softMove accessToken msgId groupId = object [
"access_token" .= accessToken
, "batch" .= [
object ["method" .= String "GET", "name" .= String "oldmsg", "relative_url" .= String (msgId `append` "?fields=from,message,id")]
, object ["method" .= String "GET", "name" .= String "imp", "relative_url" .= String "{result=oldmsg:$.from.id}?fields=impersonate_token"]
, object ["method" .= String "POST", "name" .= String "newmsg", "relative_url" .= String (groupId `append` "/feed?access_token={result=imp:$.impersonate_token}"), "body" .= String "message={result=oldmsg:$.message}"]
, object ["method" .= String "POST", "name" .= String "oldcomment", "relative_url" .= String "{result=oldmsg:$.id}/comments", "body" .= String "message=Post moved to https://workplace.facebook.com/{result=newmsg:$.id}"]
, object ["method" .= String "POST", "name" .= String "newcomment", "relative_url" .= String "{result=newmsg:$.id}/comments", "body" .= String "message=Post moved from https://workplace.facebook.com/{result=oldmsg:$.id}"]
]
]
I'm considering having JSON blobs in code or reading them in as files and using Text.Printf to splice in variables...
I mean, I can do it all like this, but would sure appreciate finding an alternative. FB's API is a bit unique in that it can't be represented as a rigid data structure like a lot of REST APIs; they call it their Graph API which is quite a bit more dynamic in use, and treating it like a rigid API has been painful thus far.
(Also, thanks to all the community help getting me this far with Haskell!)
Update: Added some comments on the "dynamic strategy" at the bottom.
In similar situations, I've used single-character helpers to good effect:
json1 :: Value
json1 = o[ "batch" .=
[ o[ "method" .= s"GET", "name" .= s"oldmsg",
"url" .= s"..." ]
, o[ "method" .= s"POST", "name" .= s"newmsg",
"url" .= s"...", "body" .= s"..." ]
]
]
where o = object
s = String
Note that the non-standard syntax (no space between one-character helper and argument) is intentional. It's a signal to me and others reading my code that these are technical "annotations" to satisfy the type checker rather than a more usual kind of function call that's actually doing something.
While this adds a little clutter, the annotations are easy to ignore while reading the code. They're also easy to forget while writing code, but the type checker catches those, so they're easy to fix.
In your particular case, I think some more structured helpers do make sense. Something like:
softMove :: Text -> Text -> Text -> Value
softMove accessToken msgId groupId = object [
"access_token" .= accessToken
, "batch" .= [
get "oldmsg" (msgId <> "?fields=from,message,id")
, get "imp" "{result=oldmsg:$.from.id}?fields=impersonate_token"
, post "newmsg" (groupId <> "...") "..."
, post "oldcomment" "{result=oldmsg:$.id}/comments" "..."
, post "newcomment" "{result=newmsg:$.id}/comments" "..."
]
]
where get name url = object $ req "GET" name url
post name url body = object $ req "POST" name url
<> ["body" .= s body]
req method name url = [ "method" .= s method, "name" .= s name,
"relative_url" .= s url ]
s = String
Note that you can tailor these helpers to the specific JSON you're generating in a particular case and define them locally in a where clause. You don't need to commit to some big chunk of ADT and function infrastructure that covers all JSON use-cases in your code, as you might do if the JSON was more unified in structure across the application.
Comments on the "Dynamic Strategy"
With respect to whether or not using a "dynamic strategy" is the right approach, it probably depends on more context than can realistically be shared in a Stack Overflow question. But, taking a step back, the Haskell type system is useful to the extent that it helps clearly model the problem domain. At its best, the types feel natural and assist you with writing correct code. When they stop doing this, you need to rethink your types.
The pain you encountered with a more traditional ADT-driven approach to this problem (rigidity of the types, proliferation of Maybes, and the "tyranny of small differences") suggests that these types were a bad model at least for what you were trying to do in this case. In particular, given that your problem was one of generating fairly straightforward JSON directives/commands for an external API, rather than doing lots of data manipulation on structures that also happened to allow JSON serialization/deserialization, modeling the data as Haskell ADTs was probably overkill.
My best guess is that, if you really wanted to properly model the FB Workplace API, you wouldn't want to do it at the JSON level. Instead, you'd do it at a higher level of abstraction with Message, Comment, and Group types, and you'd end up wanting to generate the JSON dynamically anyway, because your types wouldn't directly map to the JSON structures expected by the API.
It might be insightful to compare your problem to generating HTML. Consider first the lucid (blaze-based) or shakespeare templating packages. If you look at how these work, they don't try to build up HTML by generating a DOM with ADTs like data Element = ImgElement ... | BlockquoteElement ... and then serializing them to HTML. Presumably the authors decided that this abstraction wasn't really necessary, because the HTML just needs to be generated, not analyzed. Instead they use functions (lucid) or a quasiquoter (shakespeare) to build up a dynamic data structure representing an HTML document. The chosen structure is rigid enough to ensure certain sorts of validity (e.g., proper matching of opening and closing element tags) but not others (e.g., no one stops you from sticking a <p> child in the middle of your <span> element).
When you use these packages in a larger web app, you model the problem domain at a higher level of abstraction than HTML elements, and you generate the HTML in a largely dynamic fashion because there's not a clear one-to-one mapping between the types in your problem domain model and HTML elements.
On the other hand, there's a type-of-html package that does model individual elements, so it's a type error to try to nest a <tr> inside <td> and so on. Developing these types probably took a lot of work, and there's a lot of inflexibility "baked in", but the trade-off is a whole other level of type safety. On the other hand, this seems easier to do for HTML than to do for a particular finicky JSON API.
I'm trying to construct json text as show below. But the variables such as $token, $state, $failedServers are not been replaced with its value. Note- I don't want to use any module specifically for this to work, I just want some plain string to work. Can anyone help me ?
my $json = '{"serverToken":"$token", "state":"$state","parameters" :"$failedServers"}';
current output was:
{"serverToken":"$token", "state":"$state","parameters" :"$failedServers"}
needed output format:
{"serverToken":"1213", "state":"failed","parameters" :"oracleapps.veeralab.com,suntrust.com"}
Your variables are not being replaced, because they are inside of a single-quoted string--that is, they are inside a string quoted by ' characters. This prevents variable substitution.
You will also be much better off creating JSON using a JSON library, such as this one. Simply using a quoted string is very dangerous. Suppose your one of your variables ends up containing a special character; you will end up with invalid JSON.
{"serverToken":"123"ABC", "state":"offline", "paramameters":"bugs"}
If your variables come from user input, really bad things could happen. Imagine that $token is set to equal foo", "state":"online", "foo":"bar. Your resulting JSON structure would be:
{"serverToken":"foo", "state":"online", "foo":"bar", "state":"offline" ...
Certainly not what you want.
Possible solutions:
The most blatantly obvious solution is simply not to the ' quote character. This has the drawback of requiring you to escape your double quote (") characters, though, but it's easy:
my $json = "{\"serverToken\":\"$token\", \"state\":\"$state\",\"parameters\" :\"$failedServers\"}";
Another option is to use sprintf:
my $json = sprintf('{"serverToken":"%s", "state":"%s", "parameters":"%s"}', $token, $state, $failedServers);
But by far, the best solution, because it won't break with wonky input, is to use a library:
use JSON;
my $json = encode_json( {
serverToken => $token,
state => $state,
paramaters => $failedServers
} );
Having trouble with accessing json values formed from php array
var latlag = '<?php echo json_encode($coordinates); ?>';
alert(latlng) produces:
[{
"1280":{"lat":"-1.197070","lng":"-1.197070"},
"1239":{"lat":"-1.222410","lng":"-1.222410"},
"1258":{"lat":"-1.153020","lng":"-1.153020"},
...
}]
I've tried all sorts of ways to access lat and lag for a specific ID and the only result other than undefined has been the nth character of latlng as if its being treated like a string?!
alert(latlng[10]); # {
alert(latlng[1280]['lat]); # undefined
alert(latlng['1280'].lat); # undefined
You don't want to put the JSON in quotes, so:
var latlag = <?php echo json_encode($coordinates); ?>;
(Technically, that's not JSON at all, it's a JavaScript object initializer. But that's fine, JSON is a subset of initializer syntax and so all valid JSON texts are also valid JavaScript initializers.)
If the structure is really as you've quoted it, it's an array with one entry, which is an object with properties with names like 1280 and 1258, whose values are objects with properties named lat and lng. So you'd access those like this:
alert(latlng[0]["1280"].lat);
latlng is the array, latlng[0] is the one object it holds, and latlng[0]["1280"] is the {"lat":"-1.197070","lng":"-1.197070"} object.
You may be wondering why I've used quotes around 1280 above. It's because those keys are clearly given as strings (as is required in JSON, though not in JavaScript initializers), and so I can't be sure there aren't entries like "0012". Property names are always strings even when not written as strings, so latlng[0][1280] and latlng["0"]["1280"] both mean the same thing (because the 0 and the 1280 are converted to string [yes, really]), but naturally latlng[0]["0012"] is not the same as latlng[0][12] because the latter uses "12", not "0012", as the property name. If you know you won't have leading zeros, you can ditch the quotes.
I use twitter API to retrieve the user home timeline tweets. I use json response format. Recently the tweet id (in API it is just 'id') are retuned wrong. As an example
normally it should return like this: "id": 14057503720, (example is from twitter console)
however at my request it is returned like this: "id": 1172601832
It is 1 digit less and it is totally different. I need the proper ID because I can't make use of the parameters like since_id or max_id.
Use id_str instead of id. It doesn't seem to be documented, but if you look at the raw source of the JSON you can see that id_str for each tweet is the one that correctly corresponds to the ID of the tweet.
"[ambiguity around the encoding of numbers in textual formats] is a problem when dealing with large numbers; for example, integers greater than 2^53 cannot be exactly represented in a IEEE 754 double-precision floating-point number, so such numbers become inaccurate when parsed in a language that uses floating-point numbers (such as JavaScript). An example of numbers larger than 253 occurs on Twitter, which uses a 64-bit number to identify each tweet. The JSON returned by Twitter’s API includes tweet IDs twice, once as a JSON number and once as a decimal string, to work around the fact that the numbers are not correctly parsed by JavaScript applications"
From "Designing Data Intensive Applications" by Martin Kleppmann
It is 1 digit less and it is totally different. I need the proper ID because I can't make use of the parameters like since_id or max_id.
It is not totally different; just different. If you write both IDs in hex, you'll receive
0x345E47BE8
0x45E47BE8
Tweet IDs are 64-bit and somewhere in parsing you lose the most significant 32-bit half of it. Use id_str as other (also in the linked article) suggest.
example of how to get the ID
$url = "http://search.twitter.com/search.json?q=QUERY"; //<--- replace the word QUERY for your own query
$data = get_data($url);
$obj = json_decode($data);
function get_data($url){
$ch = curl_init();
$timeout = 5;
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
foreach ($obj->results as $item){
$text = $item->text;
$user = $item-> from_user;
$img = $item->profile_image_url;
$tweetId = $item->id_str; // <--- On this line we are getting the ID of the tweet
echo ' #';
echo $user;
echo $text;
echo 'Tweet ID: '. $tweetId; //<-- On this line we display the ID of the tweet
For more information GET search | Twitter Developers
The example request on the line 30 shows "id_str":"122032448266698752"
and thats the reason of use $tweetId = $item->id_str;
to get the id_str