Converting json to TreeMap<String, String> - json

I want to convert json to TreeMap in a simple way, here's my attempt:
extern crate serialize;
use serialize::json;
use serialize::json::ToJson;
use serialize::json::Json;
use std::collections::TreeMap;
fn main() {
let mut tree_map1 = TreeMap::new();
tree_map1.insert("key1".to_string(), "val1".to_json());
tree_map1.insert("key2".to_string(), "val2".to_json());
//... and so on, the number of keys aren't known
let json1 = json::Object(tree_map1);
let mut tree_map2 = TreeMap::new();
for (k, v) in json1.iter() { //impossible to iterate
tree_map2.insert(k.to_string(), v.to_string());
}
}
UPDATE:
How do I convert TreeMap<String, json::Json> to TreeMap<String, String> ?
let json1 = get_json1(); // json made of TreeMap<String, json::Json>
let res = match json1 {
json::Object(json2) => json2.map(|k, v| ??? ),
_ => panic!("Error")
}

Here is a demonstration of safely converting Json into TreeMap<String, String>:
use serialize::json::Json;
use std::collections::TreeMap;
fn extract_string_map(json: Json) -> Result<TreeMap<String, String>, Json> {
let json = match json {
Json::Object(json) => json,
_ => return Err(json),
};
if !json.iter().all(|(_k, v)| v.is_string()) {
return Err(Json::Object(json));
}
Ok(json.into_iter().map(|(k, v)| (k, match v {
Json::String(s) => s,
_ => unreachable!(),
}))
.collect())
}
This demonstrates a principle of avoiding panic-prone behaviour, for this cannot fail. Also it doesn’t lose any data—should the data not fit the format, the original data is returned intact for the caller to decide what to do with it, without having needed to clone data at any point.
(As a point of curiosity, I think that this restructuring of a TreeMap is going to be fairly inefficient, requiring more rebalancing of the tree than it should, because of the keys being given in order. For performance it’d be great to have a value-changing method for a TreeMap, consuming self and producing a new type more efficiently.)

json::Object is an enum variant which contains TreeMap inside it. So in order to get a TreeMap from it, you just need to unwrap it:
let json1 = json::Object(tree_map1);
let tree_map2 = match json1 {
json::Object(tm) => tm,
_ => unreachable!()
};
This will consume json1. If you don't want it, you need to clone the map:
let tree_map2 = match json1 {
json::Object(ref tm) => tm.clone(),
_ => unreachable!()
};
The latter can be rewritten less noisily with as_object() method:
let tree_map2 = json1.as_object().unwrap().clone();
If you need to obtain TreeMap<String, String> from TreeMap<String, Json> which is contained inside Object variant, you need to convert Json to String somehow. If you know in advance that all values are JSON strings, you can use pattern matching again:
let tree_map2 = match json1 {
json::Object(tm) => tm.into_iter().map(|(k, v)| (k, match v {
json::String(s) => s,
_ => unreachable!()
})).collect(),
_ => unreachable!()
};

Related

How do I split up data from a hyper JSON body so I have a hashmap with multiple keys and values?

I've managed to extract data from a POST method in hyper using the following:
use hyper::service::{make_service_fn, service_fn};
use hyper::{Body, Method, Request, Response, Server};
use std::convert::Infallible;
use std::net::SocketAddr;
use tokio;
async fn handle(_req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
match (_req.method(), _req.uri().path()) {
(&Method::GET, "/") => Ok(Response::new(Body::from("this is a get"))),
(&Method::POST, "/") => {
let byte_stream = hyper::body::to_bytes(_req).await?;
let _params = form_urlencoded::parse(&byte_stream)
.into_owned()
.collect::<HashMap<String, String>>();
However, the whole JSON body is just one key in the HashMap now. How do I split it up so I have a hashmap with multiple keys and values as opposed to one key that's the entire body?
[dependencies]
futures = "0.1"
hyper = "0.13"
pretty_env_logger = "0.3.1"
url = "2.1.1"
tokio = { version = "0.2", features = ["macros", "tcp"] }
bytes = "0.5"
There is a discrepancy between your description:
However, the whole JSON body
And your code:
let _params = form_urlencoded::parse(&byte_stream)
If your data is JSON then parse it as JSON, using the serde_json crate:
let _params: HashMap<String, String> = serde_json::from_slice(&byte_stream).unwrap();

Write a prettyPrinted JSON object with sorted keys in Swift

We often want to use JSON for human readability. As such, it is common to ask to sort the JSON keys alphabetically (or alphanumerically) in Go, in .NET, in Python, in Java, ...
But how to output a JSON with JSON keys sorted alphabetically in Swift?
PrettyPrinted output is easy:
JSONSerialization.writeJSONObject(jsonObject, to: outputStream, options: [.prettyPrinted], error: nil)
Yet the keys are not alphabetically sorted for human readability. They are likely in the order given by NSDictionary.keyEnumerator(). But sadly, we can't subclass Dictionary, NSDictionary or CFDictionary in Swift, so we can't override the behavior of keys order.
[edit: actually, we can subclass NSDictionary, see one of my answers below]
For iOS 11+ and macOS High Sierra (10.13+), a new option .sortedKeys solves the problem easily:
JSONSerialization.writeJSONObject(jsonObject, to: outputStream, options: [.sortedKeys, .prettyPrinted], error: nil)
Thank you Hamish for the hint.
Pure Swift solution for iOS 7+, macOS 10.9+ (OS X Mavericks and up).
A solution is to subclass NSDictionary (but not overriding the default init method as it wouldn't compile with Swift).
class MutableOrderedDictionary: NSDictionary {
let _values: NSMutableArray = []
let _keys: NSMutableOrderedSet = []
override var count: Int {
return _keys.count
}
override func keyEnumerator() -> NSEnumerator {
return _keys.objectEnumerator()
}
override func object(forKey aKey: Any) -> Any? {
let index = _keys.index(of: aKey)
if index != NSNotFound {
return _values[index]
}
return nil
}
func setObject(_ anObject: Any, forKey aKey: String) {
let index = _keys.index(of: aKey)
if index != NSNotFound {
_values[index] = anObject
} else {
_keys.add(aKey)
_values.add(anObject)
}
}
}
With it, we can order the keys of our object with .forcedOrdering before writing it with .prettyPrinted:
// force ordering
let orderedJson = MutableOrderedDictionary()
jsonObject.sorted { $0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending }
.forEach { orderedJson.setObject($0.value, forKey: $0.key) }
// write pretty printed
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
But be careful: you will need to subclass and sort all subdictionaries of your JSON object if you have any. Here is an extension for doing that recursion, inspired by Evgen Bodunov's gist (thank you).
extension MutableOrderedDictionary {
private static let defaultOrder: ((String, Any), (String, Any)) -> Bool = {
$0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending
}
static func sorted(object: Any, by areInIncreasingOrder: ((key: String, value: Value), (key: String, value: Value)) -> Bool = defaultOrder) -> Any {
if let dict = object as? [String: Any] {
return MutableOrderedDictionary(dict, by: areInIncreasingOrder)
} else if let array = object as? [Any] {
return array.map { sorted(object: $0, by: areInIncreasingOrder) }
} else {
return object
}
}
convenience init(_ dict: [String: Any], by areInIncreasingOrder: ((key: String, value: Value), (key: String, value: Value)) -> Bool = defaultOrder) {
self.init()
dict.sorted(by: areInIncreasingOrder)
.forEach { setObject(MutableOrderedDictionary.sorted(object: $0.value, by: areInIncreasingOrder), forKey: $0.key) }
}
}
Usage:
// force ordering
let orderedJson = MutableOrderedDictionary(jsonObject)
// write pretty printed
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
Solution for iOS 7+, macOS 10.9+ (OS X Mavericks and up).
A solution is to subclass NSDictionary in Objective-C, then use the subclass from a framework (for Application) or static library (for Command Line Tool).
For this demonstration, I will use nicklockwood/OrderedDictionary (700+ lines of code) instead of doing it from scratch, but there may be untested alternatives like quinntaylor/CHOrderedDictionary or rhodgkins/RDHOrderedDictionary. To integrate it as a Framework, add this dependency in your PodFile:
pod 'OrderedDictionary', '~> 1.4'
Then we will order the keys of our object:
import OrderedDictionary
let orderedJson = MutableOrderedDictionary()
jsonObject.sorted { $0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending }
.forEach { orderedJson.setObject($0.value, forKey: $0.key) }
(note: setObject(_,forKey:) is specific to MutableOrderedDictionary)
And finally we can write it prettyPrinted:
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
But be careful: you need to subclass and sort all subdictionaries of your JSON object.
Here is one possible workaround that works only for macOS Swift scripts. It is not for iOS.
We workaround Swift Foundation limitations with a different programming language (Python 2.7 for example).
import Cocoa
// sample jsonObject
let jsonObject = ["hello": "world", "foo": "bar"]
// writing JSON to file
let jsonPath = "myJson.json"
let outputJSON = OutputStream(toFileAtPath: jsonPath, append: false)!
outputJSON.open()
_ = JSONSerialization.writeJSONObject(jsonObject, to: outputJSON, options: [], error: nil)
outputJSON.close()
// sortedKeys equivalent using `sort_keys=True`
// prettyPrinted equivalent using `indent=2, separators=(',', ' : ')`
// unicode using `io.open` and `ensure_ascii=False`
func shell(_ args: String...) {
let task = Process()
task.launchPath = "/usr/bin/env"
task.arguments = args
task.launch()
task.waitUntilExit()
}
shell("python", "-c", ""
+ "import io\n"
+ "import json\n"
+ "jsonPath = 'myJson.json'\n"
+ "with io.open(jsonPath, mode='r', encoding='utf8') as json_file:\n"
+ " all_data = json.load(json_file)\n"
+ "with io.open(jsonPath, mode='w', encoding='utf8') as json_file:\n"
+ " json_file.write(unicode(json.dumps(all_data, ensure_ascii=False, sort_keys=True, indent=2, separators=(',', ' : '))))\n"
)

Swift 3.0 Converting String to Double returning nil?

Trying to do something so simple but it's simply not working for me.
CODE:
let long = index["Longitude"] as! String
let lat = index["Latitude"] as! String
print(long as Any)
print(lat as Any)
let doubleLong = Double(long)
print(doubleLong as Any)
DEBUG OUTPUT:
-85.1113644443208
32.880541654362
nil
^^^ Why so nil? I will add this is being pulled from a JSON Response. Perhaps this has something to do with it.
Most likely there is an additional space in your data, note:
print(Double("-85.1113644443208")) // => Optional(-85.111364444320799)
print(Double("-85.1113644443208 ")) // => nil
print(Double(" -85.1113644443208")) // => nil
Try to trim the spaces first:
let doubleLong = Double(long.trimmingCharacters(in: CharacterSet.whitespaces))
or use a NumberFormatter for parsing:
let numberFormatter = NumberFormatter()
numberFormatter.numberStyle = .decimal
numberFormatter.locale = Locale(identifier: "en_US_POSIX")
print(numberFormatter.number(from: "-85.1113644443208") as Double?) // => Optional(-85.111364444320799)
print(numberFormatter.number(from: " -85.1113644443208") as Double?) // => Optional(-85.111364444320799)
print(numberFormatter.number(from: "-85.1113644443208 ") as Double?) // => Optional(-85.111364444320799)
There are new failable initializers that allow you to do this in more idiomatic and safe way (as many answers have noted, String's double value is not very safe because it returns 0 for non number values. This means that the doubleValue of "foo" and "0" are the same.)
let myDouble = Double(myString)
This returns an optional, so in cases like passing in "foo" where doubleValue would have returned 0, the failable intializer will return nil. You can use a guard, if-let, or map to handle the Optional

TreeMap<String, String> to json

I have the following code:
extern crate serialize;
use std::collections::TreeMap;
use serialize::json;
use serialize::json::ToJson;
use serialize::json::Json;
fn main() {
let mut tree_map = get_tree_map(); // : TreeMap<String, String>
let mut tree_map2 = tree_map.iter().map(|k, v| (k, v.to_json())); //error the type of this value must be known in this context
let json1 = json::Object(tree_map2);
}
I want to convert tree_map to json. I tried to do it by converting it to TreeMap<String, Json> but failed. How can I do that?
The closure you passed to map takes two parameters, but it should take a single parameter that is a tuple type, because iter() returns an iterator over tuples (see Trait Implementations on Entries). Change |k, v| to |(k, v)| to fix this. (I found this by adding explicit type annotations on k, v: the compiler then complained about the closure not having the right number of parameters.)
There are some other errors however. Instead of using iter(), you might want to use into_iter() to avoid cloning the Strings if you don't need the TreeMap<String, String> anymore. Also, you should add .collect() after .map(...) to turn the iterator into a TreeMap. The compiler will automatically infer the type for tree_map2 based on the requirements for json::Object.
fn main() {
let mut tree_map = get_tree_map(); // : TreeMap<String, String>
let mut tree_map2 = tree_map.into_iter().map(|(k, v)| (k, v.to_json())).collect();
let json1 = Json::Object(tree_map2);
}

Ways for converting key value string to Scala Map

I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.