I am trying to retrieve and parse a JSON file using reqwest.
I used this question as a starting point but it doesn't work with my API.
The error:
Error: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let resp = reqwest::get("https://tse.ir/json/MarketWatch/data_7.json")
.await?
.json::<serde_json::Value>()
.await?;
println!("{:#?}", resp);
Ok(())
}
The API works fine with other languages. thank for your help.
Cargo.toml:
[package]
name = "rust_workspace"
version = "0.1.0"
edition = "2021"
[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
reqwest = { version = "0.11", features = ["json", "blocking"] }
tokio = { version = "1", features = ["full"] }
bytes = "1"
The error happens usually when the response doesn't contain
"content-type":"application/json"
Even though the content is a valid json you will get that error.
To solve the issue you need to use
let text_response = reqwest.get("...").await?.text().await?;
let resp: serde_json::Value = serde_json::from_str(&text_response)?;
I am trying to output the Hyper HTTP Request Object as JSON. I am using Serde to Serialize the object to JSON.
Since I am not defining the HTTP Request Struct, I can't add the attribute #[derive(Serialize, Deserialize)] to it. any ideas on how to get around this?
my code:
use hyper::body::HttpBody as _;
use hyper::{Body, Client, Method, Request, Uri};
use hyper_tls::HttpsConnector;
#[cfg(feature = "std")]
use serde::{Deserialize, Deserializer, Serialize, Serializer};
use serde_json::{json, Result as re_, Value};
use tokio::io::{stdout, AsyncWriteExt as _};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let https = HttpsConnector::new();
// Client Object
let client = Client::builder().build::<_, hyper::Body>(https);
let uri: Uri = "https://api.api.com".parse()?;
let http_body = String::from(" body");
let content_length = http_body.chars().count();
//Build Request
let req = Request::builder()
.method(Method::POST)
.uri(uri)
.header("accept", "application/xml")
.header("content-type", "application/xml")
.header("Content-Length", content_length)
.body(Body::from(http_body))?;
// Serialize it to a JSON string.
let j = serde_json::to_string(&req)?;
// Print HTTP Request
println!("{}", j);
// Await the response...
let mut resp = client.request(req).await?;
println!("Response: {}", resp.status());
Ok(())
}
stack trace shows:
| let j = serde_json::to_string(&req)?;
| --------------------- ^^^^ the trait `serde::ser::Serialize` is not implemented for `Request<Body>`
| |
| required by a bound introduced by this call
|
note: required by a bound in `serde_json::to_string`
--> /home/joel/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.79/src/ser.rs:2225:17
|
2225 | T: ?Sized + Serialize,
| ^^^^^^^^^ required by this bound in `serde_json::to_string`
Cargo.toml
[dependencies]
serde = { version = "1.0.136", features = ["derive"] }
serde_json = "1.0.79"
serde_derive = "1.0.136"
hyper = { version = "0.14.18", features = ["full"] }
tokio = { version = "1.17.0", features = ["full"] }
hyper-tls = "0.5.0"
rustls = "0.20.4"
If you want to derive de/serialize for an external crate you can follow the oficial serde guide for doing so.
Also, if you are trying to serialize it just to print it, maybe you could consider writing or using a default Debug or Display implementation over a wrapper. The default Debug one may do:
// Print HTTP Request
println!("{req:?}");
We often want to use JSON for human readability. As such, it is common to ask to sort the JSON keys alphabetically (or alphanumerically) in Go, in .NET, in Python, in Java, ...
But how to output a JSON with JSON keys sorted alphabetically in Swift?
PrettyPrinted output is easy:
JSONSerialization.writeJSONObject(jsonObject, to: outputStream, options: [.prettyPrinted], error: nil)
Yet the keys are not alphabetically sorted for human readability. They are likely in the order given by NSDictionary.keyEnumerator(). But sadly, we can't subclass Dictionary, NSDictionary or CFDictionary in Swift, so we can't override the behavior of keys order.
[edit: actually, we can subclass NSDictionary, see one of my answers below]
For iOS 11+ and macOS High Sierra (10.13+), a new option .sortedKeys solves the problem easily:
JSONSerialization.writeJSONObject(jsonObject, to: outputStream, options: [.sortedKeys, .prettyPrinted], error: nil)
Thank you Hamish for the hint.
Pure Swift solution for iOS 7+, macOS 10.9+ (OS X Mavericks and up).
A solution is to subclass NSDictionary (but not overriding the default init method as it wouldn't compile with Swift).
class MutableOrderedDictionary: NSDictionary {
let _values: NSMutableArray = []
let _keys: NSMutableOrderedSet = []
override var count: Int {
return _keys.count
}
override func keyEnumerator() -> NSEnumerator {
return _keys.objectEnumerator()
}
override func object(forKey aKey: Any) -> Any? {
let index = _keys.index(of: aKey)
if index != NSNotFound {
return _values[index]
}
return nil
}
func setObject(_ anObject: Any, forKey aKey: String) {
let index = _keys.index(of: aKey)
if index != NSNotFound {
_values[index] = anObject
} else {
_keys.add(aKey)
_values.add(anObject)
}
}
}
With it, we can order the keys of our object with .forcedOrdering before writing it with .prettyPrinted:
// force ordering
let orderedJson = MutableOrderedDictionary()
jsonObject.sorted { $0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending }
.forEach { orderedJson.setObject($0.value, forKey: $0.key) }
// write pretty printed
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
But be careful: you will need to subclass and sort all subdictionaries of your JSON object if you have any. Here is an extension for doing that recursion, inspired by Evgen Bodunov's gist (thank you).
extension MutableOrderedDictionary {
private static let defaultOrder: ((String, Any), (String, Any)) -> Bool = {
$0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending
}
static func sorted(object: Any, by areInIncreasingOrder: ((key: String, value: Value), (key: String, value: Value)) -> Bool = defaultOrder) -> Any {
if let dict = object as? [String: Any] {
return MutableOrderedDictionary(dict, by: areInIncreasingOrder)
} else if let array = object as? [Any] {
return array.map { sorted(object: $0, by: areInIncreasingOrder) }
} else {
return object
}
}
convenience init(_ dict: [String: Any], by areInIncreasingOrder: ((key: String, value: Value), (key: String, value: Value)) -> Bool = defaultOrder) {
self.init()
dict.sorted(by: areInIncreasingOrder)
.forEach { setObject(MutableOrderedDictionary.sorted(object: $0.value, by: areInIncreasingOrder), forKey: $0.key) }
}
}
Usage:
// force ordering
let orderedJson = MutableOrderedDictionary(jsonObject)
// write pretty printed
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
Solution for iOS 7+, macOS 10.9+ (OS X Mavericks and up).
A solution is to subclass NSDictionary in Objective-C, then use the subclass from a framework (for Application) or static library (for Command Line Tool).
For this demonstration, I will use nicklockwood/OrderedDictionary (700+ lines of code) instead of doing it from scratch, but there may be untested alternatives like quinntaylor/CHOrderedDictionary or rhodgkins/RDHOrderedDictionary. To integrate it as a Framework, add this dependency in your PodFile:
pod 'OrderedDictionary', '~> 1.4'
Then we will order the keys of our object:
import OrderedDictionary
let orderedJson = MutableOrderedDictionary()
jsonObject.sorted { $0.0.compare($1.0, options: [.forcedOrdering, .caseInsensitive]) == .orderedAscending }
.forEach { orderedJson.setObject($0.value, forKey: $0.key) }
(note: setObject(_,forKey:) is specific to MutableOrderedDictionary)
And finally we can write it prettyPrinted:
_ = JSONSerialization.writeJSONObject(orderedJson, to: outputJSON, options: [.prettyPrinted], error: nil)
But be careful: you need to subclass and sort all subdictionaries of your JSON object.
Here is one possible workaround that works only for macOS Swift scripts. It is not for iOS.
We workaround Swift Foundation limitations with a different programming language (Python 2.7 for example).
import Cocoa
// sample jsonObject
let jsonObject = ["hello": "world", "foo": "bar"]
// writing JSON to file
let jsonPath = "myJson.json"
let outputJSON = OutputStream(toFileAtPath: jsonPath, append: false)!
outputJSON.open()
_ = JSONSerialization.writeJSONObject(jsonObject, to: outputJSON, options: [], error: nil)
outputJSON.close()
// sortedKeys equivalent using `sort_keys=True`
// prettyPrinted equivalent using `indent=2, separators=(',', ' : ')`
// unicode using `io.open` and `ensure_ascii=False`
func shell(_ args: String...) {
let task = Process()
task.launchPath = "/usr/bin/env"
task.arguments = args
task.launch()
task.waitUntilExit()
}
shell("python", "-c", ""
+ "import io\n"
+ "import json\n"
+ "jsonPath = 'myJson.json'\n"
+ "with io.open(jsonPath, mode='r', encoding='utf8') as json_file:\n"
+ " all_data = json.load(json_file)\n"
+ "with io.open(jsonPath, mode='w', encoding='utf8') as json_file:\n"
+ " json_file.write(unicode(json.dumps(all_data, ensure_ascii=False, sort_keys=True, indent=2, separators=(',', ' : '))))\n"
)
I have a JSON String that follows a format like { name: "John" } and not { "name" : "John"} and that is resulting in a nil whenever I try to access the name key because:
Error Domain=NSCocoaErrorDomain Code=3840 "No string key for value in object around character 1."
I am looking for a function that fixes/parses/formats this JSON file into something readable? How do sites like JSON Format do it?
Funny enough, { name: "John" } makes a valid JSON object in Javascript. So your problem now becomes finding a Javascript intepreter for Swift!
There's one built-in to recent versions of Mac OS X and iOS: WKWebView. It's a web rendering engine with a Javascript parser. Link your target with WebKit and try this:
import WebKit
class MyJSONParser {
private static let webView = WKWebView()
class func parse(jsonString: String, completionHandler: (AnyObject?, NSError?) -> Void) {
self.webView.evaluateJavaScript(jsonString, completionHandler: completionHandler)
}
}
Usage:
let str = "{ firstName: 'John', lastName: 'Smith' }"
// You must assign the JSON string to a variable or the Javascript
// will return void. Note that this runs asynchronously
MyJSONParser.parse("tmp = \(str)") { result, error in
guard error == nil else {
print(error)
return
}
if let dict = result as? [String: String] {
print(dict)
} else {
print("Can't convert to Dictionary")
}
}
Swift 3
import WebKit
class MyJSONParser {
private static let webView = WKWebView()
class func parse(jsonString: String, completionHandler: #escaping (Any?, Error?) -> Void) {
self.webView.evaluateJavaScript(jsonString, completionHandler: completionHandler)
}
}
let str = "{ firstName: 'John', lastName: 'Smith' }"
// You must assign the JSON string to a variable or the Javascript
// will return void. Note that this runs asynchronously
MyJSONParser.parse(jsonString: "tmp = \(str)") { result, error in
guard error == nil else {
print(error!)
return
}
if let dict = result as? [String: String] {
print(dict)
} else {
print("Can't convert to Dictionary")
}
}
I am trying to parse JSON with the following code:
func ltchandler(response: NSURLResponse!, data : NSData!, error : NSError!) { //Is passed the results of a NSURLRequest
if ((error) != nil) {
//Error Handling Stuff
} else {
if (NSString(data:data, encoding:NSUTF8StringEncoding) == "") {
//Error Handling Stuff
} else {
var data = NSData(data: data);
// Define JSON string
var JSONString = "\(data)"
// Get NSData using string
if let JSONData = JSONString.dataUsingEncoding(NSUTF8StringEncoding, allowLossyConversion: false) {
// Parse JSONData into JSON object
var parsingError: NSError?
if let JSONObject = NSJSONSerialization.JSONObjectWithData(data, options: nil, error: &parsingError) as? [String: AnyObject] {
// If the parsing was successful grab the rate object
var rateObject: Double! = JSONObject["price"]?.doubleValue
// Make sure the rate object is the expected type
if let rate = rateObject as? Double! { // THIS IS NOT WORKING!!!
//Do stuff with data
} else {
println("Parsing Issue")
}
}
}
}
}
}
The line marked THIS IS NOT WORKING!!! is not being called.
From what I can tell, it cannot cast rateObject as a double - why not? It is not showing any errors.
To clarify, the expected behavior is that a double is created from the JSON object.
To strictly answer your question have you tried printing the rateObject. Also why are you casting to Double! rather than just Double in the problematic line?
Personally I don't use ! in almost all cases. You are better off using either non-optionals or proper optionals.
In the relevent section I would write:
// Make sure the rate object is the expected type
if let rate = JSONObject["price"]?.doubleValue {
//Do stuff with rate
} else {
print("Parsing Issue")
}
Of course if the JSONObject["price"] is not something with a doubleValue method or the method returns nil you will end up with nil and the else case being taken.
the code worked for me, try this code:
// if the value equals nil or any String, the instruction abort the if
// SWIFT 2.0 in xcode beta 5
if let rate = Double((JSONObject["price"] as? String)!){
// insert you code here
} else {
print("error message")
}