Deserialize Vec<u8> into readable (raw) json - json

I'm working with a http::request::Request that I'm getting after calling this function: https://docs.rs/kube/0.76.0/kube/core/struct.Request.html#method.patch_subresource.
The call looks like that:
let mut pp = PatchParams::default();
pp.field_manager = Some(String::from("application/apply-patch"));
let req = self
.mfs
.patch_subresource("status", &self.name, &pp, &Patch::Apply(data))
.map_err(ErrorKind::KubeApi)?;
I'm actually debugging this part, so I don't know exactly what should be in the body of this request. However, I know that req.body() is a &Vec<u8>, and I know that this data can be deserialized to json.
I just want to deserialize this Vec<u8> to readable json. Nothing else. But this proved surprisingly difficult.
I tried many approaches like that without much success (when I print body I see an array of int).
let data: &Vec<u8> = req.body();
let body: serde_json::Value = serde_json::from_slice(data).unwrap();
The output is something like that:
[123,34,115,116,97,116,117,115,34,58...
I know this isn't really an array of int, it's just the debug version of Vec<u8>. So my attempt at decoding the body to json doesn't really work here.
Note that:
let body = String::from_utf8(req.body().clone()).unwrap();
Outputs something like that:
"{\"status\":{\"conditions\":{\"generated\":{\...
-> This is the debug version of a string, not the debug version of some json
Is totally readable, but I'm really trying to get some raw json here. Any idea?

Related

Swift - Parsing JSON with Combine, URLSession DataTaskPublisher - Strange formatting

I'm looking for some guidance here on an issue I'm encountering. I am polling an API for stock data and instead of getting back an array or formatted JSON I get chunks of it. Each block of JSON seems to be properly formatted, but the JSON decoders are looking for a properly formatted array.
Prior to using combine I had a different working. I had to parse the data to remove 4 trailing characters, then I was left with lines of JSON that I could split by newline and then do a forEach loop on to parse.
This was messy but it works. The issue now is I'm polling streaming data for live quote updating in my UI. I need to have the JSON chunks read into my class, stored as a variable that will update the UI. I am not worried about all of that code once I can figure out how to get the data read properly.
I know my app is receiving the data because I see the stats in the debugger for network. I also test the same API call in a browser and see what data my app is receiving. I just don't know how to break it up in the combine framework.
I have been working with something like this so far:
return URLSession.shared.dataTaskPublisher(for: request)
.map { $0.data }
.handleEvents(receiveSubscription: { print("Receive subscription: \($0)") },
receiveOutput: { print("Receive output: \($0)") },
receiveCompletion: { print("Receive completion: \($0)") },
receiveCancel: { print("Receive cancel") },
receiveRequest: { print("Receive request: \($0)") })
//.decode(type: CryptoDataContainer.self, decoder: JSONDecoder())
.decode(type: Quote.self, decoder: JSONDecoder())
.receive(on: DispatchQueue.main)
.eraseToAnyPublisher()
}
The Quote struct is very basic:
struct Quote: Codable {
public var Symbol: String
public var Last: Double
public var Bid: Double
public var Ask: Double
public var Close: Double
}
JSON data is posted below, and it streams. I get a few chunks of it at a time, but if I can split by newline or by everything {.*} to get each JSON chunk and decode individually it would work great.
I'm really looking for Last price and Symbol out of the JSON. I could even just look for Last and start one individual streaming session for each symbol I need.
I will also need to split these up using a subject to multicast each publisher to multiple subscribers but I wanted to get the basic functionality working before complicating matters.
Update: From another post I found that the problem is the JSON formatting - The API is returning invalid JSON that is not formatted into an array - it is missing the surrounding []. This is why it doesn't parse. Another option may be to simply convert to string, add square brackets around all of the JSON passed down from the server (there is nothing but objects).. and then parse it correctly? I'm not sure how to do this within a combine map but that seems to be the path forward.
Here’s an example of the map I’m trying to use currently, since the data comes in as json chunks I need to parse them each individually.
.map ({
let stringRepresentation = String(data: $0, encoding: .utf8)
let outputJson = stringRepresentation!.split(whereSeparator: \.isNewline).map(String.init)
if outputJson.count > 1 {
print("DEBUG: outputJson.count > 1 code running..")
let formattedOutputJson:String = outputJson.joined(separator: ",")
let finalJson:String = "{\"quotes\":[" + formattedOutputJson + "]}"
let data:Data? = finalJson.data(using: .utf8)
return data!
} else {
print("DEBUG: outputJson.count = 1 (Else) code running..")
let finalJson:String = "{\"quotes\":[" + outputJson[0] + "]}"
let data:Data? = finalJson.data(using: .utf8)
return data!
}
})
Here is a better example of the JSON formatting in chunks I’m talking about:
{"heartbeat":4,"timestamp":"\/Date(1622898090819)\/"}
{"heartbeat":5,"timestamp":"\/Date(1622898095819)\/"}
{"heartbeat":6,"timestamp":"\/Date(1622898100819)\/"}
{"heartbeat":7,"timestamp":"\/Date(1622898105819)\/"}
{"heartbeat":8,"timestamp":"\/Date(1622898110819)\/"}
If I can even capture the heartbeat number as an Int to use I would be well on my way to getting my code working. The issue is I get zero data out of this. I know I’m connected from packet captures and various other tests. The data is getting to my system, and to the Xcode simulated app, I just don’t think I’m getting it to a point where I can print to the console correctly to display it. I have been working on that prior to attempting to update the UI directly.
{"Description":"E-Mini NASDAQ-100 Jun 2021","PreviousClose":13529.25,"PreviousVolume":552987,"AssetType":"FUTURE","CountryCode":"United States","Halted":false,"Bid":13513.75,"BidSize":3,"Ask":13514.25,"AskSize":1,"Currency":"USD","LastSize":1,"LastVenue":"CME","SymbolRoot":"NQ","Exchange":"CME","LastTradingDate":"2021-06-18","DisplayType":3,"FractionalDisplay":false,"MinPrice":1258200,"PointValue":20,"TradeTime":"2021-06-04T02:40:20.0000000+00:00","MaxPrice":1447600,"MinMove":25,"High52Week":14064,"Open":13531.75,"Low52Week":9591.75,"High":13541.75,"Low":13467.75,"Close":13513.75,"Symbol":"NQM21","Volume":23877,"DailyOpenInterest":231736,"ExpirationDate":"2021-06-18","Last":13513.75,"NameExt":"(D)","DataFeed":"TradeStation","IsDelayed":true,"Restrictions":[null],"VWAP":13499.877658013,"NetChange":-15.50,"NetChangePct":-0.1145665872091948925476282900,"High52WeekTimeStamp":"2021-04-29T00:00:00.0000000+00:00","Low52WeekTimeStamp":"2020-06-11T00:00:00.0000000+00:00","PreviousClosePriceDisplay":"13529.25","BidPriceDisplay":"13513.75","AskPriceDisplay":"13514.25","MinPriceDisplay":"1258200.00","MaxPriceDisplay":"1447600.00","High52WeekPriceDisplay":"14064.00","OpenPriceDisplay":"13531.75","Low52WeekPriceDisplay":"9591.75","HighPriceDisplay":"13541.75","LowPriceDisplay":"13467.75","ClosePriceDisplay":"13513.75","LastPriceDisplay":"13513.75","VWAPDisplay":"13499.88","FirstNoticeDate":"","StrikePrice":0,"TickSizeTier":0,"Underlying":"","StrikePriceDisplay":"0.00"}
{"Last":4188.5,"NameExt":"(D)","DataFeed":"TradeStation","IsDelayed":true,"Restrictions":[null],"VWAP":4184.92506004117,"Description":"E-mini S&P 500 Jun 2021","PreviousClose":4191.25,"PreviousVolume":1384271,"AssetType":"FUTURE","CountryCode":"United States","Halted":false,"Bid":4188.25,"BidSize":39,"Ask":4188.5,"AskSize":23,"Currency":"USD","LastSize":1,"LastVenue":"CME","SymbolRoot":"ES","Exchange":"CME","LastTradingDate":"2021-06-18","DisplayType":3,"FractionalDisplay":false,"MinPrice":3898,"PointValue":50,"TradeTime":"2021-06-04T02:40:10.0000000+00:00","MaxPrice":4484,"MinMove":25,"High52Week":4238.25,"Open":4190.75,"Low52Week":2977.8999,"Hi{"Volume":23880,"TradeTime":"2021-06-04T02:40:23.0000000+00:00","VWAP":13499.879455674,"Symbol":"NQM21"}
{"AskSize":18,"Symbol":"ESM21"}
{"Bid":13514.25,"BidSize":1,"AskSize":1,"BidPriceDisplay":"13514.25","Symbol":"NQM21","VWAP":13499.8800582884,"Volume":23881,"TradeTime":"2021-06-04T02:40:25.0000000+00:00"}
{"VWAP":4184.92521199492,"AskSize":19,"Volume":47807,"TradeTime":"2021-06-04T02:40:27.0000000+00:00","Symbol":"ESM21"}
{"Bid":13514,"BidSize":3,"BidPriceDisplay":"13514.00","Symbol":"NQM21","TradeTime":"2021-06-04T02:40:27.0000000+00:00","Ask":13514.75,"VWAP":13499.8806713352,"Close":13514.5,"Last":13514.5,"Volume":23882,"NetChange":-14.75,"NetChangePct":-0.1090230426668144945211301400,"AskPriceDisplay":"13514.75","ClosePriceDisplay":"13514.50","LastPriceDisplay":"13514.50"}
{"AskSize":18,"Symbol":"ESM21"}

looking for most efficient way to display JSON data received via websocket, into a Xcode/Swift dynamic table

Currently I am working on a MacOS financial market trading application that could receive 10-20 websocket messages per second containing JSON data. For now, let's assume that once the data has been displayed in a table and a new message has been received, the previous data is no longer needed. What would be the recommended, and most efficient way to get the data into the table, and to keep updating it as fast as possible?
I have the websocket connection setup and working properly (using SocketRocket. Data is coming in as it should. But I really don't want to continue further without having a better understanding of the most efficient way to present that data into the table. I was reading about DictionaryArrayController, and maybe utilizing a database library, but I think the latter would would only add unnecessary overhead for what I am trying to do.
Below is a sample of the data that I am receiving.
{
"id":"5267",
"pt":"T",
"ls":["1"],
"lp":["11968"],
"t":["1571824228"],
"v":"133758",
"h":"11981",
"l":"11928",
"bp":["0","1","2","3","4","5","6","7","8","9"],
"bs":["14","73","87","66","74","96","98","85","111","104"],
"ap":["1","2","3","4","5","6","7","8","9","10"],
"as":["69","67","62","124","89","105","97","107","113","124"]
}
The only values that I will need to display & update to the table will be coming from keys: "ls" "lp" "v" "h" "l" "bs" "as"
I don't really need a detailed coded response, but I'd never refuse it, either. Mainly just looking for thoughts so I don't have to switch to a different way, later.
I'll also attach an screen capture of an Excel file showing about how the cells will be laid out. screenshot of desired layout
Have you tried using a JSON decoder, built in to Swift?
Create an appropriate using model, using the Decodable protocol. The property names and data types must match the incoming JSON exactly
struct IncomingData: Decodable {
{
let ls: [String],
let lp: [String],
let h: String,
let as: [String]
}
The set up the parse like this:
func parseJSON(_ incomingData: Data) -> IncomingData? {
let decoder = JSONDecoder()
do {
let parsedData = try decoder.decode(IncomingData.self, from: incomingData)
print("as: \(as)"
return parsedData
}
catch {
self.delegate?.didFailWithError(error: error)
return nil
}
}

low speed working with swiftyJSON

I'm using SwiftyJSON pod and I got a JSON like this :
I'm using a simple for loop in this JSON. but the problem is that it has a very low speed, it takes about 4 seconds to do this for loop.
this my code :
for _ in jsonDariafti!["contacts"]{
if (jsonDariafti!["contacts"][i]["name"].stringValue.range(of: txtSearch.text!) != nil) {
arrayINdexPathSearch.append(i)
}
i = i + 1
}
I'm trying to do a search in this JSON. for each word that user typed he has to wait about 4 seconds. I use below for loop and change it to an array and then searched in array. it works fine. but still very low in the initial for loop, it takes about 5 seconds :
for _ in jsonDariafti!["contacts"]{
arraySearch.append(jsonDariafti!["contacts"][i]["name"].stringValue)
i = i + 1
}
is there any better way using JSONS? or any advice using JSONs?
edit :
this is how I get JSON:
#import SwiftyJSON
#import Alamofire
func getdata(){
Alamofire.request("http://rsptint.ir/getSefaresh.php", method: .get).responseJSON {
response in
if response.result.isSuccess {
self.jsonDariafti = JSON(response.result.value!)
} else{
self.showAlert(message: "مشکل در ارتباط با اینترنت")
}
}
}
I would strongly recommend ditching SwiftyJSON and the string-and-dictionary based searching you're doing, and replacing this with much simpler code based on Codable.
struct Sefaresh: Codable {
let contacts: [Contact]
}
struct Contact: Codable {
let code, name, id, carton, tedad, litr, vahed, vazn: String
}
let sefaresh = try? JSONDecoder().decode(Sefaresh.self, from: jsonData)
your first loop then becomes
for (index, contact) in sefaresh.contacts.enumerated() {
if contact.stringValue.range(of: txtSearch.text!) != nil {
arrayIndexPathSearch.append(index)
}
}
and the second is simply
for contact in sefaresh.contacts {
arraySearch.append(contact.name)
}
This might not be an answer, but rather an elaboration on the question, but with a lot of mangling and tweaking I got the following to work:
import Cocoa
let jsonData = """
{ "contacts": [
{"code":"AC60-12","name":"\u{0633}\u{0641}\u{06cc}\u{062f} \u{067e}\u{0644}\u{06cc} \u{0627}\u{0648}\u{0631}\u{0647} \u{062a}\u{0627}\u{0646}-\u{06a9}\u{0627}\u{0631}\u{062a}\u{0646} 12\u{062a}\u{0627}\u{06cc}\u{06cc}","id":"5","carton":"1","tedad":"12","litr":"12","vahed":"","vazn":"1"},
{"code":"AC60-6","name":"\u{0633}\u{0641}\u{06cc}\u{062f} \u{067e}\u{0644}\u{06cc} \u{0627}\u{0648}\u{0631}\u{0647} \u{062a}\u{0627}\u{0646}-\u{06a9}\u{0627}\u{0631}\u{062a}\u{0646} 6\u{062a}\u{0627}\u{06cc}\u{06cc}","id":"4","carton":"1","tedad":"6","litr":"6","vahed":"","vazn":"1"},
{"code":"ME1019","name":"Brilliant White \u{06cc}\u{06a9} \u{0644}\u{06cc}\u{062a}\u{0631}\u{06cc} ME 1019","id":"516","carton":"1","tedad":"6","litr":"6","vahed":"","vazn":"1"}]
}
""".data(using: .utf8)!
struct Sefaresh: Codable {
let contacts: [Contact]
}
struct Contact: Codable {
let code, name, id, carton, tedad, litr, vahed, vazn: String
}
do {
let sefaresh = try JSONDecoder().decode(Sefaresh.self, from: jsonData)
print(sefaresh)
} catch {
print(error)
}
This looks even more horrible, but it might also point out some of the problems you encountered. It might have to do with the typical encoding of a Playground (which is UTF-8), but even then it compiles properly and the result contains a lot of characters which seem to originate from the same script as you posted.
While getting there I had to cope with a lot of errors during String parsing like
Expected hexadecimal code in braces after unicode escape
and I was only able to correct that manually, any search/replace pattern would fail and even after the introduction of the curly braces the editor would sometimes only search like a drag. You seem to have hit upon something that can thoroughly confuse Xcode or even the Swift String parser in general (though much less likely, since the Playground runs very quickly once it compiles without error).
I am not sure who or what created your escaped JSON in the first place, but if it was a language which still assumes that your unicode characters should all fit within 16 bits that might explain something.
Would you care to elaborate what created your JSON in the first place? Given my experience with the Playground (it became completely unresponsive until I started introducing newlines) I can easily believe that you triggered some edge case of Swifts String parsing. Given so many escapes in a form that Swift might not understand I am pretty sure anything might happen.
If it is possible to introduce newlines after the end of your contacts that could help and of course the braces would make all the difference since they seem to be the way Swift expects your escapes to come. Maybe some other encoding (like UTF-16?) might help, but I am not versed well enough with the workings of non-latin scripts to really be able to tell.

Simple modification of JSON object without serialization gets "cannot borrow immutable borrowed content as mutable"

I have a JSON encoded object in Rust 1.6.0. I want to decode it from JSON, change the value of one key, and convert it back to a JSON encoded string again. I don't want to write a struct to hold the data.
I am using rustc_serialize, which mostly seems to be built around serializing structs and automatically doing that, but I just want a simple JSON modification.
json_contents is a String that has the original, encoded JSON object.
let new_value = json::Json::from_str(&format!("[\"http://localhost:{}\"]", port)).unwrap();
let mut myjson_0 = json::Json::from_str(&json_contents).unwrap();
let mut myjson = tilejson_0.as_object().unwrap();
myjson.insert("mykey".to_owned(), new_value);
let new_json: String = json::encode(&myjson).unwrap();
However I get the following error:
src/main.rs:53:5: 53:13 error: cannot borrow immutable borrowed content `*myjson` as mutable
src/main.rs:53 myjson.insert("mykey".to_owned(), new_value);
^~~~~~
error: aborting due to previous error
How can I compile this? Is there a better, simpler, JSON Rust library I can use?
Some debugging has fixed this problem for me:
I replaced this code:
let mut myjson = tilejson_0.as_object().unwrap();
With this, to ensure that I had the type that I thought I had:
let mut myjson: BTreeMap<String, json::Json> = tilejson_0.as_object().unwrap();
and I got this compiler error:
src/main.rs:52:54: 52:85 error: mismatched types:
expected `collections::btree::map::BTreeMap<collections::string::String, rustc_serialize::json::Json>`,
found `&collections::btree::map::BTreeMap<collections::string::String, rustc_serialize::json::Json>`
(expected struct `collections::btree::map::BTreeMap`,
found &-ptr) [E0308]
src/main.rs:52 let mut myjson: BTreeMap<String, json::Json> tilejson_0.as_object().unwrap();
Clearly I was wrong. Rather than an owned BTreeMap, I had a reference to one, &BTreeMap.
The solution was to change the line to this:
let mut myjson = tilejson_0.as_object().unwrap().to_owned();
And everything compiled and worked (so far)

Parsing JSON with SwiftyJSON

I'm having trouble parsing the following JSON file with SwiftyJSON. I've looked around the web and tried different suggested solutions with no luck.
Here is the JSON:
{'info-leag':{'Status':1,'Name':'Testing Name','url-lig':'test.testing.com','uid':'12345'}}
And my relevant code:
//initializes request
let request = NSURLRequest(URL: url!)
NSURLConnection.sendAsynchronousRequest(request, queue: NSOperationQueue.currentQueue()) { response, maybeData, error in
if let data = maybeData {
let json = JSON(data: data)
//stores data as UTF8 String
let contents = NSString(data:data, encoding:NSUTF8StringEncoding)
The first part seems to work fine, I am able to get the JSON and save it as data, at the bottom I converted it to a string to make sure that I was getting the right information, I then later print it to make sure.
I tried different things like:
let name = json["info-league"]["Name"] //can't seem to get the context
I'm trying to get the Name and uid to be saved as 2 strings as well as the Status as an int.
Thanks!
Once you've made your JSON valid like this:
{"info-league":{"Status":1,"Name":"Testing Name","url-lig":"test.testing.com","uid":"12345"}}
you will be able to use your example, it works (I just tested):
let name = json["info-league"]["Name"]
but it's better to use SwiftyJSON types:
let name = json["info-league"]["Name"].string
let status = json["info-league"]["Status"].int
so your variables are of known types for later use.
If you don't do this they will be of type JSON, a type created by SwiftyJSON, and you will have to cast them later (not a problem, depends how you're organised in your code).
Try:
let name = json["info-league"]["Name"].string