I'm looking for some guidance here on an issue I'm encountering. I am polling an API for stock data and instead of getting back an array or formatted JSON I get chunks of it. Each block of JSON seems to be properly formatted, but the JSON decoders are looking for a properly formatted array.
Prior to using combine I had a different working. I had to parse the data to remove 4 trailing characters, then I was left with lines of JSON that I could split by newline and then do a forEach loop on to parse.
This was messy but it works. The issue now is I'm polling streaming data for live quote updating in my UI. I need to have the JSON chunks read into my class, stored as a variable that will update the UI. I am not worried about all of that code once I can figure out how to get the data read properly.
I know my app is receiving the data because I see the stats in the debugger for network. I also test the same API call in a browser and see what data my app is receiving. I just don't know how to break it up in the combine framework.
I have been working with something like this so far:
return URLSession.shared.dataTaskPublisher(for: request)
.map { $0.data }
.handleEvents(receiveSubscription: { print("Receive subscription: \($0)") },
receiveOutput: { print("Receive output: \($0)") },
receiveCompletion: { print("Receive completion: \($0)") },
receiveCancel: { print("Receive cancel") },
receiveRequest: { print("Receive request: \($0)") })
//.decode(type: CryptoDataContainer.self, decoder: JSONDecoder())
.decode(type: Quote.self, decoder: JSONDecoder())
.receive(on: DispatchQueue.main)
.eraseToAnyPublisher()
}
The Quote struct is very basic:
struct Quote: Codable {
public var Symbol: String
public var Last: Double
public var Bid: Double
public var Ask: Double
public var Close: Double
}
JSON data is posted below, and it streams. I get a few chunks of it at a time, but if I can split by newline or by everything {.*} to get each JSON chunk and decode individually it would work great.
I'm really looking for Last price and Symbol out of the JSON. I could even just look for Last and start one individual streaming session for each symbol I need.
I will also need to split these up using a subject to multicast each publisher to multiple subscribers but I wanted to get the basic functionality working before complicating matters.
Update: From another post I found that the problem is the JSON formatting - The API is returning invalid JSON that is not formatted into an array - it is missing the surrounding []. This is why it doesn't parse. Another option may be to simply convert to string, add square brackets around all of the JSON passed down from the server (there is nothing but objects).. and then parse it correctly? I'm not sure how to do this within a combine map but that seems to be the path forward.
Here’s an example of the map I’m trying to use currently, since the data comes in as json chunks I need to parse them each individually.
.map ({
let stringRepresentation = String(data: $0, encoding: .utf8)
let outputJson = stringRepresentation!.split(whereSeparator: \.isNewline).map(String.init)
if outputJson.count > 1 {
print("DEBUG: outputJson.count > 1 code running..")
let formattedOutputJson:String = outputJson.joined(separator: ",")
let finalJson:String = "{\"quotes\":[" + formattedOutputJson + "]}"
let data:Data? = finalJson.data(using: .utf8)
return data!
} else {
print("DEBUG: outputJson.count = 1 (Else) code running..")
let finalJson:String = "{\"quotes\":[" + outputJson[0] + "]}"
let data:Data? = finalJson.data(using: .utf8)
return data!
}
})
Here is a better example of the JSON formatting in chunks I’m talking about:
{"heartbeat":4,"timestamp":"\/Date(1622898090819)\/"}
{"heartbeat":5,"timestamp":"\/Date(1622898095819)\/"}
{"heartbeat":6,"timestamp":"\/Date(1622898100819)\/"}
{"heartbeat":7,"timestamp":"\/Date(1622898105819)\/"}
{"heartbeat":8,"timestamp":"\/Date(1622898110819)\/"}
If I can even capture the heartbeat number as an Int to use I would be well on my way to getting my code working. The issue is I get zero data out of this. I know I’m connected from packet captures and various other tests. The data is getting to my system, and to the Xcode simulated app, I just don’t think I’m getting it to a point where I can print to the console correctly to display it. I have been working on that prior to attempting to update the UI directly.
{"Description":"E-Mini NASDAQ-100 Jun 2021","PreviousClose":13529.25,"PreviousVolume":552987,"AssetType":"FUTURE","CountryCode":"United States","Halted":false,"Bid":13513.75,"BidSize":3,"Ask":13514.25,"AskSize":1,"Currency":"USD","LastSize":1,"LastVenue":"CME","SymbolRoot":"NQ","Exchange":"CME","LastTradingDate":"2021-06-18","DisplayType":3,"FractionalDisplay":false,"MinPrice":1258200,"PointValue":20,"TradeTime":"2021-06-04T02:40:20.0000000+00:00","MaxPrice":1447600,"MinMove":25,"High52Week":14064,"Open":13531.75,"Low52Week":9591.75,"High":13541.75,"Low":13467.75,"Close":13513.75,"Symbol":"NQM21","Volume":23877,"DailyOpenInterest":231736,"ExpirationDate":"2021-06-18","Last":13513.75,"NameExt":"(D)","DataFeed":"TradeStation","IsDelayed":true,"Restrictions":[null],"VWAP":13499.877658013,"NetChange":-15.50,"NetChangePct":-0.1145665872091948925476282900,"High52WeekTimeStamp":"2021-04-29T00:00:00.0000000+00:00","Low52WeekTimeStamp":"2020-06-11T00:00:00.0000000+00:00","PreviousClosePriceDisplay":"13529.25","BidPriceDisplay":"13513.75","AskPriceDisplay":"13514.25","MinPriceDisplay":"1258200.00","MaxPriceDisplay":"1447600.00","High52WeekPriceDisplay":"14064.00","OpenPriceDisplay":"13531.75","Low52WeekPriceDisplay":"9591.75","HighPriceDisplay":"13541.75","LowPriceDisplay":"13467.75","ClosePriceDisplay":"13513.75","LastPriceDisplay":"13513.75","VWAPDisplay":"13499.88","FirstNoticeDate":"","StrikePrice":0,"TickSizeTier":0,"Underlying":"","StrikePriceDisplay":"0.00"}
{"Last":4188.5,"NameExt":"(D)","DataFeed":"TradeStation","IsDelayed":true,"Restrictions":[null],"VWAP":4184.92506004117,"Description":"E-mini S&P 500 Jun 2021","PreviousClose":4191.25,"PreviousVolume":1384271,"AssetType":"FUTURE","CountryCode":"United States","Halted":false,"Bid":4188.25,"BidSize":39,"Ask":4188.5,"AskSize":23,"Currency":"USD","LastSize":1,"LastVenue":"CME","SymbolRoot":"ES","Exchange":"CME","LastTradingDate":"2021-06-18","DisplayType":3,"FractionalDisplay":false,"MinPrice":3898,"PointValue":50,"TradeTime":"2021-06-04T02:40:10.0000000+00:00","MaxPrice":4484,"MinMove":25,"High52Week":4238.25,"Open":4190.75,"Low52Week":2977.8999,"Hi{"Volume":23880,"TradeTime":"2021-06-04T02:40:23.0000000+00:00","VWAP":13499.879455674,"Symbol":"NQM21"}
{"AskSize":18,"Symbol":"ESM21"}
{"Bid":13514.25,"BidSize":1,"AskSize":1,"BidPriceDisplay":"13514.25","Symbol":"NQM21","VWAP":13499.8800582884,"Volume":23881,"TradeTime":"2021-06-04T02:40:25.0000000+00:00"}
{"VWAP":4184.92521199492,"AskSize":19,"Volume":47807,"TradeTime":"2021-06-04T02:40:27.0000000+00:00","Symbol":"ESM21"}
{"Bid":13514,"BidSize":3,"BidPriceDisplay":"13514.00","Symbol":"NQM21","TradeTime":"2021-06-04T02:40:27.0000000+00:00","Ask":13514.75,"VWAP":13499.8806713352,"Close":13514.5,"Last":13514.5,"Volume":23882,"NetChange":-14.75,"NetChangePct":-0.1090230426668144945211301400,"AskPriceDisplay":"13514.75","ClosePriceDisplay":"13514.50","LastPriceDisplay":"13514.50"}
{"AskSize":18,"Symbol":"ESM21"}
Currently I am working on a MacOS financial market trading application that could receive 10-20 websocket messages per second containing JSON data. For now, let's assume that once the data has been displayed in a table and a new message has been received, the previous data is no longer needed. What would be the recommended, and most efficient way to get the data into the table, and to keep updating it as fast as possible?
I have the websocket connection setup and working properly (using SocketRocket. Data is coming in as it should. But I really don't want to continue further without having a better understanding of the most efficient way to present that data into the table. I was reading about DictionaryArrayController, and maybe utilizing a database library, but I think the latter would would only add unnecessary overhead for what I am trying to do.
Below is a sample of the data that I am receiving.
{
"id":"5267",
"pt":"T",
"ls":["1"],
"lp":["11968"],
"t":["1571824228"],
"v":"133758",
"h":"11981",
"l":"11928",
"bp":["0","1","2","3","4","5","6","7","8","9"],
"bs":["14","73","87","66","74","96","98","85","111","104"],
"ap":["1","2","3","4","5","6","7","8","9","10"],
"as":["69","67","62","124","89","105","97","107","113","124"]
}
The only values that I will need to display & update to the table will be coming from keys: "ls" "lp" "v" "h" "l" "bs" "as"
I don't really need a detailed coded response, but I'd never refuse it, either. Mainly just looking for thoughts so I don't have to switch to a different way, later.
I'll also attach an screen capture of an Excel file showing about how the cells will be laid out. screenshot of desired layout
Have you tried using a JSON decoder, built in to Swift?
Create an appropriate using model, using the Decodable protocol. The property names and data types must match the incoming JSON exactly
struct IncomingData: Decodable {
{
let ls: [String],
let lp: [String],
let h: String,
let as: [String]
}
The set up the parse like this:
func parseJSON(_ incomingData: Data) -> IncomingData? {
let decoder = JSONDecoder()
do {
let parsedData = try decoder.decode(IncomingData.self, from: incomingData)
print("as: \(as)"
return parsedData
}
catch {
self.delegate?.didFailWithError(error: error)
return nil
}
}
I am parsing a csv file into core data. I am parsing the file and saving the data into CD at the same time in the one function. I am interested to get some feedback if this is a good approach? It does work, the data is saved. To make things easy for now all the entity attributes are strings. I am new to swift so am looking to make sure I am not making any newbie mistakes. Here is the function.
I do have one question. I am not sure now to save data from the file to an attribute in a different entity in the same data model that has a relationship to the Asteroids entity. A resources and amount attributes in a Resources entity for example.
func parseAsteroidCSVForCD(){
let context = getContext()
let entity = NSEntityDescription.entity(forEntityName: "Asteroid", in: context)
let manageObj = NSManagedObject(entity: entity!, insertInto: context)
// Path variable to the asteroids.csv file
let path = Bundle.main.path(forResource: "asteroids", ofType: "csv")!
// Parse the file in a do catch
do {
// Use the paser to pull out the rows
let csv = try CSV(contentsOfURL: path)
let rows = csv.rows
// Go through each row to store the id, name, size, distance and value
for row in rows {
let id = row["id"]!
let name = row["name"]!
let size = row["size"]!
let distance = row["distancefromEarth"]!
let value = row["value"]!
// load into Core Data here
manageObj.setValue(id, forKey: "id")
manageObj.setValue(name, forKey: "name")
manageObj.setValue(size, forKey: "size")
manageObj.setValue(value, forKey: "value")
manageObj.setValue(distance, forKey: "distance")
do {
try context.save()
print("New Asteroid has been created")
} catch {
print(error.localizedDescription)
}
}
} catch let err as NSError {
print(err.debugDescription)
}
}
With this logic, all CSV rows will not be inserted. Because you create only one "manageObj" and keep on setting the value of every row to it. So finally you will have only one "manageObj" with the last row value.
the entity creation should be within the for loop, so that for every row, a new entity will be created. And do not save within the for loop. Once for loop is finished, save it.. A single save..
I'm new to coding. I have JSON request which returns different parameters, for example "unitid" and "buldingid".
i want to store the "buldingid" by NSUserDfaults. I use code below for storing it, but every time I get nil response for "buldingid".
reading:
NSUserDefaults.standardUserDefaults().setObject(buildingid, forKey: "buildingidKey")
NSUserDefaults.standardUserDefaults().synchronize()
writing:
let buildingid: [NSString]? = NSUserDefaults.standardUserDefaults().objectForKey("buildingidKey") as? [NSString]
i saved it in a label then stored in NSUserDfaults but it doesn't work (error: Attempt to insert non-property list object)
what should I do to get the correct response for "buldingid"?
Thanks!
I suppose from your code that buildingid is a String, so something like this should work:
NSUserDefaults.standardUserDefaults().setObject(String(buildingid), forKey: "buildingidKey")
NSUserDefaults.standardUserDefaults().synchronize()
Retrieving it should be done like this:
let buildingid = NSUserDefaults.standardUserDefaults().stringForKey("buildingidKey")
I'm having trouble parsing the following JSON file with SwiftyJSON. I've looked around the web and tried different suggested solutions with no luck.
Here is the JSON:
{'info-leag':{'Status':1,'Name':'Testing Name','url-lig':'test.testing.com','uid':'12345'}}
And my relevant code:
//initializes request
let request = NSURLRequest(URL: url!)
NSURLConnection.sendAsynchronousRequest(request, queue: NSOperationQueue.currentQueue()) { response, maybeData, error in
if let data = maybeData {
let json = JSON(data: data)
//stores data as UTF8 String
let contents = NSString(data:data, encoding:NSUTF8StringEncoding)
The first part seems to work fine, I am able to get the JSON and save it as data, at the bottom I converted it to a string to make sure that I was getting the right information, I then later print it to make sure.
I tried different things like:
let name = json["info-league"]["Name"] //can't seem to get the context
I'm trying to get the Name and uid to be saved as 2 strings as well as the Status as an int.
Thanks!
Once you've made your JSON valid like this:
{"info-league":{"Status":1,"Name":"Testing Name","url-lig":"test.testing.com","uid":"12345"}}
you will be able to use your example, it works (I just tested):
let name = json["info-league"]["Name"]
but it's better to use SwiftyJSON types:
let name = json["info-league"]["Name"].string
let status = json["info-league"]["Status"].int
so your variables are of known types for later use.
If you don't do this they will be of type JSON, a type created by SwiftyJSON, and you will have to cast them later (not a problem, depends how you're organised in your code).
Try:
let name = json["info-league"]["Name"].string