Parsing complex JSON where data and "column headers" are in separate arrays - json

Question

I have the following JSON data I get from an API:
{"datatable":
{"data" : [
["John", "Doe", "1990-01-01", "Chicago"],
["Jane", "Doe", "2000-01-01", "San Diego"]
],
"columns": [
{ "name": "First", "type": "String" },
{ "name": "Last", "type": "String" },
{ "name": "Birthday", "type": "Date" },
{ "name": "City", "type": "String" }
]}
}
A later query could result the following:
{"datatable":
{"data" : [
["Chicago", "Doe", "John", "1990-01-01"],
["San Diego", "Doe", "Jane", "2000-01-01"]
],
"columns": [
{ "name": "City", "type": "String" },
{ "name": "Last", "type": "String" },
{ "name": "First", "type": "String" },
{ "name": "Birthday", "type": "Date" }
]
}
}
The order of the colums seems to be fluid.
I initially wanted to decode the JSON with JSONDecoder, but for that I need the data array to be a dictionary and not an array.
The only other method I could think of was to convert the result to a dictionary with something like:
extension String {
func convertToDictionary() -> [String: Any]? {
if let data = data(using: .utf8) {
return try? JSONSerialization.jsonObject(with: data, options: []) as? [String: Any]
}
return nil
}
}
This will cause me however to have a lot of nested if let statements like if let x = dictOfStr["datatable"] as? [String: Any] { ... }.
Not to mention the subsequent looping through the columns array to organize the data.
Is there a better solution?
Thanks

Answer 1

You could still use JSONDecoder, but you'd need to manually decode the data array.
To do that, you'd need to read the columns array, and then decode the data array using the ordering that you got from the columns array.
This is actually a nice use case for KeyPaths. You can create a mapping of columns to object properties, and this helps avoid a large switch statement.
So here's the setup:
struct DataRow {
var first, last, city: String?
var birthday: Date?
}
struct DataTable: Decodable {
var data: [DataRow] = []
// coding key for root level
private enum RootKeys: CodingKey { case datatable }
// coding key for columns and data
private enum CodingKeys: CodingKey { case data, columns }
// mapping of json fields to properties
private let fields: [String: PartialKeyPath<DataRow>] = [
"First": \DataRow.first,
"Last": \DataRow.last,
"City": \DataRow.city,
"Birthday": \DataRow.birthday ]
// I'm actually ignoring here the type property in JSON
private struct Column: Decodable { let name: String }
// init ...
}
Now the init function:
init(from decoder: Decoder) throws {
let root = try decoder.container(keyedBy: RootKeys.self)
let inner = try root.nestedContainer(keyedBy: CodingKeys.self, forKey: .datatable)
let columns = try inner.decode([Column].self, forKey: .columns)
// for data, there's more work to do
var data = try inner.nestedUnkeyedContainer(forKey: .data)
// for each data row
while !data.isAtEnd {
let values = try data.decode([String].self)
var dataRow = DataRow()
// decode each property
for idx in 0..<values.count {
let keyPath = fields[columns[idx].name]
let value = values[idx]
// now need to decode a string value into the correct type
switch keyPath {
case let kp as WritableKeyPath<DataRow, String?>:
dataRow[keyPath: kp] = value
case let kp as WritableKeyPath<DataRow, Date?>:
let dateFormatter = DateFormatter()
dateFormatter.dateFormat = "YYYY-MM-DD"
dataRow[keyPath: kp] = dateFormatter.date(from: value)
default: break
}
}
self.data.append(dataRow)
}
}
To use this, you'd use the normal JSONDecode way:
let jsonDecoder = JSONDecoder()
let dataTable = try jsonDecoder.decode(DataTable.self, from: jsonData)
print(dataTable.data[0].first) // prints John
print(dataTable.data[0].birthday) // prints 1990-01-01 05:00:00 +0000
EDIT
The code above assumes that all the values in a JSON array are strings and tries to do decode([String].self). If you can't make that assumption, you could decode the values to their underlying primitive types supported by JSON (number, string, bool, or null). It would look something like this:
enum JSONVal: Decodable {
case string(String), number(Double), bool(Bool), null, unknown
init(from decoder: Decoder) throws {
let container = try decoder.singleValueContainer()
if let v = try? container.decode(String.self) {
self = .string(v)
} else if let v = try? container.decode(Double.self) {
self = .number(v)
} else if ...
// and so on, for null and bool
}
}
Then, in the code above, decode the array into these values:
let values = try data.decode([JSONValue].self)
Later when you need to use the value, you can examine the underlying value and decide what to do:
case let kp as WritableKeyPath<DataRow, Int?>:
switch value {
case number(let v):
// e.g. round the number and cast to Int
dataRow[keyPath: kp] = Int(v.rounded())
case string(let v):
// e.g. attempt to convert string to Int
dataRow[keyPath: kp] = Int((Double(str) ?? 0.0).rounded())
default: break
}

Answer 2

It appears that the data and columns values gets encoded in the same order so using that we can create a dictionary for column and array of values where each array is in the same order.
struct Root: Codable {
let datatable: Datatable
}
struct Datatable: Codable {
let data: [[String]]
let columns: [Column]
var columnValues: [Column: [String]]
enum CodingKeys: String, CodingKey {
case data, columns
}
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
data = try container.decode([[String]].self, forKey: .data)
columns = try container.decode([Column].self, forKey: .columns)
columnValues = [:]
data.forEach {
for i in 0..<$0.count {
columnValues[columns[i], default: []].append($0[i])
}
}
}
}
struct Column: Codable, Hashable {
let name: String
let type: String
}
Next step would be to introduce a struct for the data

Answer 3

The way I would do it is to create two model objects and have them both conform to the Codable protocol like so:
struct Datatable: Codable {
let data: [[String]]
let columns: [[String: String]]
}
struct JSONResponseType: Codable {
let datatable: Datatable
}
Then in your network call I'd decode the json response using JSONDecoder():
let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
guard let decodedData = try? decoder.decode(JSONResponseType.self, from: data) else {
// handle decoding failure
return
}
// do stuff with decodedData ex:
let datatable = decodedData.datatable
...
data in this case is the result from the URLSessionTask.
Let me know if this works.

Answer 4

Maybe try to save the given input inside a list of user objects? This way however the JSON is structured you can add them in the list and handle them after anyway you like. Maybe an initial alphabetical ordering after name would also help with the display order of users.
Here is a sample I wrote, instead of logging the info you can add a new UserObject to the list with the currently printed information.
let databaseData = table["datatable"]["data"];
let databaseColumns = table["datatable"]["columns"];
for (let key in databaseData) {
console.log(databaseColumns[0]["name"] + " = " + databaseData[key][0]);
console.log(databaseColumns[1]["name"] + " = " + databaseData[key][1]);
console.log(databaseColumns[2]["name"] + " = " + databaseData[key][2]);
console.log(databaseColumns[3]["name"] + " = " + databaseData[key][3]);
}

Answer 5

The only thing I could think of is:
struct ComplexValue {
var value:String
var columnName:String
var type:String
}
struct ComplexJSON: Decodable, Encodable {
enum CodingKeys: String, CodingKey {
case data, columns
}
var data:[[String]]
var columns:[ColumnSpec]
var processed:[[ComplexValue]]
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
data = (try? container.decode([[String]].self, forKey: .data)) ?? []
columns = (try? container.decode([ColumnSpec].self, forKey: .columns)) ?? []
processed = []
for row in data {
var values = [ComplexValue]()
var i = 0
while i < columns.count {
var item = ComplexValue(value: row[i], columnName: columns[i].name, type: columns[i].type)
values.append(item)
i += 1
}
processed.append(values)
}
}
}
struct ColumnSpec: Decodable, Encodable {
enum CodingKeys: String, CodingKey {
case name, type
}
var name:String
var type:String
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
name = (try? container.decode(String.self, forKey: .name)) ?? ""
type = (try? container.decode(String.self, forKey: .type)) ?? ""
}
}
Now you would have the processed variable which would contain formatted version of your data. Well, formatted might not be the best word, given that structure is completely dynamic, but at least whenever you extract some specific cell you would know its value, type and its column name.
I don't think you can do anything more specific than this without extra details about your APIs.
Also, please note that I did this in Playground, so some tweaks might be needed to make the code work in production. Although I think the idea is clearly visible.
P.S. My implementation does not deal with "datatable". Should be straightforward to add, but I thought it would only increase the length of my answer without providing any benefits. After all, the challenge is inside that field :)