Umlaut in URL fails Codable (Swift) - what to do? - json

Using Swift-4.1, Xcode-9.3.1, iOS-11.3.1
I use the Codable protocol to decode a JSON-file. Everything works, except until the moment where I have an Internationalised Domain-Name (in this case, with a German Umlaut "ä") in a URL (example: http://www.rhätische-zeitung.ch).
This leads to a decoder-error inside the following code:
func loadJSON(url: URL) -> Media? {
do {
let data = try Data(contentsOf: url)
let decoder = JSONDecoder()
let media = try decoder.decode(Media.self, from: data)
return media
} catch {
print("error:\(error)")
}
return nil
}
The error-message is:
The Codable protocol does not seem to be able to decode this URL form
my JSON-file into the needed Struct.
Here is the Struct:
struct Media: Codable {
var publisher: [MediaPublisher]
}
struct MediaPublisher: Codable {
var title: String?
var homepage_url: URL?
}
And here is the JSON-excerpt:
{
"publisher": [
{
"title" : "Rhätische-Zeitung",
"homepage_url" : "http://www.rhätische-zeitung.ch",
}
]
}
Since the JSON-file is coming from the outside, I have no control over the content. And therefore, replacing the URL inside the JSON is not an option !
(Therefore, I cannot replace the URL inside the JSON to an accepted Internationalized Form suche as: www.xn--rhtische-zeitung-wnb.ch) !!
I know that there are techniques to place a custom initialiser into the Struct-definition (see my trials below...) - but since new to Codable, I don't know how to do that for this current URL-Umlaut problem. The custom-initialiser I placed below does return nil for the URL at question. What do I need to change ??
Or is there another way of making this JSON-decoding of an URL with Umlaut work ??
Here is the Struct, this time with a custom initialiser:
(at least with this, I can get rid of the error-message above... But the URL is now nil it seems and that is not what I want either)
struct Media: Codable {
var publisher: [MediaPublisher]
}
struct MediaPublisher: Codable {
var title: String?
var homepage_url: URL?
// default initializer
init(title: String?, homepage_url: URL?) {
self.title = title
self.homepage_url = homepage_url
}
// custom initializer
init(from decoder: Decoder) throws {
let map = try decoder.container(keyedBy: CodingKeys.self)
self.title = try? map.decode(String.self, forKey: .title)
self.homepage_url = try? map.decode(URL.self, forKey: .homepage_url)
}
private enum CodingKeys: CodingKey {
case title
case homepage_url
}
}

I just stumbled over the same problem. The solution is actually fairly simple and works well in my brief testing.
The idea is to not let the decoder decode the value as an URL, because it expects the string behind it to be in of a certain format. What we can do to circumvent this is to decode the value as a string directly and convert that manually into a URL.
I wrote a little extension that does that job for me:
extension KeyedDecodingContainer {
/// Decodes the string at the given key as a URL. This allows for special characters like umlauts to be decoded correctly.
func decodeSanitizedURL(forKey key: KeyedDecodingContainer<K>.Key) throws -> URL {
let urlString = try self.decode(String.self, forKey: key)
// Sanitize string and attempt to convert it into a valid url
if let urlString = urlString.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed), let url = URL(string: urlString) {
return url
}
// Throw an error as the URL could not be decoded
else {
throw DecodingError.dataCorruptedError(forKey: key, in: self, debugDescription: "Could not decode \(urlString)")
}
}
}
This allows for a streamlined use in the init method.
required init(from decoder: Decoder) throws {
let values = try decoder.container(keyedBy: CodingKeys.self)
self.url = try values.decodeSanitizedURL(forKey: .url)
}
Hope that helps, even if the question is a little bit older.

Related

Swift Codable: subclass JSONDecoder for custom behavior

I'm having an inconsistent API that might return either a String or an Number as a part of the JSON response.
The dates also could be represented the same way as either a String or a Number, but are always an UNIX timestamp (i.e. timeIntervalSince1970).
To fix the issue with the dates, I simply used a custom JSONDecoder.DateDecodingStrategy:
decoder.dateDecodingStrategy = JSONDecoder.DateDecodingStrategy.custom({ decoder in
let container = try decoder.singleValueContainer()
if let doubleValue = try? container.decode(Double.self) {
return Date(timeIntervalSince1970: doubleValue)
} else if let stringValue = try? container.decode(String.self),
let doubleValue = Double(stringValue) {
return Date(timeIntervalSince1970: doubleValue)
}
throw DecodingError.dataCorruptedError(in: container,
debugDescription: "Unable to decode value of type `Date`")
})
However, no such customization is available for the Int or Double types which I'd like to apply it for.
So, I have to resort to writing Codable initializers for each of the model types that I'm using.
The alternative approach I'm looking for is to subclass the JSONDecoder and override the decode<T>(_ type: T.Type, from data: Data) throws -> T where T : Decodable method.
In that method I'd like to "inspect" the type T that I'm trying to decode to and then, if the base implementation (super) fails, try to decode the value first to String and then to the T (the target type).
So far, my initial prototype looks like this:
final class CustomDecoder: JSONDecoder {
override func decode<T>(_ type: T.Type, from data: Data) throws -> T where T : Decodable {
do {
return try super.decode(type, from: data)
} catch {
if type is Int.Type {
print("Trying to decode as a String")
if let decoded = try? super.decode(String.self, from: data),
let converted = Int(decoded) {
return converted as! T
}
}
throw error
}
}
}
However, I found out that the "Trying to decode as a String" message is never printed for some reason, even though the control reaches the catch stage.
I'm happy to have that custom path only for Int and Double types, since the T is Codable and that doesn't guarantee ability to initialize a value with the String, however, I of course welcome a more generalized approach.
Here's the sample Playground code that I came up with to test my prototype. It can be copy-pasted directly into the Playground and works just fine.
My goal is to have both jsonsample1 and jsonsample2 to produce the same result.
import UIKit
final class CustomDecoder: JSONDecoder {
override func decode<T>(_ type: T.Type, from data: Data) throws -> T where T : Decodable {
do {
return try super.decode(type, from: data)
} catch {
if type is Int.Type {
print("Trying to decode as a String")
if let decoded = try? super.decode(String.self, from: data),
let converted = Int(decoded) {
return converted as! T
}
}
throw error
}
}
}
let jsonSample1 =
"""
{
"name": "Paul",
"age": "38"
}
"""
let jsonSample2 =
"""
{
"name": "Paul",
"age": 38
}
"""
let data1 = jsonSample1.data(using: .utf8)!
let data2 = jsonSample2.data(using: .utf8)!
struct Person: Codable {
let name: String?
let age: Int?
}
let decoder = CustomDecoder()
let person1 = try? decoder.decode(Person.self, from: data1)
let person2 = try? decoder.decode(Person.self, from: data2)
print(person1 as Any)
print(person2 as Any)
What could be the reason for my CustomDecoder not working?
The primary reason that your decoder doesn't do what you expect is that you're not overriding the method that you want to be: JSONDecoder.decode<T>(_:from:) is the top-level method that is called when you call
try JSONDecoder().decode(Person.self, from: data)
but this is not the method that is called internally during decoding. Given the JSON you show as an example, if we write a Person struct as
struct Person: Decodable {
let name: String
let age: Int
}
then the compiler will write an init(from:) method which looks like this:
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
name = try container.decode(String.self, forKey: .name)
age = try container.decode(Int.self, forKey: .age)
}
Note that when we decode age, we are not calling a method on the decoder directly, but on a KeyedCodingContainer that we get from the decoder — specifically, the Int.Type overload of KeyedDecodingContainer.decode(_:forKey:).
In order to hook into the methods that are called during decode at the middle levels of a Decoder, you'd need to hook into its actual container methods, which is very difficult — all of JSONDecoder's containers and internals are private. In order to do this by subclassing JSONDecoder, you'd end up needing to pretty much reimplement the whole thing from scratch, which is significantly more complicated than what you're trying to do.
As suggested in a comment, you're likely better off either:
Writing Person.init(from:) manually by trying to decode both Int.self and String.self for the .age property and keeping whichever one succeeds, or
If you need to reuse this solution across many types, you can write a wrapper type to use as a property:
struct StringOrNumber: Decodable {
let number: Double
init(from decoder: Decoder) throws {
let container = try decoder.singleValueContainer()
do {
number = try container.decode(Double.self)
} catch (DecodingError.typeMismatch) {
let string = try container.decode(String.self)
if let n = Double(string) {
number = n
} else {
throw DecodingError.dataCorruptedError(in: container, debugDescription: "Value wasn't a number or a string...")
}
}
}
}
struct Person: Decodable {
let name: String
let age: StringOrNumber
}
You can also write StringOrNumber as an enum which can hold either case string(String) or case number(Double) if knowing which type of value was in the payload was important:
enum StringOrNumber: Decodable {
case number(Double)
case string(String)
init(from decoder: Decoder) throws {
let container = try decoder.singleValueContainer()
do {
self = try .number(container.decode(Double.self))
} catch (DecodingError.typeMismatch) {
let string = try container.decode(String.self)
if let n = Double(string) {
self = .string(string)
} else {
throw DecodingError.dataCorruptedError(in: container, debugDescription: "Value wasn't a number or a string...")
}
}
}
}
Though this isn't as relevant if you always need Double/Int access to the data, since you'd need to re-convert at the use site every time (and you call this out in a comment)

How to use JSONDecoder to decode JSON with unknown type?

Here is my scenario: I have a swift WebSocket server and a Javascript client. Over this same WebSocket I will be sending various objects that correspond to different Codable types. It is simple enough to decode if the correct type is known. The difficulty for me is to identify which type is being sent from the client. My first thought was to
use JSON that looks like the following:
{type: "GeoMarker",
data: {id: "2",
latitude: "-97.32432"
longitude: "14.35436"}
}
This way I would know to decode data using let marker = try decoder.decode(GeoMarker.self)
This seems to be straightforward, but for some reason, I just can't figure out how to extract the data object as JSON so that I can decode it using the GeoMarker type.
Another solution that I came up with was to create an intermediate type like so:
struct Communication: Decodable {
let message: String?
let markers: [GeoMarker]?
let layers: [GeoLayer]?
}
This way I would could send JSON with the following format:
{message: "This is a message",
markers: [{id: "2",
latitude: "-97.32432"
longitude: "14.35436"},
{id: "3",
latitude: "-67.32432"
longitude: "71.35436"}]
}
and use let com = try decoder.decode(Communication.self) and unwrap the optional message, markers, and layers variables. This works, but seems clunky, especially if I need more types. I will likely end up needing 8-10 after all is said and done.
I have thought through this, but don't feel like I have come up with a satisfactory solution. Would there be better approaches? Is there a standard for this kind of thing that I am unaware of?
----EDIT----
As a natural follow up, how would you go about encoding to that same JSON format, given the same circumstances above?
As your first option you can achieve it by custom decoding.
First create an enum with associated values for all of your possible data types.
struct GeoMarker: Decodable {
let id:Int
let latitude:Double
let longitude:Double
}
enum ResponseData {
case geoMarker(GeoMarker)
case none
}
Now provide custom decoding for your enum to parse all different types of your data objects.
extension ResponseData: Decodable{
enum CodingKeys: String, CodingKey {
case type = "type"
case data = "data"
}
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
let type = try container.decode(String.self, forKey: .type)
switch type {
case "GeoMarker":
let data = try container.decode(GeoMarker.self, forKey: .data)
self = .geoMarker(data)
default:
self = .none
}
}
}
You can use it something like this...
let json = """
{
"type": "GeoMarker",
"data": {
"id": 2,
"latitude": -97.32432,
"longitude": 14.35436
}
}
"""
let testRes = try? JSONDecoder().decode(ResponseData.self, from: json.data(using: .utf8)!)
if let testRes = testRes {
if case let ResponseData.geoMarker(geoMarker) = testRes {
print("\(geoMarker.id) \(geoMarker.latitude) \(geoMarker.longitude)")
}
}
To implement a custom encoder, use the following.
extension ResponseData: Encodable{
func encode(to encoder: Encoder) throws {
let container = encoder.container(keyedBy: CodingKeys.self)
if case let .geoMarker(geoMarker) = self {
try container.encode("Marker", forKey: .type)
try container.encode(geoMarker, forKey: .data)
}
}
You can use it like this:
let marker = GeoMarker(id:2, latitude: "-97.32432", longitude: "14.35436")
let encoder = JSONEncoder()
let data = try encoder.encode(.geoMarker(marker))
//Send data over WebSocket

Strings fetched with JSON-API to be converted from base64 to UTF8 in Swift

I am developing an iOS App that fetches Trivia Questions from Open Trivia Database (API)
After reading the docs and played around with it I think that the best solution is to use base64 encoding (since it seems to be supported in Swift). I have successfully fetched the data and parsed it into structs using a JSONParser. The problem that I have to solve is how to convert the values from base64 to UTF8. (The keys are read correctly, and therefore it maps to my structs)
My first idea was to use decoder.dataDecodingStrategy = .base64, but that does not seem to have any effect at all. And I am not really sure why.
Is that the right way to do it, or should I decode it myself afterwards when the strings are read in to structs?
In short, the result of the Parsing is a struct containing a responseCode as an Int and array containing structs representing the questions with the strings that I want to convert to UTF8 as members
My code for parsing looks like this:
let urlPath = "https://opentdb.com/api.php?amount=10&encode=base64"
let apiURL = URL(string: urlPath)!
URLSession.shared.dataTask(with: apiURL) { (data, response, error) in
guard let data = data else {return}
do{
let decoder = JSONDecoder()
decoder.dataDecodingStrategy = .base64
let questionData = try decoder.decode(Response.self, from: data)
print(questionData)
}catch let err{
print("Error", err)
}
}.resume()
Base64 encoding is used for properties you declared as Data, not as Strings, like so:
struct Response: Codable {
let someBaseEncodedString: Data
var someString: String? {
get {
return String(data: someBaseEncodedString, encoding: .utf8)
}
}
}
So, for the example you are giving, all the properties that are returned as a base64 encoded string should have the Data type in your struct, and then after that decoded as strings.
As suggested by other answers, you can decode Data or Base-64 String after JSONSerialization or JSONDecoder decoded the API results.
But if you prefer to write decoding initializer, you can make it as follows:
This may not be much different from your own Response, I guess.
struct Response: Codable {
var responseCode: Int
var results: [Result]
enum CodingKeys: String, CodingKey {
case responseCode = "response_code"
case results
}
}
To prepare to write a decoding initializer for Response, I would like to use some extensions:
extension KeyedDecodingContainer {
func decodeBase64(forKey key: Key, encoding: String.Encoding) throws -> String {
guard let string = try self.decode(String.self, forKey: key).decodeBase64(encoding: encoding) else {
throw DecodingError.dataCorruptedError(forKey: key, in: self,
debugDescription: "Not a valid Base-64 representing UTF-8")
}
return string
}
func decodeBase64(forKey key: Key, encoding: String.Encoding) throws -> [String] {
var arrContainer = try self.nestedUnkeyedContainer(forKey: key)
var strings: [String] = []
while !arrContainer.isAtEnd {
guard let string = try arrContainer.decode(String.self).decodeBase64(encoding: encoding) else {
throw DecodingError.dataCorruptedError(forKey: key, in: self,
debugDescription: "Not a valid Base-64 representing UTF-8")
}
strings.append(string)
}
return strings
}
}
Using these extensions above, you can define the Result type as follows:
extension Response {
struct Result: Codable {
var category: String
var type: String
var difficulty: String
var question: String
var correctAnswer: String
var incorrectAnswers: [String]
enum CodingKeys: String, CodingKey {
case category
case type
case difficulty
case question
case correctAnswer = "correct_answer"
case incorrectAnswers = "incorrect_answers"
}
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
self.category = try container.decodeBase64(forKey: .category, encoding: .utf8)
self.type = try container.decodeBase64(forKey: .type, encoding: .utf8)
self.difficulty = try container.decodeBase64(forKey: .difficulty, encoding: .utf8)
self.question = try container.decodeBase64(forKey: .question, encoding: .utf8)
self.correctAnswer = try container.decodeBase64(forKey: .correctAnswer, encoding: .utf8)
self.incorrectAnswers = try container.decodeBase64(forKey: .incorrectAnswers, encoding: .utf8)
}
}
}
(You have not mentioned if your Response (or other name?) is defined as a nested type or not, but I think you can rename or modify it yourself.)
With all things above, you can simply decode the API response as:
do {
let decoder = JSONDecoder()
let questionData = try decoder.decode(Response.self, from: data)
print(questionData)
} catch {
print("Error", error)
}
By the way, you say I think that the best solution is to use base64 encoding (since it seems to be supported in Swift), but is that really true?
Base-64 to Data is supported in JSONDecoder, but it is not what you expect. So, using another encoding can be a better choice.
But, anyway, JSON string can represent all unicode characters using only ASCII with \uXXXX or \uHHHH\uLLLL. So, I do not understand why the API designers do not provide an option Standard JSON Encoding. If you can contact to them, please tell them to provide the option, that may simplify many client side codes.

Swift - passing types dynamically to JSONDecoder

I am trying to decode from json objects with generic nested object, and for this I want to pass the type of class dynamically when decoding.
For example, my classes are EContactModel and ENotificationModel which extend ObjectModel (and :Codable)s. ENotificationModel can contain a nested ObjectModel (which can be a contact, notification or other objectmodel).
I have a dictionary of types like this:
static let OBJECT_STRING_CLASS_MAP = [
"EContactModel" : EContactModel.self,
"ENotificationModel" : ENotificationModel.self
...
]
My decoding init method in ENotificationModel looks like this:
required init(from decoder: Decoder) throws
{
try super.init(from: decoder)
let values = try decoder.container(keyedBy: CodingKeys.self)
...
//decode some fields here
self.message = try values.decodeIfPresent(String.self, forKey: .message)
...
//decode field "masterObject" of generic type ObjectModel
let cls = ObjectModelTypes.OBJECT_STRING_CLASS_MAP[classNameString]!
let t = type(of: cls)
print(cls) //this prints "EContactModel"
self.masterObject = try values.decodeIfPresent(cls, forKey: .masterObject)
print(t) //prints ObjectModel.Type
print(type(of: self.masterObject!)) //prints ObjectModel
}
I also tried passing type(of: anObjectInstanceFromADictionary) and still not working, but if I pass type(of: EContactModel()) it works. I cannot understand this, because both objects are the same (ie. instance of EContactModel)
Is there a solution for this?
You could declare your object models with optional variables and let JSONDecoder figure it out for you.
class ApiModelImage: Decodable {
let file: String
let thumbnail_file: String
...
}
class ApiModelVideo: Decodable {
let thumbnail: URL
let duration: String?
let youtube_id: String
let youtube_url: URL
...
}
class ApiModelMessage: Decodable {
let title: String
let body: String
let image: ApiModelImage?
let video: ApiModelVideo?
...
}
Then all you have to do is....
if let message = try? JSONDecoder().decode(ApiModelMessage.self, from: data) {
if let image = message.image {
print("yay, my message contains an image!")
}
if let video = message.video {
print("yay, my message contains a video!")
}
}
Alternatively, you could use generics and specify the type when calling your API code:
func get<T: Decodable>(from endpoint: String, onError: #escaping(_: Error?) -> Void, onSuccess: #escaping (_: T) -> Void) {
getData(from: endpoint, onError: onError) { (data) in
do {
let response = try JSONDecoder().decode(T.self, from: data)
onSuccess(response)
} catch {
onError(error)
}
}
}
Used this way, you just have to make sure you define your expected response type:
let successCb = { (_ response: GetUnreadCountsResponse) in
...
}
ApiRequest().get(from: endpoint, onError: { (_) in
...
}, onSuccess: successCb)
Since you define successCb as requiring a GetUnreadCountsResponse model, the API get method generic will be of type GetUnreadCountsResponse at runtime.
Good Luck!

swift - convert json type Int to String

I have json data like this code below:
{
"all": [
{
"ModelId": 1,
"name": "ghe",
"width": 2
},
{
"ModelId": 2,
"name": "ban",
"width": 3
}]
}
I try to get the modelId and convert it to String but it's not working with my code:
let data = NSData(contentsOf: URL(string: url)!)
do {
if let data = data, let json = try JSONSerialization.jsonObject(with: data as Data) as? [String: Any], let models = json["all"] as? [[String:Any]] {
for model in models {
if let name = model["ModelId"] as? String {
_modelList.append(name)
}
}
}
completion(_modelList)
}catch {
print("error")
completion(nil)
}
How to fix this issue? Thanks.
I think ModelId is integer type. So, can you try to cast it to Integer
for model in models {
if let name = model["ModelId"] as? Int{
_modelList.append("\(name)")
}
}
Hope, it will help you.
if let as? is to unwrap, not type casting. So you unwrap first, then you cast it into string.
for model in models {
if let name = model["ModelId"] as? Int {
_modelList.append("\(name)")
}
}
Currently you are looking for a wrong key ,
for model in models {
if let name = model["ModelId"] as? NSNumber {
_modelList.append(name.stringValue)
}
}
As long as you are using JSONSerialization.jsonObject to parse your JSON you have very little control over the type the deserialiser will create, you basically let the parser decide. Sensible as it is it will create "some kind of " NSNumber of an Int type from a number without quotes. This can not be cast to a String, therefore your program will fail.
You can do different things in order to "fix" this problem, I would like to suggest the Codable protocol for JSON-parsing, but this specific problem can probably only be solved using a custom initialiser which looks kind of verbose as can be seen in this question.
If you just want to convert your NSNumber ModelId to a String you will have to create a new object (instead of trying to cast in vain). In your context this might simply be
if let name = String(model["ModelId"]) { ...
This is still not an elegant solution, however it will solve the problem at hand.
another approach is:
import Foundation
struct IntString: Codable
{
var value: String = "0"
init(from decoder: Decoder) throws
{
// get this instance json value
let container = try decoder.singleValueContainer()
do
{
// try to parse the value as int
value = try String(container.decode(Int.self))
}
catch
{
// if we failed parsing the value as int, try to parse it as a string
value = try container.decode(String.self)
}
}
func encode(to encoder: Encoder) throws
{
var container = encoder.singleValueContainer()
try container.encode(value)
}
}
my solution is to create a new struct that will be able to receive either a String or and Int and parse it as a string, this way in my code i can decide how to treat it, and when my server sends me sometimes an Int value and sometimes a json with that same key as a String value - the parser can parse it without failing
of course you can do it with any type (date / double / float / or even a full struct), and even insert it with some logic of your own (say get the string value of an enum based on the received value and use it as index or whatever)
so your code should look like this:
import Foundation
struct Models: Codable {
let all: [All]
}
struct All: Codable {
let modelID: IntString
let name: String
let width: IntString
enum CodingKeys: String, CodingKey {
case modelID = "ModelId"
case name = "name"
case width = "width"
}
}
parse json into the Models struct:
let receivedModel: Decodable = Bundle.main.decode(Models.self, from: jsonData!)
assuming you'r json decoder is:
import Foundation
extension Bundle
{
func decode<T: Decodable>(_ type: T.Type, from jsonData: Data, dateDecodingStrategy: JSONDecoder.DateDecodingStrategy = .deferredToDate, keyDecodingStrategy: JSONDecoder.KeyDecodingStrategy = .useDefaultKeys) -> T
{
let decoder = JSONDecoder()
decoder.dateDecodingStrategy = dateDecodingStrategy
decoder.keyDecodingStrategy = keyDecodingStrategy
do
{
return try decoder.decode(T.self, from: jsonData)
}
catch DecodingError.keyNotFound(let key, let context)
{
fatalError("Failed to decode \(jsonData) from bundle due to missing key '\(key.stringValue)' not found – \(context.debugDescription)")
}
catch DecodingError.typeMismatch(let type, let context)
{
print("Failed to parse type: \(type) due to type mismatch – \(context.debugDescription) the received JSON: \(String(decoding: jsonData, as: UTF8.self))")
fatalError("Failed to decode \(jsonData) from bundle due to type mismatch – \(context.debugDescription)")
}
catch DecodingError.valueNotFound(let type, let context)
{
fatalError("Failed to decode \(jsonData) from bundle due to missing \(type) value – \(context.debugDescription)")
}
catch DecodingError.dataCorrupted(_)
{
fatalError("Failed to decode \(jsonData) from bundle because it appears to be invalid JSON")
}
catch
{
fatalError("Failed to decode \(jsonData) from bundle: \(error.localizedDescription)")
}
}
}