Converting associative array request parameters to JSON with Jackson - json

I am the recipient of a webhook POST that looks like this, which I have decoded for readability.
id=12345&event=this_event&payload[customer][name]=ABC Company&payload[customer][city]=New York&payload[service][name]=New Customer&payload[service][action]=New
Using Spring MVC, I can easily get this into a Map<String, Sting> that looks like this
{id=97659204, event=test, payload[customer][name]=ABC Company, payload[customer][city]=New York, payload[service][name]=New Customer, payload[service][action]=New}
I need to parse every parameter (or Map key) that starts with "payload" into a JSON object.
My desired output from parsing the request parameters that start with "payload" would look something like this
{
customer : {
name : "ABC Company",
city : "New York"
},
service : {
name : "New Customer",
action : "New"
}
}
With the final state being a call to Jackson's ObjectMapper to turn that JSON into a POJO.
Since I have no control over the data format begin sent to me, what is the best/correct option for parsing those request parameters into a JSON object?
Thanks.

I ended up writing a custom parser for the payload[][][] parameters being passed in. It uses regex matcher and then recursion to analyze each parameter, traverse a Map of 1...n Maps and then the resulting Map makes perfect JSON.
#RequestMapping(value = "/receiver", method = RequestMethod.POST)
#ResponseBody
public void receiver(#RequestParam Map<String, Object> requestBody) throws IOException {
Pattern pattern = Pattern.compile("\\[([^\\]]+)]");
Map<String, Object> payloadMap = new HashMap<>();
Matcher matcher = null;
List<String> levels = null;
for (String key : requestBody.keySet()) {
if (key.startsWith("payload")) {
matcher = pattern.matcher(key);
levels = new ArrayList<>();
while (matcher.find()) {
levels.add(matcher.group(1));
}
payloadMap = nestMap(payloadMap, levels.iterator(), (String) requestBody.get(key));
}
}
LOG.debug(mapper.writeValueAsString(payloadMap));
}
#SuppressWarnings("unchecked")
private Map<String, Object> nestMap(Map<String, Object> baseMap, Iterator<String> levels, String value) {
String key = levels.next();
if (!levels.hasNext()) {
baseMap.put(key, value);
} else {
if (!baseMap.containsKey(key)) {
baseMap.put(key, nestMap(new HashMap<String, Object>(), levels, value));
} else {
baseMap.put(key, nestMap((Map<String, Object>) baseMap.get(key), levels, value));
}
}
return baseMap;
}

Related

How to configure kafka deserializer so that it let pass only keys with values?

I have a kafka consumer which does the deserialization of JSON.
I get big messages like :
{
data1: [],
data2: ["content"],
data3: null,
data4: [],
data5: "foo",
data6: "",
data7: [""],
data8: [null],
...
dataX: []
}
with a lot of keys, where the values are null respective empty lists. I want handle only the keys, which does have values or lists with senseful content like data2 or data5 and ignore the others. Even with data6 (empty string) and data7 (list with empty string) I want stay. This shall pass.
I have this kafka consumer configuration.
private void configureAndInitConsumer() {
final Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
props.put(ConsumerConfig.GROUP_ID_CONFIG, consumerGroupId);
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, autoCommit);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, org.apache.kafka.connect.json.JsonDeserializer.class.getName());
props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, maxPollRecords);
consumer = new KafkaConsumer<>(props);
}
I see, that here the org.apache.kafka.connect.json.JsonDeserializer is configured. But the config is for kafka consumer.
I can configure the deserializer by replacing the KafkaConsumer constructor.
JsonDeserializer<JsonNode> valueDeserializer = createCustomizedJsonDeserializer();
consumer = new KafkaConsumer<>(props, new StringDeserializer(), valueDeserializer);
and then I add the method
private JsonDeserializer<JsonNode> createCustomizedJsonDeserializer() {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(MapperFeature.DEFAULT_VIEW_INCLUSION, false);
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
objectMapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, false);
return new JsonDeserializer<>(JsonNode.class, objectMapper);
}
The hint for this I got from https://github.com/spring-projects/spring-kafka/issues/915
Is is possible that the Deserializer ignores the keys without values? (empty lists or null values)
I realized that the Deserializer intern has an object mapper.
So I testest this:
public class ObjectMapperTest {
public static void main(String[] args) throws JsonMappingException, JsonProcessingException {
System.exit(new ObjectMapperTest().start());
}
private int start() throws JsonMappingException, JsonProcessingException {
String testJson = "{\"date1\":[],\"date2\":null,\"date3\":[null],\"date4\":[\"\"],\"date5\":[],\"date6\":\"\",\"date8\":\"foobar\"}";
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.disable(DeserializationFeature.ACCEPT_EMPTY_STRING_AS_NULL_OBJECT);
JsonNode jn = objectMapper.readTree(testJson);
System.out.println(jn);
return 0;
}
}
I get this result:
{"date1":[],"date2":null,"date3":[null],"date4":[""],"date5":[],"date6":"","date8":"foobar"}
I see there, that date2 is still there.

How to ignore duplicate keys while parsing json using gson?

I am getting a a duplicate key exception while parsing JSON response containing timestamps as keys using GSON. It gives the following error:
com.google.gson.JsonSyntaxException: duplicate key: 1463048935
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:186)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:141)
How do I make it ignore the duplicate entries, and just parse it to a map with any one from the duplicate entries?
Hackerman solution, tested and working using GSON v2.8.5, but use at your own risk! Whenever you update GSON to a new version, make sure to check whether this is still working!
Basically, you can use the fact that the generic ObjectTypeAdapter ignores duplicates as seen here:
Looks like MapTypeAdapterFactory checks for duplicate
V replaced = map.put(key, value);
if (replaced != null) {
throw new JsonSyntaxException("duplicate key: " + key);
}
however ObjectTypeAdapter does not
case BEGIN_OBJECT:
Map<String, Object> map = new LinkedTreeMap<String, Object>();
in.beginObject();
while (in.hasNext()) {
map.put(in.nextName(), read(in));
}
in.endObject();
return map;
What you can do now is trying to deserialize using fromJson as usual, but catch the "duplicate key" exception, deserialize as a generic Object, which will ignore duplicates, serialize it again, which will result in a JSON string without duplicate keys, and finally deserialize using the correct type it's actually meant to be.
Here is a Kotlin code example:
fun <T> String.deserialize(gson: Gson, typeToken: TypeToken<T>): T {
val type = typeToken.type
return try {
gson.fromJson<T>(this, type)
} catch (e: JsonSyntaxException) {
if (e.message?.contains("duplicate key") == true) {
gson.toJson(deserialize(gson, object : TypeToken<Any>() {})).deserialize(gson, typeToken)
} else {
throw e
}
}
}
Obviously, this adds (potentially heavy) overhead by requiring 2 deserializations and an additional serialization, but currently I don't see any other way to do this. If Google decides to add an option for a built-in way to ignore duplicates, as suggested here, better switch to that.
I couldn't use Kotlin (as answered before), so I adjusted it to Java
It could be achieved by registering the type adder:
#Test
void testDuplicatesIgnored() {
String json = "{\"key\": \"value\", \"key\": \"value2\"}";
Gson gson = new GsonBuilder()
.registerTypeAdapter(Map.class, new JsonDeserializer<Map<String, Object>>() {
#Override
public Map<String, Object> deserialize(JsonElement json1, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
return new Gson().fromJson(json1, typeOfT);
}
})
.create();
Type mapType = new TypeToken<Map<String, Object>>() {}.getType();
Map<String, Object> map = gson.fromJson(json, mapType);
System.out.println("map = " + map); // map = {key=value2}
assertThat(map).hasSize(1);
assertThat(map.get("key")).isEqualTo("value2");
}
This way all the mappings to Map.class will go through your deserializer code
Yea, looks like a dirty hack, but it works
Another way is to register type adder for your custom type to make the deserializer being called only where you need it:
#Test
void testDuplicatesIgnored() {
String json = "{\"key\": \"value\", \"key\": \"value2\"}";
Gson gson = new GsonBuilder()
.registerTypeAdapter(MapIgnoringDuplicatesContainer.class, new JsonDeserializer<MapIgnoringDuplicatesContainer>() {
#Override
public MapIgnoringDuplicatesContainer deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
Type mapType = new TypeToken<Map<String, Object>>() {}.getType();
return new MapIgnoringDuplicatesContainer(new Gson().fromJson(json, mapType));
}
})
.create();
Map<String, Object> map = gson.fromJson(json, MapIgnoringDuplicatesContainer.class).getMap();
System.out.println("map = " + map); // map = {key=value2}
assertThat(map).hasSize(1);
assertThat(map.get("key")).isEqualTo("value2");
}
private class MapIgnoringDuplicatesContainer {
private Map<String, Object> map;
public MapIgnoringDuplicatesContainer(Map<String, Object> map) {
this.map = map;
}
public Map<String, Object> getMap() {
return map;
}
}
```

Convert Guava HashMultimap to json

I want to print HashMultiMap as json.
HashMultimap<String,Object> multimap = HashMultimap.create();
multimap.put("a",Obj1);
multimap.put("a",Obj3);
multimap.put("b",Obj2);
to
{
"a":[Obj1,Obj3],
"b":[Obj2]
}
Obj1 and other objects should again be in json(to keep it clean, I have shown it as objects)
I can iterate over the individual keys and convert set of Objects to json using libraries such as Gson.
But to get the entire snapshot of the HashMultimap, I want to convert it to json and inspect it.
Gson could not convert the entire map, but could do individual values(list of objects to json)
Call asMap() on the MultiMap first. This converts the MultiMap to a standard Map where each value is a Collection.
In your example, the type of the resulting Map is Map<String, Collection<Object>>. Gson should be able to serialise this correctly.
You need to write a JsonAdapter or both JsonDeserializer and JsonSerializer. It's rather terrible, but I wanted to try.
Basically, you delegate everything to a Map<String, Collection<V>>.
static class MultimapAdapter implements JsonDeserializer<Multimap<String, ?>>, JsonSerializer<Multimap<String, ?>> {
#Override public Multimap<String, ?> deserialize(JsonElement json, Type type,
JsonDeserializationContext context) throws JsonParseException {
final HashMultimap<String, Object> result = HashMultimap.create();
final Map<String, Collection<?>> map = context.deserialize(json, multimapTypeToMapType(type));
for (final Map.Entry<String, ?> e : map.entrySet()) {
final Collection<?> value = (Collection<?>) e.getValue();
result.putAll(e.getKey(), value);
}
return result;
}
#Override public JsonElement serialize(Multimap<String, ?> src, Type type, JsonSerializationContext context) {
final Map<?, ?> map = src.asMap();
return context.serialize(map);
}
private <V> Type multimapTypeToMapType(Type type) {
final Type[] typeArguments = ((ParameterizedType) type).getActualTypeArguments();
assert typeArguments.length == 2;
#SuppressWarnings("unchecked")
final TypeToken<Map<String, Collection<V>>> mapTypeToken = new TypeToken<Map<String, Collection<V>>>() {}
.where(new TypeParameter<V>() {}, (TypeToken<V>) TypeToken.of(typeArguments[1]));
return mapTypeToken.getType();
}
}
The full code including a test can be found here.

GWT HashMap to/from JSON

I might be getting a bit tired tonight but here it goes:
I'd like to have GWT HashMap to/from JSON. How would I achieve this?
In other words, I'd like to take an HashMap, take its JSON representation, store it somewhere and get it back to its native Java representation.
Here is my quick solution:
public static String toJson(Map<String, String> map) {
String json = "";
if (map != null && !map.isEmpty()) {
JSONObject jsonObj = new JSONObject();
for (Map.Entry<String, String> entry: map.entrySet()) {
jsonObj.put(entry.getKey(), new JSONString(entry.getValue()));
}
json = jsonObj.toString();
}
return json;
}
public static Map<String, String> toMap(String jsonStr) {
Map<String, String> map = new HashMap<String, String>();
JSONValue parsed = JSONParser.parseStrict(jsonStr);
JSONObject jsonObj = parsed.isObject();
if (jsonObj != null) {
for (String key : jsonObj.keySet()) {
map.put(key, jsonObj.get(key).isString().stringValue());
}
}
return map;
}
Not the most optimized, but should be easy to code: use JSONObject.
Iterate over your map's entries and put them in a JSONObject (converting each value to a JSONValue of the appropriate type), then call toString to get the JSON representation.
For parsing, get a JSONObject back using a JSONParser, then iterate over the keySet, geting the values and putting them in your map (after unwrapping the JSONValues)
But beware of the keys you use! You cannot use any kind of key as a property name in JS; and JSON processing in the browser always involve going through a JS object (or implementing the JSON parser yourself, which won't perform the same)

How can I deserialize JSON to a simple Dictionary<string,string> in ASP.NET?

I have a simple key/value list in JSON being sent back to ASP.NET via POST. Example:
{ "key1": "value1", "key2": "value2"}
I AM NOT TRYING TO DESERIALIZE INTO STRONGLY-TYPED .NET OBJECTS
I simply need a plain old Dictionary(Of String, String), or some equivalent (hash table, Dictionary(Of String, Object), old-school StringDictionary--hell, a 2-D array of strings would work for me.
I can use anything available in ASP.NET 3.5, as well as the popular Json.NET (which I'm already using for serialization to the client).
Apparently neither of these JSON libraries have this forehead-slapping obvious capability out of the box--they are totally focused on reflection-based deserialization via strong contracts.
Any ideas?
Limitations:
I don't want to implement my own JSON parser
Can't use ASP.NET 4.0 yet
Would prefer to stay away from the older, deprecated ASP.NET class for JSON
Json.NET does this...
string json = #"{""key1"":""value1"",""key2"":""value2""}";
var values = JsonConvert.DeserializeObject<Dictionary<string, string>>(json);
More examples: Serializing Collections with Json.NET
I did discover .NET has a built in way to cast the JSON string into a Dictionary<String, Object> via the System.Web.Script.Serialization.JavaScriptSerializer type in the 3.5 System.Web.Extensions assembly. Use the method DeserializeObject(String).
I stumbled upon this when doing an ajax post (via jquery) of content type 'application/json' to a static .net Page Method and saw that the method (which had a single parameter of type Object) magically received this Dictionary.
System.Text.Json
This can now be done using System.Text.Json which is built-in to .NET Core 3.0. It's now possible to deserialize JSON without using third-party libraries.
var json = #"{""key1"":""value1"",""key2"":""value2""}";
var values = JsonSerializer.Deserialize<Dictionary<string, string>>(json);
Also available in NuGet package System.Text.Json if using .NET Standard or .NET Framework.
Make sure to read and understand:
https://github.com/dotnet/runtime/issues/30452
For those searching the internet and stumbling upon this post, I wrote a blog post on how to use the JavaScriptSerializer class.
Read more...
http://procbits.com/2011/04/21/quick-json-serializationdeserialization-in-c/
Here is an example:
var json = "{\"id\":\"13\", \"value\": true}";
var jss = new JavaScriptSerializer();
var table = jss.Deserialize<dynamic>(json);
Console.WriteLine(table["id"]);
Console.WriteLine(table["value"]);
I had the same problem, so I wrote this my self. This solution is differentiated from other answers because it can deserialize in to multiple levels.
Just send JSON string in to deserializeToDictionary function it will return non strongly-typed Dictionary<string, object> object.
Old code
private Dictionary<string, object> deserializeToDictionary(string jo)
{
var values = JsonConvert.DeserializeObject<Dictionary<string, object>>(jo);
var values2 = new Dictionary<string, object>();
foreach (KeyValuePair<string, object> d in values)
{
// if (d.Value.GetType().FullName.Contains("Newtonsoft.Json.Linq.JObject"))
if (d.Value is JObject)
{
values2.Add(d.Key, deserializeToDictionary(d.Value.ToString()));
}
else
{
values2.Add(d.Key, d.Value);
}
}
return values2;
}
Ex: This will return Dictionary<string, object> object of a Facebook JSON response.
Test
private void button1_Click(object sender, EventArgs e)
{
string responsestring = "{\"id\":\"721055828\",\"name\":\"Dasun Sameera Weerasinghe\",\"first_name\":\"Dasun\",\"middle_name\":\"Sameera\",\"last_name\":\"Weerasinghe\",\"username\":\"dasun\",\"gender\":\"male\",\"locale\":\"en_US\", hometown: {id: \"108388329191258\", name: \"Moratuwa, Sri Lanka\",}}";
Dictionary<string, object> values = deserializeToDictionary(responsestring);
}
Note: hometown further deserilize into a Dictionary<string, object>
object.
Update
My old answer works great if there is no array on JSON string. This one further deserialize in to a List<object> if an element is an array.
Just send a JSON string in to deserializeToDictionaryOrList function it will return non strongly-typed Dictionary<string, object> object or List<object>.
private static object deserializeToDictionaryOrList(string jo,bool isArray=false)
{
if (!isArray)
{
isArray = jo.Substring(0, 1) == "[";
}
if (!isArray)
{
var values = JsonConvert.DeserializeObject<Dictionary<string, object>>(jo);
var values2 = new Dictionary<string, object>();
foreach (KeyValuePair<string, object> d in values)
{
if (d.Value is JObject)
{
values2.Add(d.Key, deserializeToDictionary(d.Value.ToString()));
}
else if (d.Value is JArray)
{
values2.Add(d.Key, deserializeToDictionary(d.Value.ToString(), true));
}
else
{
values2.Add(d.Key, d.Value);
}
}
return values2;
}else
{
var values = JsonConvert.DeserializeObject<List<object>>(jo);
var values2 = new List<object>();
foreach (var d in values)
{
if (d is JObject)
{
values2.Add(deserializeToDictionary(d.ToString()));
}
else if (d is JArray)
{
values2.Add(deserializeToDictionary(d.ToString(), true));
}
else
{
values2.Add(d);
}
}
return values2;
}
}
Tried to not use any external JSON implementation so i deserialised like this:
string json = "{\"id\":\"13\", \"value\": true}";
var serializer = new JavaScriptSerializer(); //using System.Web.Script.Serialization;
Dictionary<string, string> values = serializer.Deserialize<Dictionary<string, string>>(json);
I just needed to parse a nested dictionary, like
{
"x": {
"a": 1,
"b": 2,
"c": 3
}
}
where JsonConvert.DeserializeObject doesn't help. I found the following approach:
var dict = JObject.Parse(json).SelectToken("x").ToObject<Dictionary<string, int>>();
The SelectToken lets you dig down to the desired field. You can even specify a path like "x.y.z" to step further down into the JSON object.
If you're after a lightweight, no-added-references kind of approach, maybe this bit of code I just wrote will work (I can't 100% guarantee robustness though).
using System;
using System.Collections.Generic;
using System.Text;
using System.Text.RegularExpressions;
public Dictionary<string, object> ParseJSON(string json)
{
int end;
return ParseJSON(json, 0, out end);
}
private Dictionary<string, object> ParseJSON(string json, int start, out int end)
{
Dictionary<string, object> dict = new Dictionary<string, object>();
bool escbegin = false;
bool escend = false;
bool inquotes = false;
string key = null;
int cend;
StringBuilder sb = new StringBuilder();
Dictionary<string, object> child = null;
List<object> arraylist = null;
Regex regex = new Regex(#"\\u([0-9a-z]{4})", RegexOptions.IgnoreCase);
int autoKey = 0;
for (int i = start; i < json.Length; i++)
{
char c = json[i];
if (c == '\\') escbegin = !escbegin;
if (!escbegin)
{
if (c == '"')
{
inquotes = !inquotes;
if (!inquotes && arraylist != null)
{
arraylist.Add(DecodeString(regex, sb.ToString()));
sb.Length = 0;
}
continue;
}
if (!inquotes)
{
switch (c)
{
case '{':
if (i != start)
{
child = ParseJSON(json, i, out cend);
if (arraylist != null) arraylist.Add(child);
else
{
dict.Add(key, child);
key = null;
}
i = cend;
}
continue;
case '}':
end = i;
if (key != null)
{
if (arraylist != null) dict.Add(key, arraylist);
else dict.Add(key, DecodeString(regex, sb.ToString()));
}
return dict;
case '[':
arraylist = new List<object>();
continue;
case ']':
if (key == null)
{
key = "array" + autoKey.ToString();
autoKey++;
}
if (arraylist != null && sb.Length > 0)
{
arraylist.Add(sb.ToString());
sb.Length = 0;
}
dict.Add(key, arraylist);
arraylist = null;
key = null;
continue;
case ',':
if (arraylist == null && key != null)
{
dict.Add(key, DecodeString(regex, sb.ToString()));
key = null;
sb.Length = 0;
}
if (arraylist != null && sb.Length > 0)
{
arraylist.Add(sb.ToString());
sb.Length = 0;
}
continue;
case ':':
key = DecodeString(regex, sb.ToString());
sb.Length = 0;
continue;
}
}
}
sb.Append(c);
if (escend) escbegin = false;
if (escbegin) escend = true;
else escend = false;
}
end = json.Length - 1;
return dict; //theoretically shouldn't ever get here
}
private string DecodeString(Regex regex, string str)
{
return Regex.Unescape(regex.Replace(str, match => char.ConvertFromUtf32(Int32.Parse(match.Groups[1].Value, System.Globalization.NumberStyles.HexNumber))));
}
[I realise that this violates the OP Limitation #1, but technically, you didn't write it, I did]
Mark Rendle posted this as a comment, I wanted to post it as an answer since it's the only solution that has worked so far to return the success and the error-codes json results from the Google reCaptcha response.
string jsonReponseString= wClient.DownloadString(requestUrl);
IDictionary<string, object> dict = new JavaScriptSerializer().DeserializeObject(jsonReponseString) as IDictionary<string, object>;
Thanks again, Mark!
For anyone who is trying to convert JSON to dictionary just for retrieving some value out of it. There is a simple way using Newtonsoft.JSON
using Newtonsoft.Json.Linq
...
JObject o = JObject.Parse(#"{
'CPU': 'Intel',
'Drives': [
'DVD read/writer',
'500 gigabyte hard drive'
]
}");
string cpu = (string)o["CPU"];
// Intel
string firstDrive = (string)o["Drives"][0];
// DVD read/writer
IList<string> allDrives = o["Drives"].Select(t => (string)t).ToList();
// DVD read/writer
// 500 gigabyte hard drive
I've added upon the code submitted by jSnake04 and Dasun herein. I've added code to create lists of objects from JArray instances. It has two-way recursion but as it is functioning on a fixed, finite tree model, there is no risk of stack overflow unless the data is massive.
/// <summary>
/// Deserialize the given JSON string data (<paramref name="data"/>) into a
/// dictionary.
/// </summary>
/// <param name="data">JSON string.</param>
/// <returns>Deserialized dictionary.</returns>
private IDictionary<string, object> DeserializeData(string data)
{
var values = JsonConvert.DeserializeObject<Dictionary<string, object>>(data);
return DeserializeData(values);
}
/// <summary>
/// Deserialize the given JSON object (<paramref name="data"/>) into a dictionary.
/// </summary>
/// <param name="data">JSON object.</param>
/// <returns>Deserialized dictionary.</returns>
private IDictionary<string, object> DeserializeData(JObject data)
{
var dict = data.ToObject<Dictionary<String, Object>>();
return DeserializeData(dict);
}
/// <summary>
/// Deserialize any elements of the given data dictionary (<paramref name="data"/>)
/// that are JSON object or JSON arrays into dictionaries or lists respectively.
/// </summary>
/// <param name="data">Data dictionary.</param>
/// <returns>Deserialized dictionary.</returns>
private IDictionary<string, object> DeserializeData(IDictionary<string, object> data)
{
foreach (var key in data.Keys.ToArray())
{
var value = data[key];
if (value is JObject)
data[key] = DeserializeData(value as JObject);
if (value is JArray)
data[key] = DeserializeData(value as JArray);
}
return data;
}
/// <summary>
/// Deserialize the given JSON array (<paramref name="data"/>) into a list.
/// </summary>
/// <param name="data">Data dictionary.</param>
/// <returns>Deserialized list.</returns>
private IList<Object> DeserializeData(JArray data)
{
var list = data.ToObject<List<Object>>();
for (int i = 0; i < list.Count; i++)
{
var value = list[i];
if (value is JObject)
list[i] = DeserializeData(value as JObject);
if (value is JArray)
list[i] = DeserializeData(value as JArray);
}
return list;
}
Edit: This works, but the accepted answer using Json.NET is much more straightforward. Leaving this one in case someone needs BCL-only code.
It’s not supported by the .NET framework out of the box. A glaring oversight – not everyone needs to deserialize into objects with named properties. So I ended up rolling my own:
VB.NET:
<Serializable()> Public Class StringStringDictionary
Implements ISerializable
Public dict As System.Collections.Generic.Dictionary(Of String, String)
Public Sub New()
dict = New System.Collections.Generic.Dictionary(Of String, String)
End Sub
Protected Sub New(info As SerializationInfo, _
context As StreamingContext)
dict = New System.Collections.Generic.Dictionary(Of String, String)
For Each entry As SerializationEntry In info
dict.Add(entry.Name, DirectCast(entry.Value, String))
Next
End Sub
Public Sub GetObjectData(info As SerializationInfo, context As StreamingContext) Implements ISerializable.GetObjectData
For Each key As String in dict.Keys
info.AddValue(key, dict.Item(key))
Next
End Sub
End Class
same on C#:
public class StringStringDictionary : ISerializable
{
public System.Collections.Generic.Dictionary<string, string> dict;
public StringStringDictionary()
{
dict = new System.Collections.Generic.Dictionary<string, string>();
}
protected StringStringDictionary(SerializationInfo info, StreamingContext context)
{
dict = new System.Collections.Generic.Dictionary<string, string>();
foreach (SerializationEntry entry in info)
dict.Add(entry.Name, (string)entry.Value);
}
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
foreach (string key in dict.Keys)
info.AddValue(key, dict[key]);
}
}
Called with:
string MyJsonString = "{ \"key1\": \"value1\", \"key2\": \"value2\"}";
System.Runtime.Serialization.Json.DataContractJsonSerializer dcjs = new
System.Runtime.Serialization.Json.DataContractJsonSerializer(
typeof(StringStringDictionary));
System.IO.MemoryStream ms = new
System.IO.MemoryStream(Encoding.UTF8.GetBytes(MyJsonString));
StringStringDictionary myfields = (StringStringDictionary)dcjs.ReadObject(ms);
Response.Write("Value of key2: " + myfields.dict["key2"]);
Sorry for the mix of C# and VB.NET…
I added a check for null values in the JSON to the other answer
I had same problem so I wrote this my self. This solution is
differentiated from other answers because it can deserialize in to
multiple levels.
Just send json string in to deserializeToDictionary function it
will return non strongly-typed Dictionary<string, object> object.
private Dictionary<string, object> deserializeToDictionary(string jo)
{
var values = JsonConvert.DeserializeObject<Dictionary<string, object>>(jo);
var values2 = new Dictionary<string, object>();
foreach (KeyValuePair<string, object> d in values)
{
if (d.Value != null && d.Value.GetType().FullName.Contains("Newtonsoft.Json.Linq.JObject"))
{
values2.Add(d.Key, deserializeToDictionary(d.Value.ToString()));
}
else
{
values2.Add(d.Key, d.Value);
}
}
return values2;
}
Ex: This will return Dictionary<string, object> object of a Facebook
JSON response.
private void button1_Click(object sender, EventArgs e)
{
string responsestring = "{\"id\":\"721055828\",\"name\":\"Dasun Sameera
Weerasinghe\",\"first_name\":\"Dasun\",\"middle_name\":\"Sameera\",\"last_name\":\"Weerasinghe\",\"username\":\"dasun\",\"gender\":\"male\",\"locale\":\"en_US\",
hometown: {id: \"108388329191258\", name: \"Moratuwa, Sri Lanka\",}}";
Dictionary<string, object> values = deserializeToDictionary(responsestring);
}
Note: hometown further deserialize into a Dictionary<string, object> object.
It seems all of these answers here just assume you can get that little string out of a bigger object... for people looking to simply deserealize a large object with such a dictionary somewhere inside the mapping, and who are using the System.Runtime.Serialization.Json DataContract system, here's a solution:
An answer on gis.stackexchange.com had this interesting link. I had to recover it with archive.org, but it offers a pretty much perfect solution: a custom IDataContractSurrogate class in which you implement exactly your own types. I was able to expand it easily.
I made a bunch of changes in it, though. Since the original source is no longer available, I'll post the entire class here:
using System;
using System.CodeDom;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;
namespace JsonTools
{
/// <summary>
/// Allows using Dictionary<String,String> and Dictionary<String,Boolean> types, and any others you'd like to add.
/// Source: https://web.archive.org/web/20100317222656/my6solutions.com/post/2009/06/30/DataContractSerializer-DataContractJsonSerializer-JavaScriptSerializer-XmlSerializer-for-serialization.aspx
/// </summary>
public class JsonSurrogate : IDataContractSurrogate
{
/// <summary>
/// Deserialize an object with added support for the types defined in this class.
/// </summary>
/// <typeparam name="T">Contract class</typeparam>
/// <param name="json">JSON String</param>
/// <param name="encoding">Text encoding</param>
/// <returns>The deserialized object of type T</returns>
public static T Deserialize<T>(String json, Encoding encoding)
{
if (encoding == null)
encoding = new UTF8Encoding(false);
DataContractJsonSerializer deserializer = new DataContractJsonSerializer(
typeof(T), new Type[0], int.MaxValue, true, new JsonSurrogate(), false);
using (MemoryStream stream = new MemoryStream(encoding.GetBytes(json)))
{
T result = (T)deserializer.ReadObject(stream);
return result;
}
}
// make sure all values in this are classes implementing JsonSurrogateObject.
private static Dictionary<Type, Type> KnownTypes =
new Dictionary<Type, Type>()
{
{typeof(Dictionary<String, String>), typeof(SSDictionary)},
{typeof(Dictionary<String, Boolean>), typeof(SBDictionary)}
};
#region Implemented surrogate dictionary classes
[Serializable]
public class SSDictionary : SurrogateDictionary<String>
{
public SSDictionary() : base() {}
protected SSDictionary (SerializationInfo info, StreamingContext context) : base(info, context) {}
}
[Serializable]
public class SBDictionary : SurrogateDictionary<Boolean>
{
public SBDictionary() : base() {}
protected SBDictionary (SerializationInfo info, StreamingContext context) : base(info, context) {}
}
#endregion
/// <summary>Small interface to easily extract the final value from the object.</summary>
public interface JsonSurrogateObject
{
Object DeserializedObject { get; }
}
/// <summary>
/// Class for deserializing any simple dictionary types with a string as key.
/// </summary>
/// <typeparam name="T">Any simple type that will be deserialized correctly.</typeparam>
[Serializable]
public abstract class SurrogateDictionary<T> : ISerializable, JsonSurrogateObject
{
public Object DeserializedObject { get { return dict; } }
private Dictionary<String, T> dict;
public SurrogateDictionary()
{
dict = new Dictionary<String, T>();
}
// deserialize
protected SurrogateDictionary(SerializationInfo info, StreamingContext context)
{
dict = new Dictionary<String, T>();
foreach (SerializationEntry entry in info)
{
// This cast will only work for base types, of course.
dict.Add(entry.Name, (T)entry.Value);
}
}
// serialize
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
foreach (String key in dict.Keys)
{
info.AddValue(key, dict[key]);
}
}
}
/// <summary>
/// Uses the KnownTypes dictionary to get the surrogate classes.
/// </summary>
/// <param name="type"></param>
/// <returns></returns>
public Type GetDataContractType(Type type)
{
Type returnType;
if (KnownTypes.TryGetValue(type, out returnType))
{
return returnType;
}
return type;
}
public object GetObjectToSerialize(object obj, Type targetType)
{
throw new NotImplementedException();
}
/// <summary>
/// Gets the object out of the surrogate datacontract object. This function is the reason all surrogate objects need to implement the JsonSurrogateObject class.
/// </summary>
/// <param name="obj">Result of the deserialization</param>
/// <param name="targetType">Expected target type of the deserialization</param>
/// <returns></returns>
public object GetDeserializedObject(object obj, Type targetType)
{
if (obj is JsonSurrogateObject)
{
return ((JsonSurrogateObject)obj).DeserializedObject;
}
return obj;
}
public Type GetReferencedTypeOnImport(string typeName, string typeNamespace, object customData)
{
return null;
}
#region not implemented
public object GetCustomDataToExport(MemberInfo memberInfo, Type dataContractType)
{
throw new NotImplementedException();
}
public object GetCustomDataToExport(Type clrType, Type dataContractType)
{
throw new NotImplementedException();
}
public void GetKnownCustomDataTypes(Collection<Type> customDataTypes)
{
throw new NotImplementedException();
}
public CodeTypeDeclaration ProcessImportedType(CodeTypeDeclaration typeDeclaration, CodeCompileUnit compileUnit)
{
throw new NotImplementedException();
}
#endregion
}
}
To add new supported types to the class, you just need to add your class, give it the right constructors and functions (look at SurrogateDictionary for an example), make sure it inherits JsonSurrogateObject, and add its type mapping to the KnownTypes dictionary. The included SurrogateDictionary can serve as basis for any Dictionary<String,T> types where T is any type that does deserialize correctly.
Calling it is really simple:
MyObjtype newObj = JsonSurrogate.Deserialize<MyObjtype>(jsonStr, encoding);
Note that for some reason this thing has trouble using key strings which contain spaces; they were simply not present in the final list. Might just be it's simply against json specs and the api I was calling was poorly implemented, mind you; I dunno. Anyway, I solved this by regex-replacing them with underscores in the raw json data and fixing the dictionary after the deserialization.
Based on comments above try JsonConvert.DeserializeObject<Dictionary<string,dynamic>>(json)
var json = #"{""key1"":1,""key2"":""value2"", ""object1"":{""property1"":""value1"",""property2"":[2,3,4,5,6,7]}}";
var parsedObject = JsonConvert.DeserializeObject<Dictionary<string,dynamic>>(json);
seems to work even for complex objects and lists.
My approach directly deserializes to IDictionary, without JObject or ExpandObject in between. The code uses converter, which is basically copied from ExpandoObjectConverter class found in JSON.NET sourcecode, but using IDictionary instead of ExpandoObject.
Usage:
var settings = new JsonSerializerSettings()
{
Converters = { new DictionaryConverter() },
};
var result = JsonConvert.DeserializeObject<IDictionary<string, object>>(json, settings);
Code:
// based on ExpandoObjectConverter, but using arrays instead of IList, to behave similar to System.Web.Script.Serialization.JavaScriptSerializer
public class DictionaryConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
return ReadValue(reader);
}
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(IDictionary<string, object>));
}
public override bool CanWrite
{
get { return false; }
}
private object ReadValue(JsonReader reader)
{
while (reader.TokenType == JsonToken.Comment)
{
if (!reader.Read())
throw JsonSerializationExceptionCreate(reader, "Unexpected end when reading IDictionary<string, object>.");
}
switch (reader.TokenType)
{
case JsonToken.StartObject:
return ReadObject(reader);
case JsonToken.StartArray:
return ReadList(reader);
default:
if (IsPrimitiveToken(reader.TokenType))
return reader.Value;
throw JsonSerializationExceptionCreate(reader, string.Format(CultureInfo.InvariantCulture, "Unexpected token when converting IDictionary<string, object>: {0}", reader.TokenType));
}
}
private object ReadList(JsonReader reader)
{
List<object> list = new List<object>();
while (reader.Read())
{
switch (reader.TokenType)
{
case JsonToken.Comment:
break;
default:
object v = ReadValue(reader);
list.Add(v);
break;
case JsonToken.EndArray:
return list;
}
}
throw JsonSerializationExceptionCreate(reader, "Unexpected end when reading IDictionary<string, object>.");
}
private object ReadObject(JsonReader reader)
{
IDictionary<string, object> dictionary = new Dictionary<string, object>();
while (reader.Read())
{
switch (reader.TokenType)
{
case JsonToken.PropertyName:
string propertyName = reader.Value.ToString();
if (!reader.Read())
throw JsonSerializationExceptionCreate(reader, "Unexpected end when reading IDictionary<string, object>.");
object v = ReadValue(reader);
dictionary[propertyName] = v;
break;
case JsonToken.Comment:
break;
case JsonToken.EndObject:
return dictionary;
}
}
throw JsonSerializationExceptionCreate(reader, "Unexpected end when reading IDictionary<string, object>.");
}
//based on internal Newtonsoft.Json.JsonReader.IsPrimitiveToken
internal static bool IsPrimitiveToken(JsonToken token)
{
switch (token)
{
case JsonToken.Integer:
case JsonToken.Float:
case JsonToken.String:
case JsonToken.Boolean:
case JsonToken.Undefined:
case JsonToken.Null:
case JsonToken.Date:
case JsonToken.Bytes:
return true;
default:
return false;
}
}
// based on internal Newtonsoft.Json.JsonSerializationException.Create
private static JsonSerializationException JsonSerializationExceptionCreate(JsonReader reader, string message, Exception ex = null)
{
return JsonSerializationExceptionCreate(reader as IJsonLineInfo, reader.Path, message, ex);
}
// based on internal Newtonsoft.Json.JsonSerializationException.Create
private static JsonSerializationException JsonSerializationExceptionCreate(IJsonLineInfo lineInfo, string path, string message, Exception ex)
{
message = JsonPositionFormatMessage(lineInfo, path, message);
return new JsonSerializationException(message, ex);
}
// based on internal Newtonsoft.Json.JsonPosition.FormatMessage
internal static string JsonPositionFormatMessage(IJsonLineInfo lineInfo, string path, string message)
{
if (!message.EndsWith(Environment.NewLine))
{
message = message.Trim();
if (!message.EndsWith(".", StringComparison.Ordinal))
message += ".";
message += " ";
}
message += string.Format(CultureInfo.InvariantCulture, "Path '{0}'", path);
if (lineInfo != null && lineInfo.HasLineInfo())
message += string.Format(CultureInfo.InvariantCulture, ", line {0}, position {1}", lineInfo.LineNumber, lineInfo.LinePosition);
message += ".";
return message;
}
}
Annoyingly enough, if you want to use the default model binders, it looks like you will have to use numerical index values like a form POST.
See the following excerpt from this article http://msdn.microsoft.com/en-us/magazine/hh781022.aspx:
Though it’s somewhat counterintuitive, JSON requests have the same
requirements—they, too, must adhere to the form post naming syntax.
Take, for example, the JSON payload for the previous UnitPrice
collection. The pure JSON array syntax for this data would be
represented as:
[
{ "Code": "USD", "Amount": 100.00 },
{ "Code": "EUR", "Amount": 73.64 }
]
However, the default value providers and model binders require the
data to be represented as a JSON form post:
{
"UnitPrice[0].Code": "USD",
"UnitPrice[0].Amount": 100.00,
"UnitPrice[1].Code": "EUR",
"UnitPrice[1].Amount": 73.64
}
The complex object collection scenario is perhaps one of the most
widely problematic scenarios that developers run into because the
syntax isn’t necessarily evident to all developers. However, once you
learn the relatively simple syntax for posting complex collections,
these scenarios become much easier to deal with.
I just implemented this in RestSharp. This post was helpful to me.
Besides the code in the link, here is my code. I now get a Dictionary of results when I do something like this:
var jsonClient = new RestClient(url.Host);
jsonClient.AddHandler("application/json", new DynamicJsonDeserializer());
var jsonRequest = new RestRequest(url.Query, Method.GET);
Dictionary<string, dynamic> response = jsonClient.Execute<JObject>(jsonRequest).Data.ToObject<Dictionary<string, dynamic>>();
Be mindful of the sort of JSON you're expecting - in my case, I was retrieving a single object with several properties. In the attached link, the author was retrieving a list.
A bit late to the game, but non of the above solutions pointed me in the direction of a pure and simple .NET, no json.net solution. So here it is, ended up being very simple. Below a full running example of how it is done with standard .NET Json serialization, the example has dictionary both in the root object and in the child objects.
The golden bullet is this cat, parse the settings as second parameter to the serializer:
DataContractJsonSerializerSettings settings =
new DataContractJsonSerializerSettings();
settings.UseSimpleDictionaryFormat = true;
Full code below:
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
namespace Kipon.dk
{
public class JsonTest
{
public const string EXAMPLE = #"{
""id"": ""some id"",
""children"": {
""f1"": {
""name"": ""name 1"",
""subs"": {
""1"": { ""name"": ""first sub"" },
""2"": { ""name"": ""second sub"" }
}
},
""f2"": {
""name"": ""name 2"",
""subs"": {
""37"": { ""name"": ""is 37 in key""}
}
}
}
}
";
[DataContract]
public class Root
{
[DataMember(Name ="id")]
public string Id { get; set; }
[DataMember(Name = "children")]
public Dictionary<string,Child> Children { get; set; }
}
[DataContract]
public class Child
{
[DataMember(Name = "name")]
public string Name { get; set; }
[DataMember(Name = "subs")]
public Dictionary<int, Sub> Subs { get; set; }
}
[DataContract]
public class Sub
{
[DataMember(Name = "name")]
public string Name { get; set; }
}
public static void Test()
{
var array = System.Text.Encoding.UTF8.GetBytes(EXAMPLE);
using (var mem = new System.IO.MemoryStream(array))
{
mem.Seek(0, System.IO.SeekOrigin.Begin);
DataContractJsonSerializerSettings settings =
new DataContractJsonSerializerSettings();
settings.UseSimpleDictionaryFormat = true;
var ser = new DataContractJsonSerializer(typeof(Root), settings);
var data = (Root)ser.ReadObject(mem);
Console.WriteLine(data.Id);
foreach (var childKey in data.Children.Keys)
{
var child = data.Children[childKey];
Console.WriteLine(" Child: " + childKey + " " + child.Name);
foreach (var subKey in child.Subs.Keys)
{
var sub = child.Subs[subKey];
Console.WriteLine(" Sub: " + subKey + " " + sub.Name);
}
}
}
}
}
}
You could use Tiny-JSON
string json = "{\"key1\":\"value1\", \"key2\":\"value2\"}";
IDictionary<string, string> dict = Tiny.Json.Decode<Dictionary<string, string>>(json);
I would suggest using System.Runtime.Serialization.Json that is part of .NET 4.5.
[DataContract]
public class Foo
{
[DataMember(Name = "data")]
public Dictionary<string,string> Data { get; set; }
}
Then use it like this:
var serializer = new DataContractJsonSerializer(typeof(List<Foo>));
var jsonParams = #"{""data"": [{""Key"":""foo"",""Value"":""bar""}] }";
var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonParams));
var obj = serializer.ReadObject(stream);
Console.WriteLine(obj);
Here is my solution with System.Text.Json. You get a json string for the nested objects which in own turn can be converted to needed type later on.
public static Dictionary<string,string> JsonToDictionary(this string json)
{
var objectValues = JsonSerializer.Deserialize<Dictionary<string, object>>(json);
var stringValues = objectValues.Select(o => new KeyValuePair<string, string>(o.Key, o.Value?.ToString()));
return stringValues.ToDictionary(pair => pair.Key, pair => pair.Value);
}
Here is the usage example to fetch values from a nested object:
var result= json.JsonToDictionary()["outerField"]
.JsonToDictionary()["innerField"];
Note that this solution does not cover the json objects starting as an array like [12, 13]. These objects can be read as an array in the begining and then the extension method can be applied on each item, in case the items are complex objects with their own properties.