ServiceStack ORMLite not deserializing JSON stored in DB text field - json

I have some telemetry data that was stored in a text field as JSON. I'm attempting to reverse-engineer POCOs to seamlessly extract and present that data without having to do any post-processing ForEach loops.
When I attempt to manually parse the JSON string in the data field, it works. When I am selecting via ORMLite, it comes back with the default object. What am I missing?
Works
string x = "{\"alive\":true,\"paperStatus\":\"0:NoIssue\"}";
var telemetry = JsonSerializer.DeserializeFromString<KioskTelemetryData>(x);
Doesn't populate Data field
var exp = Db.From<KioskTelemetryLog>()
.Where(q => q.Identifier == request.Identifier)
.OrderByDescending(q => q.Timestamp);
var data = Db.Select(exp);
Here is what the data records look like:
Id Identifier IsCurrent RedemptionCenterId Timestamp Status Data
1 XXX 0 NULL 2015-11-24 11:10:53.527 1 {"alive":true,"paperStatus":"1:LowPaper"}
2 XXX 0 NULL 2015-12-01 12:16:56.653 0 {"alive":true,"paperStatus":"0:NoIssue"}
1 XXX 1 NULL 2015-12-01 18:32:11.337 2 {"alive":false}
And here are the POCOs:
[Alias("TelemetryLog")]
public class KioskTelemetryLog
{
public long Id { get; set; }
public string Identifier { get; set; }
public bool IsCurrent { get; set; }
public int? RedemptionCenterId { get; set; }
public DateTime Timestamp { get; set; }
// 0 = okay, 1 = warning, 2 = error
public sbyte Status { get; set; }
public KioskTelemetryData Data { get; set; }
}
public class KioskTelemetryData
{
public bool Alive { get; set; }
public string PaperStatus { get; set; }
}

The default complex type serializer for OrmLite is the JSV Format (except for PostgreSQL which uses JSON). OrmLite does support pluggable text serializers, e.g. you can change SQL Server to use JSON for serializing complex types with:
SqlServerDialect.Provider.StringSerializer = new JsonStringSerializer();
If it's not serializing correctly it's like a serialization error, you can try enable logging for more info about the error, e.g:
LogManager.LogFactory = new ConsoleLogFactory(debugEnabled:true);

Related

How to include the bit in json while converting json to dataset

In my Dot Net Core Web project I have to dump the Json data to the the Database.
My json data is as follows
{
'FactTable': 'TranMast',
'LastCdt':'2018-03-01 01:19:17 pm',
'DigiAgentDetId':'12',
'DigiAgentId':'34',
'LastSeqNo':'23',
'Deleted':'1',
'DataSet':{
'TranDet':[
{'Id':'1','TranMastId':'53_25','AcMastId':'sdsd','Account':'sdsd','Amount':'23000','MemName':'dsds','LocId':'ewsds','PartyMastId':'dsdsd','PartyMemId':'23232','AcParentId':'dsdsd','Deleted':'1'},
{'Id':'2','TranMastId':'53_25','AcMastId':'sdsd','Account':'sdsd','Amount':'24000','MemName':'dsds','LocId':'ewsds','PartyMastId':'dsdsd','PartyMemId':'23232','AcParentId':'dsdsd','Deleted':'1'},
{'Id':'3','TranMastId':'53_25','AcMastId':'sdsd','Account':'sdsd','Amount':'25000','MemName':'dsds','LocId':'ewsds','PartyMastId':'dsdsd','PartyMemId':'23232','AcParentId':'dsdsd','Deleted':'1'}
]
}
}
I am construting above json to dataset as following
var myUtil = JsonConvert.DeserializeObject<MyTableUtilClass>(json);
Where MyTableUtilClass have the follwing structure
public class MyTableUtilClass
{
public string LastCdt { get; set; }
public string DigiAgentId { get; set; }
public string DigiAgentDetId { get; set; }
public string LastSeqNo { get; set; }
public string FactTable { get; set; }
public DataSet DataSet { get; set; }
}
While trying to dump the table in the dataset to the database. in follwing code i am getting the error
int result = adapter.Update(table);
The Error I Am getting is
Failed to convert parameter value from a String to a Boolean
I know the couse of the problem is the field name Deleted is of type bit.
Then in which format i have to include the bit data in the json string.

ASP.NET MVC controller losing a day on conversion

I'm sending some data from a local application to another one (ASP.NET MVC). The data is an array of DTOs. I am testing this on my local machine.
The local application is a Web Forms and is sending the data using a service. I do not use JsonResult.
The serialzed DTO is sent correctly, but the dates are deserialized incorrectly on the MVC side, being a day behind then the ones sent.
The DTO class looks like this:
[DataContract]
public class ProjectInfoDto
{
[DataMember]
public long ProjectId { get; set; }
[DataMember]
public string ProjcetCode { get; set; }
[DataMember]
public DateTime BeginDate { get; set; }
[DataMember]
public DateTime FinalDate { get; set; }
}
I am serializing the data using the below method:
public static string JsonSerialize<T>(T obj)
{
var serializer = new DataContractJsonSerializer(obj.GetType());
var ms = new MemoryStream();
serializer.WriteObject(ms, obj);
string retVal = Encoding.Default.GetString(ms.ToArray());
ms.Dispose();
return retVal;
}
The JSON sent to the MVC application looks like this (with correct dates):
{"ProjectId":"222","ProjcetCode":"OP9089","BeginDate":"/Date(1461790800000+0300)/","FinalDate":"/Date(1557435600000+0300)/" }
The method receiving the data looks like this:
public ActionResult GetProjectData(ProjectInfoDto[] info) {...}
The original dates are (from the database):
BeginDate -> 2016-04-28 00:00:00.000
FinalDate -> 2019-05-10 00:00:00.000
The dates on the MVC side when looking at the received array:
BeginDate -> 2016-04-27 00:00:00.000
FinalDate -> 2019-05-09 00:00:00.000
I don't know why the dates are a day behind when received.
I found a suggestion in this question Is there a way to override how DataContractJsonSerializer serializes Dates?.
The answer in the question suggests creating an instance method and decorate it using the [OnSerializing] attribute. This will allow custom serialization for certain attributes.
Full code below:
[DataContract]
public class ProjectInfoDto
{
[DataMember(Name = "BeginDate")]
private string beginDate;
[DataMember(Name = "ExecutieDataFinal")]
private string finalDate;
[DataMember]
public long ProjectId { get; set; }
[DataMember]
public string ProjcetCode { get; set; }
public DateTime BeginDate { get; set; }
public DateTime FinalDate { get; set; }
[OnSerializing]
void OnSerializing(StreamingContext context)
{
beginDate = BeginDate.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture);
finalDate = FinalDate.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture);
}
}
The idea is to leave the DateTime properties intact, and use the private string fields for the serialzation. In the OnSerializing method, you convert the DateTime properties to the required format. As per the suggestions in the comments to my question, I used the ISO-8601 format ("yyyy-MM-dd").

Dream Factory .NET SDK Post records Unique Integrity Error

In teh .NET SDK your create record method passes a 0 for id on all records as this is unknown until teh response comes back with the id's populated etc
It Seems that the JSON DefaultValueHandling = DefaultValueHandling.Ignore is not working on the freshly minted int '0' id's
An therefore the body has the id:0 and trys inserts the records with id:0 on all and trips a Unique constraint on the inner exception in fiddler
I had a similar problem with DreamFactory
I added a conditional Property Serialization attribute to the Poco/DTO for the staff record as an example
internal class StaffRecord
{
public bool ShouldSerializeUid()
{
return Uid != 0;
}
public int Uid { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public int Age { get; set; }
public bool Active { get; set; }
public override string ToString()
{
return string.Format("{0}: name = {1} {2}, age = {3}, active = {4}", Uid, FirstName, LastName, Age, Active);
}
}
This now works as expected on both serialization/deserialization
Here is the docs in the JSON.NET docs
Conditional Property Serialization
Cheers :D

C# JSON data serialized and binded to DataGridView

I have this data class for storing data parsed from JSON formatted web data (using Json.NET library):
[Serializable()]
public class MovieData
{
public string FilePath { get; set; }
public string OrigName { get; set; }
[JsonProperty(PropertyName = "id")]
public int Id { get; set; }
[JsonProperty(PropertyName = "year")]
public int Year { get; set; }
[JsonProperty(PropertyName = "genres")]
public string[] Genres { get; set; }
}
The next class is for to be able serialize collection of MovieData objects:
[Serializable()]
[XmlRoot("MovieCollection")]
public class MovieCollection
{
[XmlArray("Movies")]
[XmlArrayItem("Movie", typeof(Movie))]
public List<Movie> movies = new List<MovieData>();
}
Finally, I need to bind such a collection of MovieData to DataGridView (or single MovieData object to DataGridViewRow), like:
dgvMovies.DataSource = movieCollection.movies;
Is it possible to bind it without hard-setting of DataGridViewColumn collection before? Native data types are not problem, problem is string[] Genres array, which I need to format it in DataGridView in some way, like:
"genres[0] / genres[0] / ... genres[n]"
At this moment, while simply setting DataSource to collectin, this array is ignored (is not displayed anyway).
In MovieData class, you can add the following property :
public string GenresAsString
{
get { return String.Join("/", Genres); }
set { Genres = value.Split('/'); }
}
You will surely have to improve the setter to make it more resilient (triming, removing empty genres) if you plan to let the user modify this value.
Else you can remove the setter.

DateTimes deserializing wrong: JsonConvert is returning wrong dates

I'm retrieving data from Solr in my code to get a list of Events. The results I get is formatted like so:
public class SearchResults<T> where T : Result
{
public SearchResults()
{
Results = new List<T>();
}
public IEnumerable<T> Results { get; set; }
public int Total { get; set; }
public IEnumerable<FacetField> FacetFields { get; set; }
}
so I get a list of results, a total count, and a list of facetfields. The list of results in this case is a list of EventResults:
public class EventResult : Result
{
public string Location { get; set; }
public string DisplayDate { get; set; }
public DateTime StartDate { get; set; }
public DateTime EndDate { get; set; }
public string Time { get; set; }
public string ImageUrl { get; set; }
public string WebsiteUrl { get; set; }
public string WebsiteText { get; set; }
public string CustomUrl { get; set; }
public string CustomUrlText { get; set; }
public string Description { get; set; }
public string Latitude { get; set; }
public string Longitude { get; set; }
public IEnumerable<string> TaxonomyTypes { get; set; }
public IEnumerable<string> TaxonomyTypesId { get; set; }
public IEnumerable<string> TaxonomyTopics { get; set; }
public IEnumerable<string> TaxonomyTopicsId { get; set; }
public IEnumerable<string> TaxonomyLocations { get; set; }
public IEnumerable<string> TaxonomyLocationsId { get; set; }
public override void Load(XElement data)
{
}
}
and this is my code...
using (StreamReader sr = new StreamReader(stream))
{
var responseData = sr.ReadToEnd();
// added for testing
var startdate = responseData.Substring((responseData.IndexOf("StartDate") + 12), ((responseData.IndexOf("EndDate")-3) - (responseData.IndexOf("StartDate") + 12)));
var Results = JsonConvert.DeserializeObject<SearchResults<EventResult>>(responseData);
.....
}
I put in a search that will return exactly one event for simplicity, so responseData contains only one event in Results. When I look at the raw responseData while debugging, it looks like this:
{"Results":[{"Location":"Online","DisplayDate":"Jul 23, 2014","StartDate":"2014-07-23T00:00:00Z","EndDate":"2014-07-25T00:00:00Z","Time":"","Speakers":"","ImageUrl":"","WebsiteUrl":"","WebsiteText":"","CustomUrl":"","CustomUrlText":"","Description":"","Latitude":"","Longitude":"","TaxonomyTypes":[],"TaxonomyTypesId":[],"TaxonomyTopics":[],"TaxonomyTopicsId":[],"TaxonomyLocations":[],"TaxonomyLocationsId":[],"Id":"768","Title":"MFin Online Chat - 12:00 p.m.","Source":{"doc":{"str":[{"#name":"id","#text":"event_768"},{"#name":"s_eventId","#text":"768"},{"#name":"s_contact"},{"#name":"t_contact"},{"#name":"s_description"},{"#name":"t_description"},{"#name":"s_dateDisplay","#text":"Jul 23, 2014"},{"#name":"s_location","#text":"Online"},{"#name":"t_location","#text":"Online"},{"#name":"s_name","#text":"MFin Online Chat - 12:00 p.m."},{"#name":"t_name","#text":"MFin Online Chat - 12:00 p.m."},{"#name":"s_openTo"},{"#name":"t_openTo"},{"#name":"s_sponsors"},{"#name":"t_sponsors"},{"#name":"s_time"},{"#name":"s_latitude"},{"#name":"s_longitude"},{"#name":"s_speakers"},{"#name":"t_speakers"},{"#name":"s_customUrlText"},{"#name":"s_customUrl"},{"#name":"s_imageUrl"},{"#name":"s_websiteText"},{"#name":"s_websiteUrl"},{"#name":"t_taxonomy_topics"},{"#name":"t_taxonomy_types"},{"#name":"t_taxonomy_locations"},{"#name":"s_type","#text":"Event"},{"#name":"s_folderId","#text":"101"}],"arr":{"#name":"text","str":[null,null,"Online","MFin Online Chat - 12:00 p.m.",null,null,null,null,null,null]},"date":[{"#name":"dt_startDate","#text":"2014-07-23T00:00:00Z"},{"#name":"dt_endDate","#text":"2014-07-25T00:00:00Z"}],"long":{"#name":"_version_","#text":"1482239673606602769"}}}}],"Total":1,"FacetFields":[{"Name":"mv_taxonomy_topics","Values":[]},{"Name":"mv_taxonomy_topicsId","Values":[]},{"Name":"mv_taxonomy_types","Values":[]},{"Name":"mv_taxonomy_typesId","Values":[]},{"Name":"mv_taxonomy_locations","Values":[]},{"Name":"mv_taxonomy_locationsId","Values":[]}]}
There's a lot there, but the important thing is that StartDate is set to 2014-07-23T00:00:00Z, or July 23 2014. There are two occurrences of StartDate in the data, but they are both the same. To double check, I added the line to pull out the StartDate using substring, and sure enough, var startdate = 2014-07-23T00:00:00Z
However, at the next line, when I check Results (the deserialized object returned by JsonConvert), the one event in Results.Results has the wrong start date:
Results.Results[0].StartDate = 7/22/2014 8:00:00 PM
It seems to be doing this consistently; in every search I've done, all the results I've checked have had a start date of one day earlier than the StartDate in the responseData.
I can't seem to step into the JsonConvert.DeserializeObject method, so I'm not sure how to debug this issue; it's this one step where things go wrong, and I have no idea what the problem is. The StartDate (and probably EndDate, though I haven't checked) is the only thing getting deserialized wrong.
The Microsoft Json serializer (and resulting Json deserializer in JavaScript) will convert the values to UTC, even if you specify a time as UTC on the server when setting the variable. I had this problem recently and it about drove me crazy since it only happens when returning the data from the server, not when sending from JavaScript. In the end I had to convert the date to a string before returning to the client, then used the momentjs library to convert it back to a date and time. Everything worked perfectly then.
public class EventResult : Result
{
...
public DateTime StartDate { get; set; }
public string StartDateString { get {return StartDate.ToString() } };
...
}
Then in your JS:
var startDate = moment(Results.Results[0].StartDateString)