I am facing an issue in which I am returning JSON string as AJAX Response. I have logging framework in-between which makes use of "periodic-rotating-file-handler" for logging the messages.
One of the user message which is getting logged is JSON String which doesn't come in correct format and " (Quotes) are getting converted into """ in the Log file
Related
I'm grabbing a JSON payload from a REST API which I intend to modify and then send back. I convert the payload to an object using ConvertFrom-Json, manipulate it using .Remove(), and convert it back to JSON with ConvertTo-Json.
When I send the modified payload back to the API, I get a "(400) Bad Request." Thinking I screwed something up when modifying the payload, I removed that part of the code but still got the same error, even though I'm "just" converting the unmodified payload from and to JSON.
Sending back an unmodified payload should result in a "(422) UNPROCESSABLE ENTITY" and if I send the payload back without even converting from and to JSON that's the response I get from the API.
In this case the -Depth parameter isn't the culprit as far as I can tell (I've added -Depth 100 just to be sure). I also output the payload before the ConvertFrom-Json and after the ConvertTo-Json operations, and the output looks identical on screen.
If I copy the output after the modifications and conversions from and to JSON and then paste that as the body in a Postman call - where all the other parameters are the same as in my code - the API call works as expected.
Checking the API logs on the server only contains this error: "BadRequest: 400 Bad Request: Failed to decode JSON object: 'utf8' codec can't decode byte 0xf6 in position 2976: invalid start byte"
This makes me think that the conversion changes the encoding of the string but I can't for the life of me figure out how that would be possible.
After a day and a half trying to get to the bottom of this I'm more than happy get any and all feedback on this problem. :)
Edit: This must work with PowerShell 5.1 as that's the only version I can safely assume is installed.
Thanks.
The solution to this was to use PowerShell 7.2. I spent almost 2 days trying to work around the limitations of JSON conversions in Windows PowerShell 5.1 to no avail.
In PowerShell 6.2+ you can add an -EscapeHandling variable to ConvertTo-Json with the value of EscapeNonAscii that will convert any non ASCII characters back to Unicode.
The downside to this is that I now have to ensure that there is PowerShell 6.2+ on the servers I remote into, but that's easier to control and manage than the results of the API call.
I've got a rsyslog server sending logs to logstash (6.8). For the (very) vast majority of logs that I receive (windows, unix, etc) everything parses just fine.
input {
codec => json {
charset => "ISO-8859-1"
}
}
}
However when I receive some logs (I stress some) for event ID 4688 that contain binary at the end of the log event like the below example - I receive the error:
Json ParserError: Unexpected end-of-input within/between Object
entries
-NoProfile -NonInteractive -EncodedCommand KABOAGUAdwAtAEkAdABlAG0AIAAtAFQAeQBwAGUAIABEAGkAcgBlAGMAdABvAHIAeQAgAC0AUABhAHQAaAAgACQAZQBuAHYAOgB0AGUAbQBwACAALQBOAGEAbQBlACAAIgBhAG4AcwBpAGIAbABlAC0AdABtAHAALQAxADQAMwAzADAAOAA3ADYANQA5AC4AMgA4AC0ANwA0ADQANAA1ADMANgA1ADQAMQA2ADgANwAyACIAKQAuAEYAdQBsAGwATgBhAG0AZQAgAHwAIABXAHIAaQB0AGUALQBIAG8AcwB0ACAALQBTAGUAcABhAHIAYQB0AG8AcgAgACcAJwA7AA==
I'm at a loss at how to parse it. I don't really care about the binary, is there a way for me to chop off a log event after it see's -EncodedCommand before it gets run through a filter? All other 4688 events parse without any issue and a grok filter (if it makes it past the json parse) passes without issue either.
I have an api endpoint that responds with a JSON Array as a string.
Correspondingly, I have an interface that matches the JSON response
I have a service that makes the request to get the array of users and logs the first record to the console.
Expected Results
I expect to get a UserDetails object back and should print all the contents of index 0 to the console.
Actual Results
In the console I just see the character '['. It seems that res variable is still being treated as a string, and not a UserDetails array.
I have been struggling for house to try and figure out what is causing this behavior
I found the problem for anyone who is interested.
My server is returning the response in the body as a JSON string (this is a requirement from AWS API Gateway). The following has to be done to get the data to correctly parse.
I'm currently using GCE Container VMs (not GKE) to run Docker Containers which write their JSON formated log to the Console. The Log Information is automatically collected and stored in Stackdriver.
Problem: Stackdriver displays the data-field of the jsonPayload as text - not as JSON. It looks like the quotes of the fields within the payload are escaped and therefore not recognized as JSON structure.
I used both, logback-classic (like explained here) and slf4j/log4j (using JSONPattern) to generate JSON output (which looks fine), but the output is not parsed correctly.
I assume that, I have to configure somewhere that the output is JSON structured, not plain text. So far I haven't found an option where to do this when using a Container VM.
What does your logger output into stdout?
You shouldn't create a jsonPayload field yourself in your log output. That field gets automatically created when your logs get parsed and meet certain criteria.
Basically, write your log message into a message field of your JSON output and any additional data as additional fields. Stackdriver strips all special fields from your JSON payload and if there is nothing left then your message will end up as textPayload otherwise you'll get a jsonPayload with your message and other fields.
Full documentation is here:
https://cloud.google.com/logging/docs/structured-logging
I use the JSON in python version 3.4 for the network programming. The server program was built with another language and it expects to recieve the message in string. I put the JSON query into string and when send it with sock.send(query.encode()), but I get the message incorrect data type. Is there exist a way to send the string without converting it?