Unable to process JSON input when ingesting logs to Oracle - json

Question

Error:
send: b'{"specversion": "1.0", "logEntryBatches": [{"entries": [{"data": "{\\"hello\\": \\"oracle\\", \\"as\\": \\"aaa\\"}", "id": "ocid1.test.oc1..jkhjkhh23423fd", "time": "2021-04-01T12:19:28.416000Z"}], "source": "EXAMPLE-source-Value", "type": "remediationLogs", "defaultlogentrytime": "2021-04-01T12:19:28.416000Z"}]}'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Date: Fri, 02 Apr 2021 07:39:16 GMT
header: opc-request-id: ER6S6HDVTNWUOKCJ7XXZ/OpcRequestIdExample/770899C2C7CA6ABA11D996CC57E8EE8F
header: Content-Type: application/json
header: Connection: close
header: Content-Length: 79
Traceback (most recent call last):
File "tool.py", line 45, in <module>
put_logs_response = loggingingestion_client.put_logs(
File "/home/ubuntu/.local/lib/python3.8/site-packages/oci/loggingingestion/logging_client.py", line 172, in put_logs
return self.base_client.call_api(
File "/home/ubuntu/.local/lib/python3.8/site-packages/oci/base_client.py", line 276, in call_api
response = self.request(request)
File "/home/ubuntu/.local/lib/python3.8/site-packages/oci/base_client.py", line 388, in request
self.raise_service_error(request, response)
File "/home/ubuntu/.local/lib/python3.8/site-packages/oci/base_client.py", line 553, in raise_service_error
raise exceptions.ServiceError(
oci.exceptions.ServiceError: {'opc-request-id': 'ER6S6HDVTNWUOKCJ7XXZ/OpcRequestIdExample/770899C2C7CA6ABA11D996CC57E8EE8F', 'code': 'InvalidParameter', 'message': 'Unable to process JSON input', 'status': 400}
I am trying to send json data to Oracle logs, but getting the above error. I am using json.dumps(data) to convert the dict to string. Kindly let me know if any workaround available to this.
Code:
data = {'hello':'oracle', "as":"aaa"}
put_logs_response = loggingingestion_client.put_logs(
log_id="ocid1.log.oc1.iad.<<Log OCID>>",
put_logs_details=oci.loggingingestion.models.PutLogsDetails(
specversion="1.0",
log_entry_batches=[
oci.loggingingestion.models.LogEntryBatch(
entries=[
oci.loggingingestion.models.LogEntry(
data= json.dumps(data),
id="ocid1.test.oc1..jkhjkhh23423fd",
time=datetime.strptime(
"2021-04-01T12:19:28.416Z",
"%Y-%m-%dT%H:%M:%S.%fZ"))],
source="EXAMPLE-source-Value",
type="Logs",
defaultlogentrytime=datetime.strptime(
"2021-04-01T12:19:28.416Z",
"%Y-%m-%dT%H:%M:%S.%fZ"))]),
timestamp_opc_agent_processing=datetime.strptime(
"2021-04-01T12:19:28.416Z",
"%Y-%m-%dT%H:%M:%S.%fZ"),
opc_agent_version="EXAMPLE-opcAgentVersion-Value",
opc_request_id="ER6S6HDVTNWUOKCJ7XXZ/OpcRequestIdExample/")

Answer 1

This exception indicates that you have an InvalidParameter in your JSON input.
oci.exceptions.ServiceError: {'opc-request-id': 'ER6S6HDVTNWUOKCJ7XXZ/OpcRequestIdExample/770899C2C7CA6ABA11D996CC57E8EE8F', 'code': 'InvalidParameter', 'message': 'Unable to process JSON input', 'status': 400}
The InvalidParameter is your timestamp, which is date - 2021-04-01T12:19:28.416Z.
According to Oracle's documentation you need to use a RFC3339-formatted date-time string with milliseconds precision when creating a LogEntry.
This code snippet is from oci-python-sdk - log_entry.py, but it doesn't mention the milliseconds precision like Oracle's documentation.
#time.setter
def time(self, time):
"""
Sets the time of this LogEntry.
Optional. The timestamp associated with the log entry. An RFC3339-formatted date-time string.
If unspecified, defaults to PutLogsDetails.defaultlogentrytime.
:param time: The time of this LogEntry.
:type: datetime
"""
self._time = time
This code create a UTC RFC3339 complaint timestamp with milliseconds precision
from datetime import datetime
from datetime import timezone
current_utc_time_with_offset = datetime.now(timezone.utc).isoformat()
print(current_utc_time_with_offset)
#output
2021-04-06T13:00:52.706040+00:00
current_utc_time_with_timezone = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
print(current_utc_time_with_timezone)
#output
2021-04-06T13:09:10.053432Z
This Stack Overflow question is worth a read:
What's the difference between ISO 8601 and RFC 3339 Date Formats?
This article is also useful:
Understanding about RFC 3339 for Datetime and Timezone Formatting in Software Engineering

Answer 2

Your data looks fine , I think the issue is that Time precision is more than millisec. it should work fine if you loose the trailing zeros in time.
Format a datetime into a string with milliseconds
https://docs.oracle.com/en-us/iaas/api/#/en/logging-dataplane/20200831/LogEntry/

Answer 3

Your time is RFC3339 but precision is more than millisec
'{"specversion": "1.0", "logEntryBatches": [{"entries": [{"data": "{\"hello\": \"oracle\", \"as\": \"aaa\"}", "id": "ocid1.test.oc1..jkhjkhh23423fd", "time": "2021-04-01T12:19:28.416000Z"}], "source": "EXAMPLE-source-Value", "type": "remediationLogs", "defaultlogentrytime": "2021-04-01T12:19:28.416000Z"}]}'
See https://docs.oracle.com/en-us/iaas/api/#/en/logging-dataplane/20200831/LogEntry/
The timestamp associated with the log entry. An RFC3339-formatted date-time string with milliseconds precision. If unspecified, defaults to PutLogsDetails.defaultlogentrytime.