I am trying to ingest the json message into AZURE Event Hub
My problem is size of json message since Event hub has a limit of 1MB
I have one big json message which consists of multiple json message
DATA = [{"Id": "393092", "UID": "7f0034ee", "date": "2023-01-06", "f_id": "430", "origin": "CN"}, {"Id": "393092", "UID": "7f0034ee", "date": "2023-01-06", "f_id": "430", "origin": "CN"}, {"Id": "393092", "UID": "7f0034ee", "date": "2023-01-06", "f_id": "430", "origin": "CN"}]
This Data is an example.
DATA is already in json format but DATA contains 10000+ json event with same format
I would like to ingest this json message into event hub
Can anyone help me out How can I ingest this one big message into event hub? by slicing it or some other way
Thanks a lot !
Yoonsik
I tried slicing it but the number of events within one json message always different and very big...
Related
I'm trying to load a json file from a server response and parsing it in flutter, the model i create is working for all the other fields but i'm in trouble with this class
this is a part of the JSON response:
"episodes": {
"1": [
{
"id": "63",
"episode_num": 1,
"title": "Some Name",
"container_extension": "mp4",
"info": {
"director": "",
"plot": "",
"cast": "",
"rating": "",
"releasedate": "",
"movie_image": "",
"genre": "",
"duration_secs": 6495,
"duration": "01:48:15"
}
}
]
}
in this case the entry under episodes is just one but this will represents a season and all the episode inside it, so under episodes many of this entry (undefined number during coding) can be present
At this time, using online json to dart converter tools i can be able to retrive just this one entry but if a response have more than 1 season i can't see it.
There is any way to handle this?
EDIT:
Solved using a for cicle with max value = (json['episodes'].length + 1).
For the info stored inside each 'episodes' value i can use
json['episodes']['$i']
Valid JSON is always convertible to a Dart data structure. But what you may be asking is "can I get nested objects from this?", and that just depends on how hard you want to work. Some JSON-to-Dart tools are better than others and some JSON values are impossible for any automated tool to make sense of. Only real answer is: "it depends".
i'm making a new application using Parse.com as a backend, i'm trying to make less requests to the Parse, I have a class which is pointing to another object of another class.
Class1(things):
ObjectID Name Category(pointer)
JDFHSJFxv Apple QSGKqf343
Class2(Categories):
ObjectID Name Number Image
QSGKqf343 Fruits 45 http://myserver.com/fruits.jpeg
when i'm trying to retreive data for my first class things using REST API i'm getting this json object :
{
"results": [
{
"Name": "Apple",
"createdAt": "2015-07-12T02:50:20.291Z",
"objectId": "JDFHSJFxv",
"category": {
"__type": "Pointer",
"className": "Teams",
"objectId": "QSGKqf343"
},
"updatedAt": "2015-07-12T02:55:33.696Z"
}
]
}
the json doesn't contains all the data included in the object i'm pointing to, I will have to make another request to get all the data of that object,
is There any way to fix that
You need to tell Parse to return the related object in your query, via the include key.
e.g., add the following to your CURL --data-urlencode 'include=category'
I am writing a mobile client that consumes JSON data from a 3rd party server (one which I have no control on). The problem is when I do a get on the following URL
curl http://server.com/zm/api/events.json
It returns a big list of events structured like so:
"events": [
{
"Event": {
"Id": "280",
"MonitorId": "1",
"Name": "Familyroom-280",
"Cause": "Motion",
"StartTime": "2015-04-12 06:54:43",
"EndTime": "2015-04-12 06:55:27",
"Width": "1280",
"Height": "960",
"Length": "44.24",
"Frames": "74",
"AlarmFrames": "23",
"TotScore": "973",
"AvgScore": "42",
"MaxScore": "279",
"Archived": "0",
"Videoed": "0",
"Uploaded": "0",
"Emailed": "0",
"Messaged": "0",
"Executed": "0",
"Notes": "Motion: mudroom door, study door"
}
}, //and many more such "Event" events inside the array
I'm new to JSON, but when I read up, its specification say the server MUST provide a mechanism to extract any sub object. So, for example, I want to construct a URL to ONLY retrieve events which have a MonitorId of 1
I've tried
curl http://server.com/zm/api/events/Event/MonitorId/1.json
curl http://server.com/zm/api/Event/MonitorId/1.json
curl http://server.com/zm/api/events.json?MonitorId="1"
curl http://server.com/zm/api/events.json?Event.MonitorId="1"
But I can't seem to get it right. Can someone advise what is the URL I need to construct to only return elements where MonitorId="1"?
thanks!
By REST you don't start to construct URLs. That's the responsibility of the server, if it does not provide hyperlinks with URLs (or URI templates), then we are not talking about REST.
Try a different response format, maybe there is HTML, ATOM, JSON-LD, HAL+JSON, etc... something which contains hyperlinks.
I thought this is like xpath and I can drill down to whatever level I
want.
Most of REST services do not support custom queries.
Take a look at ZoneMinder docs:
Return a list of events for a specific monitor Id
Example :
curl -XGET http://server.com/zm/api/events/index/MonitorId:1.json
Or you can just filter your JSON output with the jq command.
Example:
curl http://server.com/zm/api/events.json | jq '.events | map (.Event | select(.MonitorId==1))'
What is the right way to format your responses in JSON and why? I've seen different services do it two ways, consider a simple GET /users resource:
{
"success": true,
"message": "User created successfully",
"data": [
{"id": 1, "name": "John"},
{"id": 2, "name": "George"},
{"id": 3, "name": "Bob"},
{"id": 4, "name": "Jane"}
]
}
That is how I usually do that. I have some abstract helper fields like success and message, there may be some more but the question is if should I nest the data in the data field to an array called the same way as the resource - users:
{
"success": true,
"message": "User created successfully",
"data": {
"users": [
{"id": 1, "name": "John"},
{"id": 2, "name": "George"},
{"id": 3, "name": "Bob"},
{"id": 4, "name": "Jane"}
]
}
}
Even if we don't use the abstraction:
{
"users": [
{"id": 1, "name": "John"},
{"id": 2, "name": "George"},
{"id": 3, "name": "Bob"},
{"id": 4, "name": "Jane"}
]
}
Seems the users key is obsolete as any client will know the route they called, which consists of /users, where users are mentioned, and the client code like
$users = $request->perform('http://this.api/users')->body()->json_decode();
looks much better than
$users = $request->perform('http://this.api/users')->body()->json_decode()->users;
as it avoids repeated users.
One use case where the envelope can be useful is when you are expecting to be dealing with large lists and need to do pagination to prevent huge response payloads. The envelope is a good place to put the pagination meta data:
{
"users": [...],
"offset": 0,
"limit": 50,
"total": 10000
}
(This is what we do in a RESTful API I'm working on)
Clearly this is only relevant for requests that return lists of things (e.g. /users/) and not for requests that return single entities (e.g. /users/42) and even for requests that return lists, you don't have to use an envelope - one alternative would be to use response headers for this meta data instead.
PS. I would only advise having a success and message fields if you have a concrete use case for them. Otherwise don't bother, they are simply unnecessary.
Just to get on the same page, data is a field in a JSON object. In the first example the value of data is an array. In the second example the value of data is an object.
Either is valid, so to answer your question: no it is not necessary to nest named objects in an named object. It is necessary that all fields of an object be named, but you are free to nest arrays within an object.
It really just depends on what the processor expects. If data can be anything, then the first approach is fine. If code expects the value of the data field to be an object, then you have to use something like the second example.
According to your comment which you added to first comment: more descriptive data is better data as every information is useful for consumer of you API - REST endpoint. So if you know that the content is user, or whatever, it's better to use it in schema or endpoint url.
Better description = better consuption :-)
var data = xhr.responseText;
When I output this console.log(xhr.responseText). Below is my output
["{id:1,name\":\"JOHN\",\"city\":\"null\"}"
,"{\"id\":2,\"name\":\"MICHEAL\,\"city\":\"null\"}"]
How do I get id, name. I tried like this data.id but I get this error
jquery JSON.parse: unexpected end of data.
Update
I am using code igniter with data mapper so my data mapper is giving that json response. Do you know, how I can resolve it.
You've already been told what the problem is in the comments: the JSON generated by the server is invalid. You are probably not using a library to encode your JSON, don't ever encode it by hand.
Your JSON should probably look like the following (when pretty printed) http://jsfiddle.net/7FKWr/
[
{"id": 1, "name": "JOHN", "city": null},
{"id": 2, "name": "MICHEAL", "city": null}
]