GET request with JSON body in SugarCRM - json

I'm using SugarCRM rest API, and according to the documentation, to get a set of records, I have to use /<module> GET endpoint and pass JSON in the body to filter the query.
First, is it even possible to have a body in a GET request ?
and how can I build this kind of request then ?
I'm using postman and tried to pass parameters as query strings but it's not possible though.

As far as I know you have to put everything in the query string, which might look different to what you'd expect.
Example for a request to /Users:
{
max_num: 100,
fields: ["first_name", "last_name"],
filter: [
{"user_name":"admin"}
{"status":"Active"}
]
}
Written as query string this request will look like this:
/rest/v10/Users?max_num=100&fields=first_name,last_name&filter[0][user_name]=admin&filter[1][status]=Active
Observations regarding the query string format:
There is no { or }, the values of the request object are placed directly in the query string
Key-Value pairs are assigned with =, and separated by & (instead of : and ,)
There are no " or ' quotes at all, strings are written without those
An array of values (here: fields) is just one assignment with all values separated by ,
An array of objects (here: filter) has one Key-Value pair per bottom value and uses [ and ] to indicate the "path" to each value. Using 0-based numerical indices for arrays
Notes
Keep in mind there are length limits to URL incl. query string. E.g. 4096 bytes/chars for Apache 2, if I remember correctly. If you have to send very elaborate requests, you might want to use POST /rest/v10/<module>/filter instead.
URL-escaped (usually not necessary) the example filter would look like this:
/rest/v10/Users?max_num%3D100%26fields%3Dfirst_name%2Clast_name%26filter%5B0%5D%5Buser_name%5D%3Dadmin%26filter%5B1%5D%5Bstatus%5D%3DActive

Related

Processing a Kafka message using KSQL that has a field that can be either an ARRAY or a STRUCT

I'm consuming a Kafka topic published by another team (so I have very limited influence over the message format). The message has a field that holds an ARRAY of STRUCTS (an array of objects), but if the array has only one value then it just holds that STRUCT (no array, just an object). I'm trying to transform the message using Confluent KSQL. Unfortunately, I cannot figure out how to do this.
For example:
{ "field": {...} } <-- STRUCT (single element)
{ "field": [ {...}, {...} ] } <-- ARRAY (multiple elements)
{ "field": [ {...}, {...}, {...} ] <-- ARRAY (multiple elements)
If I configure the field in my message schema as a STRUCT then all messages with multiple values error. If I configure the field in my message schema as an ARRAY then all messages with a single value error. I could create two streams and merge them, but then my error log will be polluted with irrelevant errors.
I've tried capturing this field as a STRING/VARCHAR which is fine and I can split the messages into two streams. If I do this, then I can parse the single value messages and extract the data I need, but I cannot figure out how to parse the multivalue messages. None of the KSQL JSON functions seem to allow parsing of JSON Arrays out of JSON Strings. I can use EXTRACTJSONFIELD() to extract a particular element of the array, but not all of the elements.
Am I missing something? Is there any way to handle this reasonably?
In my experience, this is one use-case where KSQL just doesn't work. You would need to use Kafka Streams or a plain consumer to deserialize the event as a generic JSON type, then check object.get("field").isArray() or isObject(), and handle accordingly.
Even if you used a UDF in KSQL, the STREAM definition would be required to know ahead of time if you have field ARRAY<?> or field STRUCT<...>
I finally solved this in a roundabout way...
First, I created an initial stream reading the transaction as a stream of bytes using KAFKA format instead of JSON format. This allows me to put a filter conditional filter on the data so I can fork the stream into a version for the single (STRUCT) variation and a version for the multiple (ARRAY) variation.
The initial stream looks like:
CREATE OR REPLACE STREAM `my-topic-stream` (
id STRING KEY,
data BYTES
)
WITH (
KAFKA_TOPIC='my-topic',
VALUE_FORMAT='KAFKA'
);
Forking that stream looks like this with a second for a multiple version filtering for IS NOT NULL:
CREATE OR REPLACE STREAM `my-single-stream`
WITH (
kafka_topic='my-single-topic'
) AS
SELECT *
FROM `my-topic-stream`
WHERE JSON_ARRAY_LENGTH(EXTRACTJSONFIELD(FROM_BYTES(data, 'utf8'), '$.field')) IS NULL;
At this point I can create a schema for both variations, explode field, and merge the two streams back together. I don't know if this can be refined to be more efficient, but this successfully processes the transactions as I wanted.

How to extract all id's from a JSON response using JSON JMESPasth Extractor in Jmeter?

I created a get request to get all the entries which is 997 entries and used JSON JMESPath Extractor to store the entries ids in a list. The JMESPath expression I used is entries[*].id and I used ForEach Controller to loop through the list and use each value of the list in next http request. The problem is that I want to get the ids of all 997 entries in the list, not just 10 values which is given at once by JMESPath extractor. I tried to take the length of entries as len and used While Controller to loop 100 times so that I can execute the next http request about 1000 times with condition ((${len}+3)/10). But, no matter how many times the loop executes, the values in the list obtained from JMESPath Extractor remains the same.
The Json response I get is similar as shown below.
{
"total": 997,
"entries": [
{
"id": 123389,
"parent-customer": [
{
"id": 123775,
"value": "Donal Thornber"
}
]
}
}
I have only kept one entry for reference.
If you need to get all the IDs in a single shot - the easiest would be going for JSON Extractor, it provides .. deep scan operator so the expression like $..id will match all the id attributes values no matter where they are:
More information: API Testing With JMeter and the JSON Extractor

How to parse Cosmosdb insert return value of (insertedIds)

When I insert an object into CosmosDB (MongoDB API), the result contains a property insertedIds.
When I console.log(insertedIds) I get
[ 5a6c46c85ac3cc4bb01ebcbb,
5a6c46c85ac3cc4bb01ebcbc,
5a6c46c85ac3cc4bb01ebcbd ]
and the typeof() is an object, for each element, although I'm not sure why - they just seem to be strings.
When I go through and JSON.stringify each element, I get (with surrounding double quotes) "5a6c46c85ac3cc4bb01ebcbb","5a6c46c85ac3cc4bb01ebcbc","5a6c46c85ac3cc4bb01ebcbd"
What is the right way to parse a CosmosDB Insert Result, and get the insertedIds as an array of strings?
Do I really have to go through and "stringify" then "de-string by removing quotes" for each returned Id? That is a huge overhead with large arrays.
Note: I believe this has something to do with MongoDB's bson/strict json: https://docs.mongodb.com/manual/reference/mongodb-extended-json/ but still not sure how to parse it.
you could use the following example:
JSON.parse(this_is_double_quoted);
JSON.parse("House");
.parse would return the value without quotes. more information can be found in the following post: remove double quotes from Json return data using Jquery

Is {0:{"id":1,...},{"id:2,....}} a other reprensation of a JSON list like [{"id":1,...},{"id:2,....}]

I have a little dilema. I have a backend/Frontend Application that comunicates with a JSON based REST Api.
The backend is written in PHP(Symfony/jmsserializer) and the Frontend in Dart
The communication between these two has a little Problem.
For most List Data the backend responds with a JSON like
[{"id":1,...},{"id:2,....}]
But for some it responds with
{"0":{"id":1,...}, "1":{"id:2,....}}
Now my Question is should the backend respond with the later at all or only with the first?
Problem
You usually have a list of objects. You sometimes get an object with sub-objects as properties.
Underlying issue
JS/JSON-Lists are ordered from 0 upwards which means that if you have PHP-Array which does not respect this rule json_encode will output a JS/JSON-Object instead using the numeric indices as keys.
PHP-Arrays are ordered maps which have more features that the JSON-Lists. Whenever you're using those extra features you won't be able to translate directly into JSON-Lists without loosing some information (ordering, keys, skipped indices, etc.).
PHP-Arrays and JSON-Objects on the other hand are more ore less equivalent in terms of features and can be correctly translated between each other without any loss of information.
Occurence
This happens if you have an initial PHP-Array of values which respects the JS/JSON-List rules but the keys in the list of objects are modified somehow. For example if you have a custom indexing order {"3":{}, "0":{}, "1":{}, "2":{}} or if you have (any) keys that are strings (ie. not numeric).
This always happens if you want to use the numeric id of the object as the numeric index of the list {"123":{"id": 123, "name": "obj"}} even if the numeric ids are in ascending order... so long as they are not starting from 0 upwards it's not a json-list it's a json-object.
Specific case
So my guess is that the PHP code in the backend is doing something like fetching a list of objects but its modifying something about it like inserting them by (string) keys into the array, inserting them in a specific order, removing some of them.
Resolution
The backend can easily fix this by using array_values($listOfObjects) before using json_encode which will reindex the entire list by numeric indices of ascending value.
Arrays and dictionaries are two separate types in JSON ("array" and "object" respectively), but PHP combines this functionality in a single array type.
PHP's json_encode deals with this as follows: an array that only contains numeric keys ($array = ['cat', 'dog']) is serialized as JSON array, an associative array that contains non-numeric keys ($array = ['cat' => 'meow', 'dog' => 'woof']) is serialized as JSON object, including the array's keys in the output.
If you end up with an associative array in PHP, but want to serialize it as a plain array in JSON, just use this to convert it to a numerical array before JSON encoding it: $array = array_values($array);

JSON oData.metadata

I have questions about JSON returning from the server using the Microsoft oData API.
Cannot figure it out.
Query1:
http://localhost:63717/odata/City(1)
Fiddler returns the raw data below.
Everything is in its own brackets.
{
"odata.metadata":"http://localhost:63717/odata/$metadata#City/#Element","CityID":1,"CityName":"Minnetonka","CityAddr1":null,"CityAddr2":null,"CityCity":null,"CityState":null,"CityZip":null,"CityPhone":null,"CityFAX":null,"CityExtent":"-93.53,44.88,-93.39,44.93","CityHeaderImage":null
}
Query2:
http://localhost:63717/odata/City?$filter=CityName eq 'Minnetonka'
Fiddler returns the raw data below.
Data is in two sets of bracketed data
{
"odata.metadata":"http://localhost:63717/odata/$metadata#City","value":[
{
"CityID":1,"CityName":"Minnetonka","CityAddr1":null,"CityAddr2":null,"CityCity":null,"CityState":null,"CityZip":null,"CityPhone":null,"CityFAX":null,"CityExtent":"-93.53,44.88,-93.39,44.93","CityHeaderImage":null
}
]
}
What do I have to do to format my JSON coming back for $filters in the oData request?
That odata.metadata is killing me in Query2.
Please explain what I am doing wrong.
In the first example, you have just one City element (denoted by City(1) in the request and #City/#Element in the result path).
In the second example, the value property in result is showing an array of City types (a listing of one or more objects). [ ... ] denotes an array in JavaScript. For a $filter type query, this is what I would expect. You can also see that the response path is less specific (#City instead of #City/#Element).
The path shown in the odata.metadata property value describes the structure of the element being returned, as I showed two examples above. The format of the return data will change depending on how you request it.
If you're having trouble parsing the JSON returned, consider using a library to do the heavy lifting for you. For example:
datajs
JayData
Breeze.js
[Source]
You are not doing anything wrong, the two formats actually represent two different forms of result.
The first you are requesting a single item as you are specifying the key for the entity.
In the second you are potentially asking for a list of entities. The Odata.Metadata is separate in this response otherwise it would be repeated for every item returned and would be a waste in terms of content length.
Because of the way that you are addressing the entity.
With //localhost:63717/odata/City(1) you are addressing one entity ("/entityset/key"). You will always return back one City (if one exists). There is no need for it to return an array because it will never return more than one.
With //localhost:63717/odata/City you are addressing a collection of entities ("/entityset"). 0 to n City entities could be returned, hence the need for a collection.