Temporary Tables (CTE) in MongoDB - mysql

I know that common table expressions (CTE) a.k.a. "temporary named result sets" can be used in SQL to generate a temporary table, but can this be done in MongoDB? I want a document, but it's only for temporary use in my query.
Can you create a temporary table in MongoDB without creating a new collection?
For example, if I were to try to recreate the code below in Mongo...
Example CTE Table in SQL:
n
f1
f2
1
20
12
2
40
0.632
3
60
0.647
WITH RECURSIVE example (n, f1, f2) AS
( SELECT 1, 20, 12
UNION ALL SELECT
n + 1,
n * 20,
least(6*n, $globalVar * 100),
FROM example WHERE n < 3
) SELECT * FROM example

It seems that there is no general equivalent for CTE in MongoDB. However, for OP's example, it is possible to wrangle the output of $range to produce a similar effect.
// whichever collection doesn't matter; as long as it has 1 document then it should be fine
db.collection.aggregate([
{
// jsut take 1 document
"$limit": 1
},
{
// use $range to generate iterator [1, 2, 3]
"$addFields": {
"rg": {
"$range": [
1,
4
]
},
globalVar: 0.001
}
},
{
// do the mapping according to logic
"$addFields": {
"cte": {
"$map": {
"input": "$rg",
"as": "n",
"in": {
n: "$$n",
f1: {
"$multiply": [
"$$n",
20
]
},
f2: {
"$cond": {
"if": {
$lt: [
{
"$multiply": [
"$$n",
6
]
},
{
"$multiply": [
"$globalVar",
100
]
}
]
},
"then": {
"$multiply": [
"$$n",
6
]
},
"else": {
"$multiply": [
"$globalVar",
100
]
}
}
}
}
}
}
}
},
{
// wrangle back to expected form
"$unwind": "$cte"
},
{
"$replaceRoot": {
"newRoot": "$cte"
}
}
])
Here is the Mongo playground for your reference.

Related

Modify nested arrays in JSON (Groovy)

I'm trying to parse and mofidy JSON with Groovy. Source JSON from REST API looks like:
[
{
"id":27858,
"type":"ad",
"stats":[
{
"day":"2021-01-21",
"sex":[
{
"impressions_rate":0.349,
"value":"f"
},
{
"impressions_rate":0.621,
"value":"m",
"clicks_rate":0.22
}
],
"age":[
{
"impressions_rate":0.217,
"value":"18-21"
}
]
},
{
"day":"2021-02-25",
"sex":[
{
"impressions_rate":0.349,
"value":"f"
},
{
"impressions_rate":0.651,
"value":"m"
}
],
"age":[
{
"impressions_rate":0.217,
"value":"18-21"
}
]
}
]
},
{
"id":565291,
"type":"ad",
"stats":[
{
"day":"2021-03-21",
"sex":[
{
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
},
{
"impressions_rate":0.551,
"value":"m"
}
],
"age":[
{
"impressions_rate":0.17,
"value":"18-21"
}
]
}
]
}
]
It's an array with some ids and data for them. I want to grab id, day inside stats array and elements from sex array. After all manipulations my JSON should be like this:
[
{
"id": 27858,
"day": "2021-01-21",
"impression_rate": 0.349,
"value": "f"
},
{
"id": 27858,
"day": "2021-01-21",
"impression_rate": 0.621,
"value": "f",
"clicks_rate": 0.22
},
{
"id": 27858,
"day": "2021-02-25",
"impressions_rate":0.349,
"value":"f"
},
{
"id": 27858,
"day": "2021-02-25",
"impressions_rate":0.651,
"value":"m"
},
{
"id": 565291,
"day": "2021-03-21",
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
},
{
"id": 565291,
"day": "2021-03-21",
"impressions_rate":0.78,
"value":"f",
"clicks_rate":0.33
}
]
So, the main goal is - loop through all ids -> elements in sex array (for each id) and add to these elements day and id mapped fields. I tried to start with empty map with inject, but after 1 hour of debugging i still can't achieve desired output, so maybe better to loop through existed values in array? But I can't even reach sex array.
import groovy.json.*
def json = new JsonSlurper().parseText '''...'''
List expected = json.inject([]){ r, e ->
Map ids = e.findAll {
k, v -> k == "id"
}
e.each{ k, v ->
if( (v.any{ it.sex } )
v.each{ r << ids + it }
}
return r
}
If you have nested structures, that contain again nested structures,
a good option, to get a flat result, is to use collectMany; like
collect it transforms each item of the iterated container, but the
results gets concated.
E.g. you can collectMany on your outer data, then again on the
stats, and finally just collect over sex.
def data = new groovy.json.JsonSlurper().parse("data.json" as File)
println data.collectMany{ o ->
o.stats.collectMany{ i ->
i.sex.collect{ it + [id: o.id, day: i.day] }
}
}
// [[impressions_rate:0.349, value:f, id:27858, day:2021-01-21],
// [impressions_rate:0.621, value:m, clicks_rate:0.22, id:27858, day:2021-01-21],
// [impressions_rate:0.349, value:f, id:27858, day:2021-02-25],
// [impressions_rate:0.651, value:m, id:27858, day:2021-02-25],
// [impressions_rate:0.78, value:f, clicks_rate:0.33, id:565291, day:2021-03-21],
// [impressions_rate:0.551, value:m, id:565291, day:2021-03-21]]

jq sort by version as string

I'm trying to sort the following json reponse to pick the latest version:
[
{
"TagVersion": "1.0.11"
},
{
"TagVersion": "1.1.8"
},
{
"TagVersion": "1.0.10",
},
{
"TagVersion": "1.0.9",
},
{
"TagVersion": "1.0.77"
}
]
Correct sorting should be:
{
"TagVersion": "1.0.9",
},
{
"TagVersion": "1.0.10",
},
{
"TagVersion": "1.0.11"
},
{
"TagVersion": "1.0.77"
},
{
"TagVersion": "1.1.8"
}
I'm currently able to do part of the job. It's working for simple cases (all version part major/minor/bug has the same number of digits).
jq -r [.[]]|max_by(.TagVersion|split(".") | map(tonumber)
The best way to do it in my mind should be to add a multiplication the each part. Example:
# With condition than every "part" as a maximum of 2 digits. It won't work with 3 digits
# Version 1.23.87
1 * 1000 + 23 * 10 + 87 = 1317
# Version 3.0.0
1 * 1000 + 0 * 10 + 0 = 3000
# Version 1.89.78
1 * 1000 + 89*10 + 78 = 1968
Does anybody have an idea to implement this? 🙂
Turn each component into a number, then sort on the array of integers.
jq 'sort_by(.TagVersion|split(".")|map(tonumber))'
Output:
[
{
"TagVersion": "1.0.9"
},
{
"TagVersion": "1.0.10"
},
{
"TagVersion": "1.0.11"
},
{
"TagVersion": "1.0.77"
},
{
"TagVersion": "1.1.8"
}
]

Elasticsearch Nest incorrect function_Score JSON generation

I am trying to build the following function for the function_score search query:
{
"filter": {
"range": {
"availabilityAverage": {
"gt": 0
}
}
},
"field_value_factor": {
"field": "availabilityAverage",
"factor": 1,
"modifier": "log1p"
},
"weight": 100
}
This is currently my .Net code
.FieldValueFactor(ff => ff
.Field(fff => fff.StandardPriceMin)
.Factor(2)
.Modifier(FieldValueFactorModifier.Log1P)
.Weight(100)
.Filter(faf => faf
.Range(r => r
.Field(rf => rf.AvailabilityAverage)
.GreaterThan(0.0)
)
)
)
However, this is the result of the NEST query:
{
"filter": {
"range": {
"availabilityAverage": {
"gt": 0.0
}
}
},
"field_value_factor": {
"factor": 2.0,
"field": "standardPriceMin",
"modifier": "log1p",
"filter": {
"range": {
"availabilityAverage": {
"gt": 0.0
}
}
},
"weight": 100.0
},
"weight": 100.0
}
It is adding correctly the filter and weight on the outside of field_value_factor but also including the 'Filter' and 'weight' on the inside as a child element. This is not the case for others such as RandomScore() with exact same format but only with field_value_factor.
I tried several different combinations but neither provided expected result. Is it normal that the NEST is generating this JSON?
Thanks in advance.
It looks like there's a bug in how IFieldValueFactorFunction is being serialized, resulting in filter and weight being included twice, outside of "field_value_factor" and inside. I've opened a pull request to address.

Add new fields to nested JSON array in JSONB

I have a nested JSON structure stored in a PostgreSQL table.
Table users:
id | content [JSON]
JSON:
{
"purchases": [
{
"id": 1,
"costs": [
{
"amount": 23
},
{
"amount": 34
}
]
},
{
"id": 2,
"costs": [
{
"amount": 42
}
]
}
]
}
I would like to add a field "jsonClass": "Static" to all the objects within the costs array so I have following in the end:
{
"purchases": [
{
"id": 1,
"costs": [
{
"jsonClass": "Static",
"amount": 23
},
{
"jsonClass": "Static",
"amount": 34
}
]
},
{
"id": 2,
"costs": [
{
"jsonClass": "Static",
"amount": 42
}
]
}
]
}
I couldn't figure out how to add values to such a nested structure. Anyone knows how to achieve such thing? The only way I found was to make it a text and do string replace which is not very performant and I have a lot of such entries.
Unfortunately, due to having to change multiple sub-objects, I don't know of a better way than to deconstruct and then reconstruct the object. It gets pretty hairy.
UPDATE users
SET content=(
SELECT jsonb_agg(purchase)
FROM (
SELECT jsonb_build_object('id', pid, 'purchases', jsonb_agg(cost)) AS purchase
FROM (
SELECT pid, cost || '{"jsonClass":"static"}'::jsonb AS cost
FROM (
SELECT purchase->'id' AS pid, jsonb_array_elements(purchase->'costs') AS cost
FROM jsonb_array_elements(content::jsonb->'purchases') AS purchase
) AS Q
) AS R
GROUP BY pid
) AS S
);
Fiddle
EDIT: Sorry about all the edits, forgot to test for multiple rows. Should be good now. It might be possible to simplify it a bit more, not sure.

Am I duplicating data in my API?

My gradepoints and percents objects hold the same values of grades with different keys. Please take a look at my json below and let me know if I'm doing it right. Is there a way to optimize this API?
I could provide the percents along with the gradepoints after a comma like "a1": "10,90" but this way I will need to split them up on client side JS, which I'm restraining from.
{
"gradepoints": [
{
"a1": 10
},
{
"a1": 10
},
{
"c2": 5
},
{
"e1": "eiop"
},
{
"d": 4
},
{
"b1": 8
}
],
"percents": [
{
"a1": 90
},
{
"a1": 90
},
{
"c2": 45
},
{
"e1": "eiop"
},
{
"d": 36
},
{
"b1": 72
}
],
"gpa": 7.4,
"overall": 70.3,
"eiop": 2
}
I would do it something like this:
{
grades: [
{ name: "a1",
gradepoint: 10,
percent: 90
},
{ name: "a1",
gradepoint: 10,
percent: 90
},
{ name: "c2",
gradepoint: 5,
percent: 45
},
...
],
gpa: 7.4,
overall: 70.3,
eiop: 2
}
Related data should be kept together in an object.
If it weren't for the duplicate a1 entries, I would probably make grades be an object, with the names as keys. But an object can't have duplicate keys, so it has to be put in the values.