How do I determine the column type name from the columnType integer value in mysql2? - mysql

When querying with mysql2, the third argument to the callback function fields has the following documentation:
console.log(fields); // fields contains extra meta data about results, if available
Ok, great. But when I look at the values in the fields array, I see the following:
[
{
"_buf":{},
"_clientEncoding":"utf8",
"_catalogLength":3,
"_catalogStart":10,
"_schemaLength":0,
"_schemaStart":14,
"_tableLength":11,
"_tableStart":15,
"_orgTableLength":0,
"_orgTableStart":27,
"_orgNameLength":2,
"_orgNameStart":31,
"characterSet":63,
"encoding":"binary",
"name":"id",
"columnLength":11,
"columnType":3, // <-- column type 3? This is an INTEGER field
"flags":1,
"decimals":0
},
{
...
"columnType":253, // <-- column type 253? This is a VARCHAR field
...
}
]
How do I determine what the actual column type is from these integer values?

After doing some more digging, I was able to find the answer by looking at the source. mysql2 exposes a Types field that has the column type names as keys with the column id as values.
At the time of posting, that list looks like this:
{
"DECIMAL": 0,
"TINY": 1,
"SHORT": 2,
"LONG": 3,
"FLOAT": 4,
"DOUBLE": 5,
"NULL": 6,
"TIMESTAMP": 7,
"LONGLONG": 8,
"INT24": 9,
"DATE": 10,
"TIME": 11,
"DATETIME": 12,
"YEAR": 13,
"NEWDATE": 14,
"VARCHAR": 15,
"BIT": 16,
"JSON": 245,
"NEWDECIMAL": 246,
"ENUM": 247,
"SET": 248,
"TINY_BLOB": 249,
"MEDIUM_BLOB": 250,
"LONG_BLOB": 251,
"BLOB": 252,
"VAR_STRING": 253,
"STRING": 254,
"GEOMETRY": 255
}

Related

How do I get sum of a field in solr in 8.11.1 version

"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"total_amountA": 10,
"total_amountB": 5,
"id": "2"
},
{
"total_amountA": 10,
"total_amountB": 5,
"id": "1"
}
]
}
i want to get the sum of all tatal_amount.
like this -> total_amountA : 20, total_amountB : 10
how can i get the sum of all value..? plese help me
The Stats Component gives you a set of statistics for a given field for a given query.
http://localhost:8983/solr/corename/select?q=your:query&stats=true&stats.field=total_amount
Should give you an additional stats key in the response with sum among the values calculated.

How to insert JSON containing escape codes into a JSONB column in PostgreSQL using GORM

I'm trying to store the JSON bytes to PostgreSQL, but there's a problem.
\u0000 cannot be converted to text.
As you can see below, the JSON contains escape sequences such as \u0000, which it seems PostgreSQL is interpreting as unicode characters, not JSON strings.
err := raws.SaveRawData(data, url)
// if there is "\u0000" in the bytes
if err.Error() == "ERROR: unsupported Unicode escape sequence (SQLSTATE 22P05)" {
// try to remove \u0000, but not work
data = bytes.Trim(data, "\u0000")
e := raws.SaveRawData(data, url) // save data again
if e != nil {
return e // return the same error
}
return nil
}
Origin API data can be access form Here. There is \u0000 in it:
{
"code": 0,
"message": "0",
"ttl": 1,
"data": {
"bvid": "BV1jb411C7m3",
"aid": 42443484,
"videos": 1,
"tid": 172,
"tname": "手机游戏",
"copyright": 1,
"pic": "http://i0.hdslb.com/bfs/archive/c76ee4798bf2ba0efc8449bcb3577d508321c6c5.jpg",
"title": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
"pubdate": 1549100438,
"ctime": 1549100438,
"desc": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
"desc_v2": [
{
"raw_text": "bgm:逮虾户\n今天先水一期冰塔的,明天再水\\u0000绿塔的,后天就可以下红莲啦,计划通嘿嘿嘿(º﹃º )",
"type": 1,
"biz_id": 0
}
],
"state": 0,
"duration": 265,
"rights": {
"bp": 0,
"elec": 0,
"download": 1,
"movie": 0,
"pay": 0,
"hd5": 0,
"no_reprint": 1,
"autoplay": 1,
"ugc_pay": 0,
"is_cooperation": 0,
"ugc_pay_preview": 0,
"no_background": 0,
"clean_mode": 0,
"is_stein_gate": 0
},
"owner": {
"mid": 39699039,
"name": "明眸-雅望",
"face": "http://i0.hdslb.com/bfs/face/240f74f8706955119575ea6c6cb1d31892f93800.jpg"
},
"stat": {
"aid": 42443484,
"view": 1107,
"danmaku": 7,
"reply": 22,
"favorite": 5,
"coin": 4,
"share": 0,
"now_rank": 0,
"his_rank": 0,
"like": 10,
"dislike": 0,
"evaluation": "",
"argue_msg": ""
},
"dynamic": "#崩坏3#",
"cid": 74479750,
"dimension": {
"width": 1280,
"height": 720,
"rotate": 0
},
"no_cache": false,
"pages": [
{
"cid": 74479750,
"page": 1,
"from": "vupload",
"part": "冰塔:我连你的大招都敢硬抗,所以告诉我谁才是生物女王?!单s冰塔怒砍档案女王巴德尔,谁,才是生物一姐?(手动滑稽)",
"duration": 265,
"vid": "",
"weblink": "",
"dimension": {
"width": 1280,
"height": 720,
"rotate": 0
}
}
],
"subtitle": {
"allow_submit": false,
"list": []
},
"user_garb": {
"url_image_ani_cut": ""
}
}
}
The struct for save is:
type RawJSONData struct {
ID uint64 `gorm:"primarykey" json:"id"`
CreatedAt time.Time `json:"-"`
DeletedAt gorm.DeletedAt `json:"-" gorm:"index"`
Data datatypes.JSON `json:"data"`
URL string `gorm:"index" json:"url"`
}
datatypes.JSON is from gorm.io/datatypes. It seems just is json.RawMessage, it is (extend from?) a []byte.
I use PostgreSQL's JSONB type for storage this data.
Table:
create table raw_json_data
(
id bigserial not null constraint raw_json_data_pke primary key,
created_at timestamp with time zone,
deleted_at timestamp with time zone,
data jsonb,
url text
);
The Unicode escape sequence \u0000 is simply not supported in Postgres TEXT and JSONB columns:
The jsonb type also rejects \u0000 (because that cannot be represented in PostgreSQL's text type)
You can change the column type to JSON:
create table Foo (test JSON);
insert into Foo (test) values ('{"text": "明天再水\u0000绿塔的"}');
-- works
The json data type stores an exact copy of the input text
This has the advantage of keeping the data identical to what you received from the API, in case the escape sequence has some meaning that you need to preserve.
It'll also allow you to query using Postgres JSON operators (e.g. ->>), albeit converting a JSON field with \u0000 to text will still fail:
select test->>'text' from Foo
-- ERROR: unsupported Unicode escape sequence
Columns of type BYTEA also accept any byte sequence without having to manipulate the data. In Gorm, use type:bytea tag:
type RawJSONData struct {
// ... other fields
Data string `gorm:"type:bytea" json:"data"`
}
If any of the above is not acceptable for you, then you must sanitize the input string...

jsonb_each to access elements inside objects

I have a JSON column (called "roi") which contains users' Instagram performance. This is the roi column:
{
"data": {
"campaignName": "Master Cosy",
"currency": "GBP",
"reportData": {
"AAAAAAAAAA": {
"id": "0f20d833-d0f-bdb7-19",
"name": "cornish_gregorys",
"thumbnail": "https://sstagram.com/v/t51.2885-19/s320x320/87244862_1017848048596",
"Name": "cornisorys",
"instagramCount": 2319,
"instagramEngagementFactor": 0,
"instagramAuthorised": true,
"hasPosts": true,
"budget": 0,
"derivedFee": 0,
"inventoryItems": [],
"trackedAssetsStats": {
"totalAssets": 9,
"facebook": {
"count": 0
},
"instagram": {
"total": 9,
"stories": 9,
"carousels": 0,
"videos": 0,
"images": 0,
"igtvs": 0
},
"BBBBBBBBBBBBB": {
"id": "d3d30db4-0b453dfc3ae2a09",
"name": "itssdha",
"thumbnail": "https://in9809609728_n.jpg?_nc_ht=instagram.fhel5-1.fna.fbcdn.net&_nc_ohc=Se3ySAoqnFwAX4f6&oeF1623",
"Name": "itsshdha",
"instagramCount": 26700,
"instagramEngagementFactor": 0,
"instagramAuthorised": true,
"hasPosts": true,
"budget": 0,
"derivedFee": 0,
"inventoryItems": [],
"trackedAssetsStats": {
"totalAssets": 5,
"facebook": {
"count": 0
},
"instagram": {
"total": 9,
"stories": 9,
"carousels": 0,
"videos": 0,
"images": 0,
"igtvs": 0}, etc.....
After "reportData" I have the specific names of the users (in this case AAAAAAAA and BBBBBBBBB) and within them the performance of their Instagram accounts. How can I access all the metrics within the object username without having to type the specific username (AAAAAAAA and BBBBBBBBB)
My query is this:
roi -> 'date' -> 'reportData' -> 'AAAAAAA' -> 'instagramCount' -> etc ....
But I need something to 'jump' this part -> 'AAAAAAA' -> and go straight to the metrics, in this case 'instagramCount', etc...
From what I have read I may need to use jsonb_each, does anyone know how to use it?
demos:db<>fiddle
You have several ways.
Use jsonb_each() to expand all users' data: You can create a record per user and than ask for the count afterwards
SELECT
users.value -> 'instagramCount'
FROM
mytable,
jsonb_each(mydata -> 'data' -> 'reportData') as users
Since Postgres 12 you can use JSONpath for that, to achieve the same:
SELECT
jsonb_path_query(mydata, '$.**.instagramCount')
FROM mytable

Custom JSON response structure in Laravel

I have an application form which text data is stored in one table. Let's say table A (only one entry). The attachments are stored in table B which have multiple entries (since the user can add multiple attachments) and they are referenced by an application number.
I want to send the data as a JSON response which is what I am doing via a leftJoin clause. I noticed that the data from table A is looping over and over until it exausts table B.
JSON sample:
{
"id": 1,
"BR_ID": 5,
"ClientID": 237,
"SLT_CODE": 12,
"PURPOSE": 1,
"PAMT": "100,000.00",
"TERMS": 36,
"PPMT_MODE": 4,
"PPMT_MFAC": 0,
"PPMT_MOD2": 4,
"INT_RATE": 12,
"PEN_RATE": 24,
"OL_TEMP_APP_NO": "04152020566167",
"APPLICATION_NO": null,
"ActivityListID": 1,
"DateRequested": "2020-04-15 14:22:36",
"DateApproved": null,
"ClientRemarks": "Every fight needs mending\nEvery start has an end\nLike the sunrise and the sunset\nThat's just how it is",
"FilePath": "C:\\Users\\Personal\\laravel\\storage\\app\\public\\documents\\04152020566167.20200221_141512.jpg",
"AttachmentName": "04152020566167.20200221_141512.jpg",
"created_at": null,
"updated_at": null
}
Basically all the data throughout the loop is the same except "FilePath" and "AttachmentName" since they are the ones from table B.
Is there anyway where I can structure my JSON response as:
{
"id": 1,
"BR_ID": 5,
"ClientID": 237,
"SLT_CODE": 12,
"PURPOSE": 1,
"PAMT": "100,000.00",
"TERMS": 36,
"PPMT_MODE": 4,
"PPMT_MFAC": 0,
"PPMT_MOD2": 4,
"INT_RATE": 12,
"PEN_RATE": 24,
"OL_TEMP_APP_NO": "04152020566167",
"APPLICATION_NO": null,
"ActivityListID": 1,
"DateRequested": "2020-04-15 14:22:36",
"DateApproved": null,
"ClientRemarks": "Every fight needs mending\nEvery start has an end\nLike the sunrise and the sunset\nThat's just how it is",
"Attachments":
[
{
"FilePath": "C:\\Users\\Personal\\laravel\\storage\\app\\public\\documents\\04152020566167.20200221_141512.jpg",
"AttachmentName": "04152020566167.20200221_141512.jpg",
},
{
"FilePath": "C:\\Users\\Personal\\laravel\\storage\\app\\public\\documents\\04152020566167.20200228_170949.jpg",
"AttachmentName": "04152020566167.20200228_170949.jpg"
}
]
"created_at": null,
"updated_at": null
}
This is my query:
$newLoans = Loans::
where('tblloanapp.ActivityListID', 1)
->leftJoin('tblattachments', 'tblattachments.OL_TEMP_APP_NO', '=', 'tblloanapp.OL_TEMP_APP_NO')
->get();
return response()->json($newLoans);
Model Code
class Loans extends Model
{
public function attachments(){
return $this->hasMany('App\Models\Attachments','application_id','id');
//params: model name, reference column(foreign key), column from this table
}
}
Controller
Loans::with('attachments')->where('tblloanapp.ActivityListID', 1)->get();
Hope this helps.
Let me know if in case it doesn't work

How can I get only the JSON keys without using tilde sign?

I am trying to get (via JSON Path) only key values from the JSON below but with no success.
I cannot use the tilde sign (~) because JMeter's JSON Path Extractor works under JSON PATH 4.0 and ~ is not recognized.
{
"facetCount": {
"designer": {
"4856430": 2,
"7313551": 14,
"7018102": 8,
"306838": 1,
"85146": 146,
"2654979": 11,
"221111": 4,
"180510": 40,
"3344622": 59,
"472718": 73,
"107993": 19,
"166170": 58,
"6908": 2,
"426629": 1,
"1358858": 9,
"9879178": 6,
"55006": 43,
"285396": 2,
"3355": 9,
"215501": 8,
"4968477": 4,
"11349629": 7,
"11229643": 27,
"11355128": 9,
"7093068": 3,
"11098281": 2,
"5833751": 1,
"4741301": 1,
"9198104": 21,
"991324": 4
},
"attributes": {
"135979:77": 290,
"135979:83": 27,
"136227:20": 141,
"136227:78": 670,
"135985:44": 123,
"135985:43": 669,
"135979:62": 700,
"135979:61": 1188,
"136644:176": 2,
"136331:7": 1,
"136331:8": 3,
"136641:190": 13,
"136641:191": 12,
"136061:144": 3
},
"category": {
"136103": 208,
"136105": 147,
"137322": 2,
"136389": 120,
"136215": 236,
"136214": 954,
"136216": 217,
"136217": 352,
"136218": 452,
"136219": 40,
"136480": 4,
"136220": 111,
"136221": 288,
"136222": 58,
"136223": 369,
"136224": 163,
"136986": 3,
"136307": 1125,
"136059": 10,
"136308": 956,
"136315": 984,
"136003": 574,
"136045": 267,
"136035": 1501,
"135985": 1380,
"137134": 27,
"136309": 60,
"137323": 9,
"136390": 1,
"136021": 16,
"136322": 1951,
"137166": 16,
"137317": 7,
"136005": 4,
"135983": 4019,
"136033": 1513,
"136310": 1224,
"136392": 18,
"135981": 2430,
"136031": 16,
"136326": 1312,
"136061": 79
},
"colour": {
"1": 41686,
"7": 14593,
"5": 9596,
"18": 1,
"13": 5185,
"6": 5259,
"3": 6391,
"11": 5715,
"12": 1537,
"4": 8767,
"16": 1466,
"9": 8590,
"15": 1730,
"8": 8333,
"14": 3208,
"2": 13269,
"10": 2730
},
"ninetyminutes": {
"3": 309
},
"sameday": {
"3": 1714,
"42": 254
},
"size": {
"135972:1620": 523,
"136657:2650": 1,
"136657:2850": 1
},
"location": {
"3": 2674,
"4": 7671,
"5": 35808,
"6": 2761,
"7": 11948
},
"labels": {
"1300": 2969
}
}
}
I would like to get the keys that are under facetCount element (designers, attributes, colour, etc.) and also (another JSON Path expression) get the keys that are inside these keys, such as 4856430 from designers, 135979:77 from attributes, and so on.
Could you help me, please?
Thanks in advance!
You can use JSR223 PostProcessor to get key in object JSON
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.eachWithIndex{ def node, int idx ->
log.info('Key ' + idx + '=' + node.getKey())
vars.put('key_' + idx, node.getKey())
}
And you can get key with a variable like that:
${key_0}, ${key_1}, ...
More detail check here How to extract values from json in jmeter when the keys are unkown?
JMeter's JSON test elements rely on Jayway Jsonpath which doesn't has this tilde operator for querying the keys, you will have to go for:
JSR223 PostProcessor
Groovy language
JsonSlurper
Add JSR223 PostProcessor as a child of the request which returns the above JSON and use the following code:
For direct keys (designer, attributes, etc)
def counter = 1
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.each { facet ->
vars.put('key_' + counter, facet.key)
counter++
}
vars.put('key_matchNr', counter - 1 as String)
For child keys (4856430, 135979:77, etc)
def counter = 1
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.each { child ->
child.value.keySet().each { key ->
vars.put('childKey_' + counter, key)
counter++
}
}
vars.put('childKey_matchNr', counter -1 as String)
More information: Groovy - Parsing and producing JSON