Multi row insert using Knex.js - mysql

Am trying to build an multi-row insert query using Knex.js
My post request contains a variable which is formatted in the following format : [{addon_name:'sugar'},{addon_name:'milk'}]
My DB table has only one column namely addon_name
My knex query in my node application goes as follows
knex(`<table_name>`).insert(req.body.`<param_name>`))
expected op
insert into `<tablename>`(`addon_name`) values (sugar), (milk);
but the code dosn't work. Any comments ?
Error Details
{ [Error: insert into `table_name` (`0`, `1`, `10`, `11`, `12`, `13`, `14`, `15`, `16`, `17`, `18`, `19`, `2`, `20`, `21`, `22`, `23`, `24`, `25`, `26`, `27`, `28`, `29`, `3`, `30`, `31`, `32`, `33`, `34`, `35`, `36`, `37`, `38`, `39`, `4`, `40`, `41`, `5`, `6`, `7`, `8`, `9`) values ('[', '{', 'm', 'e', ':', '\'', 's', 'u', 'g', 'a', 'r', '\'', 'a', '}', ',', '{', 'a', 'd', 'd', 'o', 'n', '_', 'n', 'd', 'a', 'm', 'e', ':', '\'', 'm', 'i', 'l', 'k', '\'', 'd', '}', ']', 'o', 'n', '_', 'n', 'a') - ER_BAD_FIELD_ERROR: Unknown column '0' in 'field list']
code: 'ER_BAD_FIELD_ERROR',
errno: 1054,
sqlState: '42S22',
index: 0 }

Though this is an old question, I am replying here just for others who stumble upon this.
Knex now supports multi-row inserts like this:
knex('coords').insert([{x: 20}, {y: 30}, {x: 10, y: 20}])
outputs:
insert into `coords` (`x`, `y`) values (20, DEFAULT), (DEFAULT, 30), (10, 20)
There's also the batchInsert utility will inserts a batch of rows wrapped inside a transaction.

req.body.<param_name> is always a string. Most probably this will work for you:
knex(table_name).insert(JSON.parse(req.body.param_name)));
What you are seeing in your error is Knex treating the string as an array of chars, trying to push it to the table.
In the error, the following:
values ('[', '{', 'm', 'e', ':', '\'', 's', ...
Is actually your string being broken down: [{me:\'s....

Thanks. I changed the structure of my input in post method, to an comma separated string. That way it gets easier to parse the input and model it the way I need.
post method input : "milk,sugar"
code
//Knex accepts multi row insert in the following format [{},{}] => we need to
//model our input that way
var parsedValues = [];
try {
var arr = req.body.addons.split(',');
}catch(err){
return res.send({ "Message": "405" }); // Data not sent in proper format
}
for (var i in arr) {
parsedValues.push({addon_name: arr[i]});
}
console.log(parsedValues);
knex(`<table_name>`).insert(parsedValues).then(function (rows){
console.log(rows);
return res.send({ "Message": "777" }); // Operation Success
}).catch(function (err){
console.log(err)
return res.send({ "Message": "403" }); // PK / FK Violation
});

You can use batch insert
DB.transaction(async (t: Knex.Transaction) => {
return await t
.batchInsert("addon_name", addon_nameRecords)
.returning("id");
});

Related

BLEU - Error N-gram overlaps of lower order

I ran the code below
a = ['dog', 'in', 'plants', 'crouches', 'to', 'look', 'at', 'camera']
b = ['a', 'brown', 'dog', 'in', 'the', 'grass', ' ', ' ']
from nltk.translate.bleu_score import corpus_bleu
bleu1 = corpus_bleu(a, b, weights=(1.0, 0, 0, 0))
print(bleu1)
This is the error
The hypothesis contains 0 counts of 3-gram overlaps. Therefore the
BLEU score evaluates to 0, independently of how many N-gram overlaps
of lower order it contains. Consider using lower n-gram order or use
SmoothingFunction() warnings.warn(_msg)
Can someone tell me what is the problem here? I can not find the solution on google. Thank you.
Best,
DD
I found the solution. Basically, I need a list inside a list for list 'a'. So code below will work without error.
a = [['dog', 'in', 'plants', 'crouches', 'to', 'look', 'at', 'camera']]
b = ['a', 'brown', 'dog', 'in', 'the', 'grass', ' ', ' ']
from nltk.translate.bleu_score import corpus_bleu
bleu1 = corpus_bleu(a, b, weights=(1.0, 0, 0, 0))
print(bleu1)

List of tuples with strings not all arguments converted during string formatting

I am trying to insert hundreds of rows into a MySQL db at once. There are two types of records, unanswered calls and answered calls. I am putting all records into a list of tuples, and each record is it's own tuple, so that I can use the executemany function. I am getting a TypeError: not all arguments converted during string formatting, and I don't understand why.
answered = []
unanswered = []
insertQuery = """ INSERT INTO cdr (recno, relcause, starttime, answertime, endtime, releasecausetext, releasecausecode, 1streleasedialog,
origtrunk, callingnumber, orighost, callednumber, desthost, origcallid, origremotepayloadip, origremotepayloadport,
origlocalpayloadip, origlocalpayloadport, termtrunk, termsourcenumber, termsourcehost, termdestnumber, termdesthostname,
termcallid, termremotepayloadip, termremotepayloadport, termlocalpayloadip, termlocalpayloadport, duration, postdialdelay,
ringtime, durationms, routetableused, origtidalias, termtidalias, termpddms, reasoncause, mappedcausecode, mappedreasoncause,
reasoncausetext, origmos, termmos) VALUES ('%s'); """
for y in cdrList:
#Check to make sure record does not exist
sqlQuery = "select * from cdr where recno = %d and origcallid = %s;" % (int(y[0]), y[13])
if cursor.execute(sqlQuery):
print("Record exists")
else:
if y[7]=='NA':
unanswered.append((y[0], y[5],extractSqlDate(y[6]), 'null', extractSqlDate(y[8]), y[10], y[11], y[12], y[13], y[15], y[16], y[17], y[18], y[19], y[20], y[21], y[22], y[23], y[32], y[34], y[35], y[36], y[37], y[38], y[39], y[40], y[41], y[42], y[53], y[54], y[55], y[56], y[60], y[66], y[67], y[71], y[78], y[79], y[80], y[81], y[85], y[88]))
else:
answered.append((y[0], y[5],extractSqlDate(y[6]), extractSqlDate(y[7]), extractSqlDate(y[8]), y[10], y[11], y[12], y[13], y[15], y[16], y[17], y[18], y[19], y[20], y[21], y[22], y[23], y[32], y[34], y[35], y[36], y[37], y[38], y[39], y[40], y[41], y[42], y[53], y[54], y[55], y[56], y[60], y[66], y[67], y[71], y[78], y[79], y[80], y[81], y[85], y[88]))
try:
print(answered)
cursor.executemany(insertQuery, answered)
cursor.executemany(insertQuery, unanswered)
db.commit()
print("Record inserted successfully")
except MySQLdb.Error as e:
print(e)
I have confirmed that each element in each tuple in the list is a string:
Successfully connected to database
/PATH/20190610/20190610-0015-1750147245-1750147250.cdr
[('1750147245', '0001', '2019-06-10 00:10:50', '2019-06-10 00:10:59', '2019-06-10 00:11:13', 'Normal BYE', ' 200', 'O', '001102', '+tn', 'ip', '+tn', 'ip', '273418599_83875291#ip', 'ip', '20530', 'ip', '11944', '000020', '+tn', 'ip', 'tn', 'ip', '4121333-0-2851866068#ip', 'ip', '16840', 'ip', '11946', '13', '1', '8', '13450', '50', 'C - Peerless C6933_04 Origin', 'P - Thirdlane 6', '1150', '', '200', '', '', '0', '0')]
I found the problem. The tuple was returning strings, so the insert query was trying to insert values like this: ''value''. I removed the ' around the %s, and, based on #jonrsharpe's comment, added %s for each other value, and it worked.

How do I use expression with bulk insert using npm mysql?

This question answers 99% of what I am looking for...
How do I do a bulk insert in mySQL using node.js
var sql = "INSERT INTO Test (name, email, n) VALUES ?";
var values = [
['demian', 'demian#gmail.com', 1],
['john', 'john#gmail.com', 2],
['mark', 'mark#gmail.com', 3],
['pete', 'pete#gmail.com', 4]
];
conn.query(sql, [values], function(err) {
if (err) throw err;
conn.end();
});
If I want to pass an expression, such as NOW(), how would I do that? If I pass it in the array, it would count as a string. Since VALUES is a ? that gets populated by the array, I can't easily inject an expression. Any ideas?
Basically it seems impossible, so you should make a query string you want.
I would try this as follows.
var sql = "INSERT INTO Test (name, email, n, modified_on) VALUES ?";
var values = [
['demian', 'demian#gmail.com', 1, '::NOW()'],
['john', 'john#gmail.com', 2, '::UNIX_TIMESTAMP()'],
['mark', 'mark#gmail.com', 3, '::DATE()'],
['pete', 'pete#gmail.com', 4, '::NOW()']
];
var formattedQuery = connection.format(sql, [values]).replace(/'::(.*?)'/g, '$1');
connection.query(formattedQuery, function(err) {
});
fomattedQuery is as follows.
INSERT INTO Test (name, email, n, modified_on) VALUES ('demian', 'demian#gmail.com', 1, NOW()), ('john', 'john#gmail.com', 2, UNIX_TIMESTAMP()), ('mark', 'mark#gmail.com', 3, DATE()), ('pete', 'pete#gmail.com', 4, NOW())
I hope this helps.

Is there an equivalent to "tr" within SQL?

To remind you, tr
reads a byte stream from its standard input and writes the result to
the standard output. As arguments, it takes two sets of characters
(generally of the same length), and replaces occurrences of the
characters in the first set with the corresponding elements from the
second set. -- Wikipedia
I need to this within the context of an UPDATE statement and, ideally, without doing 26 nested REPLACE function calls.
EDIT:
For you nosy-Parkers who just have to know what I'm doing: I want to make the in-database equivalent of upsidedowntext.com. Right now, I do this:
SELECT
REVERSE(
REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(UPPER(value),
'A', 'ɐ'),
'B', 'q'),
'C', 'ɔ'),
'D', 'p'),
'E', 'ǝ'),
'F', 'ɟ'),
'G', 'b'),
'H', 'ɥ'),
'I', 'ı'),
'J', 'ظ'),
'K', 'ʞ'),
'L', 'ן'),
'M', 'ɯ'),
'N', 'u'),
'O', 'o'),
'P', 'd'),
'Q', 'b'),
'R', 'ɹ'),
'S', 's'),
'T', 'ʇ'),
'U', 'n'),
'V', 'ʌ'),
'W', 'ʍ'),
'X', 'x'),
'Y', 'ʎ'),
'Z', 'z')) from table;

Compare two strings according to defined values in a query

I have to compare two string fields containing letters but not alphabetically.
I want to compare them according to this order :
"J" "L" "M" "N" "P" "Q" "R" "S" "T" "H" "V" "W" "Y" "Z"
So if I compare H with T, H will be greater than T (unlike alphabetically)
And if I test if a value is greater than 'H' (> 'H') I will get all the entries containing the values ("V" "W" "Y" "Z") (again, unlike alphabetical order)
How can I achieve this in one SQL query?
Thanks
SELECT *
FROM yourtable
WHERE
FIELD(col, 'J', 'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'H', 'V', 'W', 'Y', 'Z') >
FIELD('H', 'J', 'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'H', 'V', 'W', 'Y', 'Z')
^ your value
Or also:
SELECT *
FROM yourtable
WHERE
LOCATE(col, 'JLMNPQRSTHVWYZ')>
LOCATE('H', 'JLMNPQRSTHVWYZ')
Please see fiddle here.
You can do
SELECT ... FROM ... ORDER BY yourletterfield='J' DESC, yourletterfield='L' DESC, yourletterfield='M' DESC, ...
The equality operator will evaluate to "1" when it's true, "0" when false, so this should give you the desired order.
There's actually a FIELD() function that will make this a bit less verbose. See this article for details.
SELECT ... FROM ... ORDER BY FIELD(yourletterfield, 'J', 'L', 'M', ...)