Does anyone know what could be causing this error? I'm trying to convert a MySQL site to Postgres so I can host on Heroku. I'm new to database syntax, and this problem has been bugging me for days.
PG::Error: ERROR: syntax error at or near "ON"
LINE 1: ...tores ("key", "value") VALUES ('traffic:hits', 0) ON DUPLICA...
^
Here's the github page for the site I'm trying to convert. https://github.com/jcs/lobsters
This is the query. I added the backslash double quotes in replace of `.
if Rails.env == "test"
Keystore.connection.execute("INSERT OR IGNORE INTO " <<
"#{Keystore.table_name} (\"key\", \"value\") VALUES " <<
"(#{q(key)}, 0)")
Keystore.connection.execute("UPDATE #{Keystore.table_name} " <<
"SET \"value\" = \"value\" + #{q(amount)} WHERE \"key\" = #{q(key)}")
else
Keystore.connection.execute("INSERT INTO #{Keystore.table_name} (" +
"\"key\", \"value\") VALUES (#{q(key)}, #{q(amount)}) ON DUPLICATE KEY " +
"UPDATE \"value\" = \"value\" + #{q(amount)}")
end
Postgres' INSERT doesn't support MySQL's variant INSERT ... ON DUPLICATE KEY UPDATE.
For alternatives see the answers to this question.
I was working on this exact code last night, here's an initial take at how I fixed it, following this answer :
def self.put(key, value)
key_column = Keystore.connection.quote_column_name("key")
value_column = Keystore.connection.quote_column_name("value")
if Keystore.connection.adapter_name == "SQLite"
Keystore.connection.execute("INSERT OR REPLACE INTO " <<
"#{Keystore.table_name} (#{key_column}, #{value_column}) VALUES " <<
"(#{q(key)}, #{q(value)})")
elsif Keystore.connection.adapter_name == "PostgreSQL"
Keystore.connection.execute("UPDATE #{Keystore.table_name} " +
"SET #{value_column} =#{q(value)} WHERE #{key_column} =#{q(key)}")
Keystore.connection.execute("INSERT INTO #{Keystore.table_name} (#{key_column}, #{value_column}) " +
"SELECT #{q(key)}, #{q(value)} " +
"WHERE NOT EXISTS (SELECT 1 FROM #{Keystore.table_name} WHERE #{key_column} = #{q(key)}) "
)
elsif Keystore.connection.adapter_name == "MySQL" || Keystore.connection.adapter_name == "Mysql2"
Keystore.connection.execute("INSERT INTO #{Keystore.table_name} (" +
"#{key_column}, #{value_column}) VALUES (#{q(key)}, #{q(value)}) ON DUPLICATE KEY " +
"UPDATE #{value_column} = #{q(value)}")
else
raise "Error: keystore requires db-specific put method."
end
true
end
There's a number of things to be fixed in the lobsters codebase beyond just this for postgres compatability - found mysql specific things in other controller files. I'm currently working on them at my own lobsters fork at https://github.com/seltzered/journaltalk - postgres fixes should be commited on there in the coming day or two.
Related
I'm trying to insert JSON data into an MySQL database:
def mapClients():
for d in devices:
clientMap = d['dot11.device']['dot11.device.associated_client_map'].keys()
for item in clientMap:
clientList = kr.device_by_mac(item)
times = kr.device_summary_since()
for c in clientList:
sqlMac = c['kismet.device.base.macaddr'],
sqlType = c['kismet.device.base.type'],
sqlManuf = c['kismet.device.base.manuf'],
ktime = c['kismet.device.base.last_time'],
for t in ktime:
sqlTime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(t))
cur.execute("INSERT INTO devices(apID,mac,type,manuf,last_seen) VALUES(1,'" + str(sqlMac) + "','" + str(sqlType) + "','" + str(sqlManuf) + "','" + sqlTime + "');")
conn.commit()
mapClients()
This returns the following error:
pymysql.err.ProgrammingError: (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'VALUES(1,'(u'58:E2:8F:CF:20:B3',)','(u'Wi-Fi Client',)','(u'Apple',)','20-10-201' at line 1")
I can see from the error that the various values are being suffixed with a 'u'. I understand through a lot of searching and learning that (I think) this means the data is unicode.
What I want to do is find a way of converting/decoding the data so the INSERT statements work. Some of the variables are tuples, some strings. Any help much appreciated.
You are inserting tuples, not strings; remove the trailing commas:
sqlMac = c['kismet.device.base.macaddr']
sqlType = c['kismet.device.base.type']
sqlManuf = c['kismet.device.base.manuf']
ktime = c['kismet.device.base.last_time']
sqlTime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(ktime))
It’s the trailing comma that turns those expressions into tuples, and str() on a tuple gives you the container Unicode string as the u'....' representation that then clashes with the ' quoting you are adding.
Note that removes the need to loop over ktime!
Next, you really want to use SQL parameters, not string concatenation. Use placeholders instead of '" + str(...) + "', and leave handling of quoting to the database adapter:
cur.execute("""
INSERT INTO devices (apID, mac, type, manuf, last_seen)
VALUES (1, %s, %s, %s, %s)
""", (sqlMac, sqlType, sqlManuf, sqlTime))
The %s are the placeholders; depending on your exact MySQL Python library, you may need to use ? questionmarks instead.
Not only would this let you avoid having to think about quoting, it also removes a serious security issue: the JSON you load could contain a SQL injection attack and SQL parameters are the best way to neutralise that attack vector.
For the error message you posted, you have forgotten to place the closing parentheses before the VALUES in your SQL Query. The query should be like:
cur.execute("INSERT INTO devices(apID,mac,type,manuf,last_seen) VALUES(1,'" + str(sqlMac) + "','" + str(sqlType) + "','" + str(sqlManuf) + "','" + str(sqlTime) + "');")
Following SQL statement is creating an error with message :
"Message: Fatal error encountered during command execution."
"Inner exception: Parameter '#LastUserID' must be defined."
If I directly use LAST_INSERT_ID() instead of LastUserID, it always returns zero (hence fails at second insert) when executed like this.
I don't see my syntax is different than in mySQL document.
Could some one help me ?
string Query = #"INSERT INTO login (" +
"LOGIN_EMAIL," +
"LOGIN_PASSWORD," +
"LOGIN_SALT," +
"LOGIN_LAST_LOGIN_DATE," +
// "LOGIN_LAST_LOGIN_LOCATION," +
"LOGIN_ACCOUNT_STATUS," +
"LOGIN_LOGIN_ATTEMPTS," +
"LOGIN_CREATED_DATE) " +
"VALUES (" +
"#Parameter2," +
"#Parameter3," +
"#Parameter4," +
"#Parameter5," +
// "#Parameter6," +
"#Parameter6," +
"#Parameter7," +
"#Parameter8); " +
"SET #LastUserID = LAST_INSERT_ID(); " +
"INSERT INTO user_role (" +
"USER_ROLE_USER_ID," +
"USER_ROLE_ROLE," +
"USER_ROLE_STATUS," +
"USER_ROLE_CREATED_DATE) " +
"SELECT " +
"#LastUserID," +
"#Parameter9," +
"#Parameter10," +
"#Parameter11 " +
"FROM dual WHERE NOT EXISTS (SELECT USER_ROLE_USER_ID FROM user_role " +
"WHERE USER_ROLE_USER_ID = #LastUserID AND USER_ROLE_ROLE = #Parameter9)";
MySqlCommand oCommand = new MySqlCommand(Query, oMySQLConnecion);
oCommand.Transaction = tr;
Create a procedure in which you first do your insert, cache the last inserted id, do the other insert and let it print out your parameters with a bool if your last insert worked or not. That way you can debug it properly.
In general you should avoid concatinating strings to generate sql-commands or you might get troubles with parameters containing unexpected characters or be hit by a injection.
Simple fix : Replace "$LastUserID" with "$'LastUserID'". The ephostophy makes the difference.
I'm trying to use Groovy withBatch function and it's really slow (15 sec). I've tried with different batch sizes (10, 400 ...) and it always take a lot of time doing each batch.
It's the second query that I write with it and there are both slow.
Here's my code. Is there a bug in it or am I using it the wrong way ?
static updateCSProducts(def conn, def webProductRows){
conn.withBatch(400, """
UPDATE cscart_products p
SET usergroup_ids=:usergroup_ids,
b2b_group_ids=:b2b_group_ids,
b2b_desc_hide=:b2b_desc_hide
WHERE product_code = :product_code
OR product_id = (SELECT product_id FROM cscart_product_options_inventory WHERE product_code = :product_code)
""") { ps ->
webProductRows.each({row ->
ProductType type = ProductType.fromCode(row.type)
String userGroupIds = type.getProductAvailabilityUserGroup().collect{it.getId()}.join(",")
String b2bGroupIds = type.getB2bUserGroup().collect{it.getId()}.join(",")
boolean b2bDescHide = !type.getB2bUserGroup().isEmpty()
println row.id + " " + userGroupIds + " " + b2bGroupIds + " " + b2bDescHide
ps.addBatch(product_code:row.id, usergroup_ids:userGroupIds, b2b_group_ids:b2bGroupIds, b2b_desc_hide:b2bDescHide)
})
}
}
I'm using MySql as Database. When I'm looking at SQL connections, I don't really see any connection running a query while I'm waiting for the next batch.
EDIT:
I've removed the queries and it still very slow.
Heres the updated version:
conn.withBatch(400, """
UPDATE cscart_products p
SET usergroup_ids=:usergroup_ids,
b2b_group_ids=:b2b_group_ids,
b2b_desc_hide=:b2b_desc_hide
WHERE p.product_code = :product_code
""") { ps ->
webProductRows.each({row ->
ProductType type = ProductType.fromCode(row.type)
String userGroupIds = type.getProductAvailabilityUserGroup().collect{it.getId()}.join(",")
String b2bGroupIds = type.getB2bUserGroup().collect{it.getId()}.join(",")
String b2bDescHide = !type.getB2bUserGroup().isEmpty() ? 'Y' : 'N'
println row.id + " " + userGroupIds + " " + b2bGroupIds + " " + b2bDescHide
ps.addBatch(product_code:row.id, usergroup_ids:userGroupIds, b2b_group_ids:b2bGroupIds, b2b_desc_hide:b2bDescHide)
})
}
You're running tons of queries on each and every update. You'd be better off retrieving a list of data and then looping over that. It's not withBatch that's the bottleneck, it's your implementation.
So i am trying to make a function out of many functions (i believe it is called recursion, read a post about it earlier on this forum)
When i try to make a number of things into 1 big function so i can call upon it later it doesn't seem to be working but when i take away the "func _hello()" and the "endfunc" from the end, everything seems to be working fine. Can someone please explain this to me. I know the problem is occurring because of the the "Conversion" function but i can't seem to understand why this is happening. Please help, language used here is AutoIt
;;;****Program adds spaces *****
;;;***** the input variable here is $New*****
Global $final
Global $Hexadec
Func _hello()
$DataToBeDecrypted = "55fdaf fdafd"
$2space = $DataToBeDecrypted
$New = $2space
$AddingSpace = StringSplit($New, "")
$Final = ""
If Conversion($AddingSpace[0]) Then
For $Spacing = 1 to $AddingSpace[0] Step 2
$Final = $Final & $AddingSpace[$Spacing] & $AddingSpace[$Spacing+1] & " "
Next
MsgBox(0, "Adding space to the message so it can be converted back to Hex", $Final)
Else
MsgBox(0, "Result", "String does not contain an even number of characters.")
EndIf
Func Conversion($Hexadec)
Return Mod($Hexadec, 2) = 0
EndFunc
;;;***The final value is stored in the $final variable****
;***** Hexadecimals to ASCII*****
;;***Input variable is $HexadecimaltoASCII2******
$HexadecimalToASCII2 =$final
$HexadecimalsToASCII = ChrH($HexadecimalToASCII2)
$Ascii2Hex = Sub($HexadecimalsToASCII)
$v5ar = Chr($HexadecimalsToASCII);char
MsgBox(0,"Hex to ASCII",$HexadecimalsToASCII)
Func ChrH($v8)
Local $v5=""
$A1 = StringSplit($v8, " ")
For $count = 1 To $A1[0]
$v5 &= Chr(Dec($A1[$count]))
Next
Return $v5
endFunc
Func Sub($v8)
Local $v9=""
For $count = 1 To StringLen($v8)
If StringLen(Hex(Asc(StringMid($v8, $count, 1)),2)) = 1 Then
$v9 &= "0" & Hex(Asc(StringMid($v8, $count, 1)))
Else
$v9 &= Hex(Asc(StringMid($v8, $count, 1)),2)
EndIf
If $count <> StringLen($v8) Then $v9 &= " "
Next
Return $v9
endFunc
;*****HEXADECIMAL to ASCII*****
EndFunc
It seems like you never call your Hello() function. To execute a function you have to call it.
Try adding Hello() at the top of the file and it should work.
so today I just started learning this language, and I'm trying to make my script with parts of example scripts from https://www.autoitscript.com/autoit3/docs/functions.
I'm trying to run a mysql query and I'm getting "Variable Undeclared" error even if it declared as a Global one (atleast i think i declared it as one..)
Basically what my script should do is make an ID for every machine in my LAN store it into a txt file in AppData and then insert it into a db.
#include <MsgBoxConstants.au3>
#include <FileConstants.au3>
#include "EzMySql.au3"
#include <Array.au3>
Example()
Func Example()
$id = Random(1, 1000, 1);Numar random de la 1 la 100
Local Const $sFilePath = #AppDataDir & "\id.txt" ; Selectare %appdata% si id.txt
Local $iFileExists = FileExists($sFilePath)
If $iFileExists Then
Else
; Create a temporary file to write data to.
If Not FileCreate($sFilePath, $ID & #CRLF) Then Return MsgBox($MB_SYSTEMMODAL, "", "O eroare s-a produs in timp ce se scria fila temporara")
; Open the file for writing (append to the end of a file) and store the handle to a variable.
Global $hFileOpen = FileOpen($sFilePath, $FO_APPEND)
If $hFileOpen = -1 Then
MsgBox($MB_SYSTEMMODAL, "", "O eroare s-a produs in timp ce se citea fila.")
Return False
EndIf
; Read the contents of the file using the handle returned by FileOpen.
Global $sFileRead = FileRead($hFileOpen)
; Close the handle returned by FileOpen.
FileClose($hFileOpen)
EndIf
EndFunc
; Create a file.
Func FileCreate($sFilePath, $sString)
Local $bReturn = True ; Create a variable to store a boolean value.
If FileExists($sFilePath) = 0 Then $bReturn = FileWrite($sFilePath, $sString) = 1 ; If FileWrite returned 1 this will be True otherwise False.
Return $bReturn ; Return the boolean value of either True of False, depending on the return value of FileWrite.
EndFunc ;==>FileCreate
$name=#ComputerName
If Not _EzMySql_Startup() Then
MsgBox(0, "Error Starting MySql", "Error: "& #error & #CR & "Error string: " & _EzMySql_ErrMsg())
Exit
EndIf
If Not _EzMySql_Open("127.0.0.1", "root", "", "vrgaming", "3306") Then
MsgBox(0, "Error opening Database", "Error: "& #error & #CR & "Error string: " & _EzMySql_ErrMsg())
Exit
EndIf
If Not _EzMySql_Exec("INSERT INTO `lan` (id, nume) VALUES ('"& $sFileRead &"', '"& $name &"')") Then
MsgBox(0, "Error opening Database", "Error: "& #error & #CR & "Error string: " & _EzMySql_ErrMsg())
Exit
EndIf
_EzMySql_Close()
_EzMySql_ShutDown()
Exit
There are a number of logic issues with the code, that makes it a little tricky to follow.
The error in this case is almost certainly caused because "id.txt" exists, and so $iFileExists evaluates to true, and the global variable is never created.
Then you are creating a new "id.txt", for writing, before reading all the data, so $sFileRead will always be blank and FileRead will set #error as you are trying to read from a file opened for writing.
Now, I think you want the $sFileRead to be whatever the contents of id.txt is, and id.txt should be created with a new random id if it doesn't exist. If so then you need to move the code that opens and reads the file outside of the Else...Endif block (in addition to changing FileOpen to use $FO_READ).
This answer doesn't address the questionable programming practice and code structure.