I have the following query that works in MYSQL
$query->whereRaw('
ST_Distance_Sphere(
point(address->>"$.longitude", address->>"$.latitude"),
point(?, ?)
) * .000621371192 < ?
', [$longitude, $latitude, $distance]);
When i deployed to heroku im getting this error
SQLSTATE[42703]: Undefined column: 7 ERROR: column "$.longitude" does not exist
LINE 3: point(address->>"$.longitude", address->>"$.latitu...
Im using postgres on heroku and i know the issue is something to do with the json operator.
What is the $.<json key> syntax called? and how do i convert this raw sql query to postgres?
In the worse case i leaning on switching my database to mysql if i cant solve this by Wednesday as it wont be a big deal
Postgres uses double quotes as delimiter for columns, so you must use single quotes to indicate strings. Laravel will take care of it
st_distance_sphere(
st_point(address->>'$.longitude',address->>'$.latitude'),st_point(?, ?)
)
Related
I'm using mysql 5.7 in two enviroments (staging and prod)
both with this same version
but when I try this query:
SELECT * FROM `EVENT_ONDEMAND_MODULE_CONTENT`
WHERE JSON_EXTRACT(EXTRA_INFO, '$.UPLOAD_ID') = 'Jh24ACh59sJ6mSSpQKXORJVscmy2iD3Dy1B0115L01hbI';
In staging that works, but in production I got this error:
SQLSTATE[22032]: <>: 3141 Invalid JSON text in argument 1 to function json_extract: "Invalid escape character in string."
{"UPLOAD_ID":"Jh24ACh59sJ6mSSpQKXORJVscmy2iD3Dy1B0115L01hbI","POLICY":"signed","FILE_NAME":"VIDEO 10s.mp4","FILE_MIME_TYPE":"video\/mp4","READY":false}
I copy the json from prod to staging and the query worked too.
Anybody knows if have something I must do to this function Json_extract work in prod??
Thanks in advance!
I am using gem rails~> 5.2 and gem mysql2 >= 0.3.13, < 0.5.
I have a model Lawer, which has an array column lawer_filed [sic].
# Model lawer.rb
serialize :lawer_field, Array
Then I created a Lawer, and I can get the lawer_field value as follows:
=> Lawer.first.lawer_field
=> ["2", "3", "5"]
Now, I want to find one Lawer with a query using lawer_field. I tried:
#lawer = Lawer.where("lawer_field && ARRAY[?]", "2")
which raised an error like this:
ActiveRecord::StatementInvalid (Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '['2']) LIMIT 11' at line 1: SELECT `lawers`.* FROM `lawers` WHERE (lawer_field && ARRAY['2']) LIMIT 11)
There is a mistake in my SQL syntax, but I don't how to fix it. Can anyone help?
MySQL, unlike PostgreSQL, does not support arrays in database. Therefore you needed to add this line:
serialize :lawer_field, Array
This means that you have a string field in your database, but whenever ActiveRecord is unpacking results returned by the database, it maps them directly to an instance of Ruby Array. What this means is that your only option to filter the results in the database is with any MySQL string comparison functions, LIKE, etc.
Your options are to either use LIKE or perform some other String functions (which will not perform well as you will be unable to use indices) or build another table, add a has_many association to it and use MySQL the way it was supposed to be used. You could also, of course, migrate to PostgreSQL, but that seems to be the most extreme option.
EDIT: you could also consider using MySQL`s JSON, which has been added recently. That depends on your version of MySQL though.
I didn't try this answer because I don't have any Rails project ready for testing, but I think the problem is in the syntax.
I think it should be something like this:
Lawer.where("lawer_field IN (?)", "2")
I am trying to convert Oracle table data into JSON files. I have three databases and the below code gives output as JSON file in one DB but the other two databases throw ORA-00907: missing right parenthesis error.
Syntactically it is correct, as it gave output in one DB. Don't understand what is going wrong.
This is in Oracle DB, How do I find out which version of Oracle is installed in those DB's and if they are 12.2 and above, Is there a way to fix this issue? All I want is to convert the output of a select statement to a json file. Thanks in advance
code:
SELECT JSON_OBJECT ( 'empid' value eid , 'name' value ename , 'add' value eaddr )
FROM abc.emp
JSON_Object is available from Oracle version 12.2 .
Run the query Select * from v$version to check your oracle version
I want to use Spark to process some data from a JDBC source. But to begin with, instead of reading original tables from JDBC, I want to run some queries on the JDBC side to filter columns and join tables, and load the query result as a table in Spark SQL.
The following syntax to load raw JDBC table works for me:
df_table1 = sqlContext.read.format('jdbc').options(
url="jdbc:mysql://foo.com:3306",
dbtable="mydb.table1",
user="me",
password="******",
driver="com.mysql.jdbc.Driver" # mysql JDBC driver 5.1.41
).load()
df_table1.show() # succeeded
According to Spark documentation (I'm using PySpark 1.6.3):
dbtable: The JDBC table that should be read. Note that anything that is valid
in a FROM clause of a SQL query can be used. For example, instead of a
full table you could also use a subquery in parentheses.
So just for experiment, I tried something simple like this:
df_table1 = sqlContext.read.format('jdbc').options(
url="jdbc:mysql://foo.com:3306",
dbtable="(SELECT * FROM mydb.table1) AS table1",
user="me",
password="******",
driver="com.mysql.jdbc.Driver"
).load() # failed
It threw the following exception:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'table1 WHERE 1=0' at line 1
I also tried a few other variations of the syntax (add / remove parentheses, remove 'as' clause, switch case, etc) without any luck. So what would be the correct syntax? Where can I find more detailed documentation for the syntax? Besides, where does this weird "WHERE 1=0" in error message come from? Thanks!
For reading data from JDBC source using sql query in Spark SQL, you can try something like this:
val df_table1 = sqlContext.read.format("jdbc").options(Map(
("url" -> "jdbc:postgresql://localhost:5432/mydb"),
("dbtable" -> "(select * from table1) as table1"),
("user" -> "me"),
("password" -> "******"),
("driver" -> "org.postgresql.Driver"))
).load()
I tried it using PostgreSQL. You can modify it according to MySQL.
table = "(SELECT id, person, manager, CAST(tdate AS CHAR) AS tdate, CAST(start AS CHAR) AS start, CAST(end AS CHAR) as end, CAST(duration AS CHAR) AS duration FROM EmployeeTimes) AS EmployeeTimes",
spark = get_spark_session()
df = spark.read.format("jdbc"). \
options(url=ip,
driver='com.mysql.jdbc.Driver',
dbtable=table,
user=username,
password=password).load()
return df
I had heaps of trouble with Spark JDBC incompatability with MYSQL timestamps. The trick is to convert all your timestamp or duration values to a string prior to having the JDBC touch them. Simply cast your values as strings and it will work.
Note: You will also have to use AS to give the query an alias for it to work.
With Spark 2.2 on Python connecting to a MySQL (5.7.19) I'm able to run the following when I use table="(SELECT * FROM a_table) AS my_table".
from pyspark.sql import SparkSession
my_spark = SparkSession \
.builder.appName("myApp") \
.config("jars", "/usr/local/spark-2.2.2-bin-hadoop2.7/jars/mysql-connector-java-5.1.45-bin.jar") \
.getOrCreate()
my_df = my_spark.read.jdbc(
url="jdbc:mysql://my_host:3306/my_db",
table="(SELECT * FROM a_table) AS my_table",
properties={'user': 'my_username', 'password': 'my_password'}
)
my_df.head(20)
I think it may be a bug in Spark SQL.
It seems that either this or this line gives you the error. Both use a Scala string interpolation to replace table with dbtable.
s"SELECT * FROM $table WHERE 1=0"
That's where you can find table1 WHERE 1=0 from the error you've faced since the above pattern would become:
SELECT * FROM (select * from table1) as table1 WHERE 1=0
which looks incorrect.
There is indeed a MySQL-specific dialect - MySQLDialect - that overrides getTableExistsQuery with its own:
override def getTableExistsQuery(table: String): String = {
s"SELECT 1 FROM $table LIMIT 1"
}
so my bet is that the other method getSchemaQuery is the source of the error. That's highly unlikely tough given you use Spark 1.6.3 while the method has #Since("2.1.0") marker.
I'd highly recommend checking out the logs of MySQL database and see what query is executed that leads to the error message.
I'm trying to insert into a remote mysql database. I am able to connect correctly and can query 'select' no problem from it. However, I cannot perform inserts into the same table that I can select from. I suspect it has something to do with my binds, but this is nearly identical to what I was using to get sqlite3 working which I think uses the same Arel to insert.
#result = #db.query("insert into lead_to_processes (case_number, style_of_case) values (?,?)", [
self.case_number.to_blob.force_encoding("UTF-8"),
self.style_of_case.to_blob.force_encoding("UTF-8")
]
)
Ultimate goal is to be able query a remote database from inside of a model and insert data into it. I've tried using Octopus and that didn't quite work because the tables will be different from the databases.
I have full permissions with this user on the database.
So following guidance from comments i changed the syntax and am getting a different error
Mysql2::Error: You have an error in your SQL syntax;
However i'm doing the query like this now
#db = Mysql2::Client.new(connectionstring)
#case_number = #db.escape(self.case_number)
#style_of_case = #db.escape(self.style_of_case)
#db.query("insert into lead_to_processes (case_number, style_of_case) VALUES
(#{#case_number}, #{#style_of_case})
Any ideas or guidance? I've also tried this with '' encapsulating the variables that i'm inserting
I guess there were some weird characters in my code so I had to force UTF-8 encoding and then removed the characters using gsub below, everything is flowing now.
Thanks for the advice
#db.escape(self.style_of_case.force_encoding("UTF-8"))
#db.escape(self.case_number.gsub(/[\xC2]/,'').gsub(/[\xA0]/,'').force_encoding("UTF-8"))
Is it possible that you are missing an end quote?
this
#db.query("insert into lead_to_processes (case_number, style_of_case) VALUES
(#{#case_number}, #{#style_of_case})
should be
#db.query("insert into lead_to_processes (case_number, style_of_case) VALUES
(#{#case_number}, #{#style_of_case}") <== notice the quote at the end.