I have the following where clause in a t-sql statement:
where a.CELL_GEOM.STIntersects(
STGeomFromText('POLYGON((-25.43623984375 44.257784519021, 21.62918984375 44.257784519021, 21.62918984375 60.752403080295, -25.43623984375 60.752403080295, -25.43623984375 44.257784519021))', 4326)) = 'true'
The line reports the error:
'STGeomFromText' is not a recognized built-in function name.
How can I use the well known text value shown in this function? Am I just using incorrect syntax with the STGeomFromText as that is the only example I have seen.
Try prefixing the name of the function with geometry:: like in the documentation?
E.g. : geometry::STGeomFromText( ...
Link to documentation
Related
I've got a confusing issue on Airflow which I don't understand.
I have a SQL scripts folder at DML/analytics/my_script.sql. The MySQL operator works perfectly in normal circumstances, but does not when I try to call it from a Python operator as follows. This is necessitated by needing to pass in XCOM values from another task:
def insert_func(**kwargs):
run_update = MySqlOperator(
sql='DML/analytics/my_script.sql',
task_id='insert_func',
mysql_conn_id="bi_mysql",
params={
"table_name": table_name,
'ts': kwargs['task_instance'].xcom_pull(key='return_value',task_ids='get_existing_data')
},
)
run_update.execute(context=kwargs['task_instance'])
with DAG("my_dag", **dag_params) as dag:
with TaskGroup(group_id='insert') as insert:
get_existing_data = PythonOperator(
task_id='get_existing_data',
python_callable=MySQLGetRecord,
op_kwargs={
'target_db_conn_id':'bi_mysql',
'target_db':'analytics',
'sql': f'SELECT invoice_date FROM analytics.{table_name} ORDER BY 1 DESC'
}
),
insert = PythonOperator(
task_id='insert',
python_callable=insert_func
)
get_existing_data >> insert_func
The error I get is: MySQLdb._exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'DML/analytics/my_script.sql' at line 1")
Clearly it is trying to run the literal string passed in the sql parameter rather than using it as a file location. Why is this happening? Again, this works if I move the run_update task to the my_dag with clause, but I need to do it this way to get the XCOM value from get_existing_data, correct...?
When you are using operator as normal (e.g to be used by Airflow) then Airflow is responsible for the whole task lifecycle. This means Airflow handles the templating, executing pre_execute(), executing execute(), executing on_faulure/retries etc...
What you did is using operator inside operator -> PythonOperator that contains MySqlOperator. In this case the inner operator (MySqlOperator) is just a regular Python class. While it's called Operator - it's is not a "real" Operator.
You are not enjoying any of the lifecycle steps as you might expect.
You might have already realised it as by your own example you specifically triggered the execute():
run_update.execute(context=kwargs['task_instance'])
Notice you didn't need to do this for the PythonOperaor.
You can see in the code base that Airflow invokes render_templates before it invokes pre_execute() and before it invokes execute().
This means that if you want the MySqlOperator to be templated you need to call the function that does the templating before you invoke the execute()
That said - I strongly encourage you - Do not use operator inside operator.
From your code I don't see reason why you can't just use MySqlOperator directly without the PythonOperaor but should there be a reason the proper way to handle it is to create a CustomMySqlOperator that handles the logic you seek. By doing so you will not have problems with using .sql files.
I am trying to capitalize the first letters in a text column and I use the following:
UPDATE Table
SET Column=UPPER(LEFT(Column,1))+LOWER(SUBSTRING(Column,2,LEN(Column)));
But I am getting this error
#1305 - FUNCTION Databasename.LEN does not exist
LEN isn't a valid name for a builtin MySQL function.
If you want to use a function named LEN, you would need to create a function of that name.
MySQL does provides a builtin CHAR_LENGTH function. But you don't even need to use that.
The SUBSTRING function doesn't require a third argument. I think this would work for your purpose, just:
SUBSTRING(Column,2)
https://dev.mysql.com/doc/refman/5.7/en/string-functions.html
As explained, your syntax is incorrect. A working solution is this:
UPDATE Table SET Column = CONCAT(UCASE(LEFT(Column, 1)), SUBSTRING(Column, 2));
Your use of '+' will not work with MySQL, even if you fix the issue with LEN.
This is the final working query which is a mix of what #spencer7593 and #gview suggested.
UPDATE Table SET Column=Concat(UPPER(LEFT(Column,1)), LOWER(SUBSTRING(Column,2)));
I have a JSONB column which holds arrays of strings, eg: ["foo", "bar"]
I want to write the jOOQ equivalent of:
SELECT * FROM sometable WHERE somecolumn ?| <mylist>
...where should be bound to a java List of string tag names.
There doesn't appear to be any direct support for ?| in jOOQ 3.8. I have looked at binding to raw sql in a condition but I'm not quite sure the syntax; it gets even worse if trying to use the ? postgres operator which conflicts with the binding expression.
UPDATE: the stacktrace with 3.8.3
I stripped this down to a minimal test. When adding a condition like this using jOOQ 3.8.3:
query.addConditions(DSL.condition("sometable.tags ?| array['sometag']"));
Produces a stacktrace like this:
Caused by: org.postgresql.util.PSQLException: No value specified for parameter 1.
at org.postgresql.core.v3.SimpleParameterList.checkAllParametersSet(SimpleParameterList.java:228)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:163)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:622)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:472)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:465)
at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:194)
at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:269)
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:348)
... 36 more
An issue related to jOOQ parsing ?| and similar operators as bind variables has been addressed in jOOQ 3.8.3: https://github.com/jOOQ/jOOQ/issues/5307
JDBC limitation
Note, in addition to the above, there is also a JDBC limitation that I have documented in a separate question. In jOOQ, you can work around this JDBC limitation by specifying:
Settings settings = new Settings().withStatementType(StatementType.STATIC_STATEMENT);
See also: http://www.jooq.org/doc/latest/manual/sql-execution/statement-type
Or, alternatively, by falling back to using the jsonb_exists_any() function instead of the ?| operator.
I have a sql query like this
select column from table where path = left('INPUTSTRING', length(path));
and trying to accomplish it in hql like this,
return session.createQuery("from Table where Path = left(:input, length(Path))").
query.setParameter("input", inputPath).
.list();
and getting an error like this
Caused by: org.hibernate.hql.ast.QuerySyntaxException: unexpected token: left near line 1
how to get this done? What is the corresponding string function in hql? Is there a solution for this using criteria query apis?
Yes, left() is not supported by the MySQLDialect. See the list of HQL supported functions on API docs.
Now you have left with 2 options.
Use session.createSQLQuery() method.
Create Your own Dialect class by extending the MySQLDialect and register the function there. This is told at hibernate forum here explained well in a blog post here.
I'm not sure if HQL does this for you , but you can use IQuery/session.CreateSQLQuery() to use a raw SQL query to populate a mapped entity. I've never used it for substrings, but have used it for aggregate functions. Check chapter 13 of the NHibernate docs and see if that does it for you. You can check the query substitution available in Nhibernate - here
Is it possible to create an SQL view in MySQL that takes an input variable say as a function argument? I have found this caption from the MySQL web site but am not sure how to use it as I am quite new to SQL functions. When I run this in the MySQL command prompt ,it gives me errors. Also I am not sure if this is even what I am looking for?
create function book_subject
returns varchar(64) as
return #book_subject;
create view thematical_books as
select title, author
from books
where subject = book_subject();
You are getting errors because the CREATE FUNCTION syntax is incorrect (gotta love those MySQL manual user comments!). The correct syntax for creating this function is as follows:
CREATE FUNCTION book_subject()
RETURNS VARCHAR(64)
RETURN #subject;
The CREATE VIEW syntax is correct.
In order to use the view, you'll need to set the variable #book_subject before you select from the view:
SET #book_subject = 'Epic Poems';
Then when you do a:
SELECT *
FROM thematical_books;
It will return the title and author of all of the books that have a subject of 'Epic Poems'
This is a trick to get around the restriction of MySQL views that "The SELECT statement [of the view] cannot refer to system or user variables." You use a function that just returns the variable, and that function gets called each time the view is used.
This is about as close as you are likely to get. There isn't an official way to get any arguments passed into a view (because how do you supply the argument when the view is referenced in the FROM clause). Using a session global variable and a function as shown is about the only way to achieve the effect. It is sneaky and bug-prone - not good attributes for clean maintainable code.