I'm working with read/write through the cache via Apache Ignite and encountered the following problem:
In lastest versions of postgres there is special json type of data and jsonb add-on to work with json in database.
In Apache Ignite this functions don't implemented, how I know. Moreover, when Trying to do this part, I found it possible to read json from the database as PGobject, but there is no way to add jsonb processing to the built-in SQL query parser.
For example, I'm trying send next query:
SELECT jdata->>'tag1' FROM jsontest;
And get exception:
Syntax error in SQL statement "SELECT JDATA-[*]>>'tag1' FROM JSONTEST; "; SQL statement:
SELECT jdata->>'tag1' FROM jsontest; [42000-195]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.message.DbException.getSyntaxError(DbException.java:191)
at org.h2.command.Parser.getSyntaxError(Parser.java:533)
at org.h2.command.Parser.getSpecialType(Parser.java:3842)
at org.h2.command.Parser.read(Parser.java:3352)
at org.h2.command.Parser.readIf(Parser.java:3259)
at org.h2.command.Parser.readSum(Parser.java:2375)
at org.h2.command.Parser.readConcat(Parser.java:2341)
at org.h2.command.Parser.readCondition(Parser.java:2172)
at org.h2.command.Parser.readAnd(Parser.java:2144)
at org.h2.command.Parser.readExpression(Parser.java:2136)
at org.h2.command.Parser.parseSelectSimpleSelectPart(Parser.java:2047)
at org.h2.command.Parser.parseSelectSimple(Parser.java:2079)
at org.h2.command.Parser.parseSelectSub(Parser.java:1934)
at org.h2.command.Parser.parseSelectUnion(Parser.java:1749)
at org.h2.command.Parser.parseSelect(Parser.java:1737)
at org.h2.command.Parser.parsePrepared(Parser.java:448)
at org.h2.command.Parser.parse(Parser.java:320)
at org.h2.command.Parser.parse(Parser.java:296)
at org.h2.command.Parser.prepareCommand(Parser.java:257)
at org.h2.engine.Session.prepareLocal(Session.java:573)
at org.h2.engine.Session.prepareCommand(Session.java:514)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1204)
at org.h2.jdbc.JdbcPreparedStatement.(JdbcPreparedStatement.java:73)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:288)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:402)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryDistributedSqlFields(IgniteH2Indexing.java:1365)
... 9 more
And when I'm trying to research structure of Ignite or H2 databse engine, there is no way to add processing of such queries.
So, maybe someone has met this or a similar problem and can advise how to solve it?
Related
I'm getting the following error message when trying to reflect any of my SQL views:
sqlalchemy/dialects/mysql/reflection.py", line 306, in _describe_to_create
buffer.append(" ".join(line))
TypeError: sequence item 2: expected str instance, bytes found
I have tried using both the autoload_with and autoload=True options in my select query constructor to no avail.
I have the appropriate permissions on my view. My query is pretty simple:
company_country = Table('company_country', metadata, autoload_with=engine)
query = select(company_country.c.country)
return query
I've tried the inspect utility and it does not list my SQL view, nor does the reflecting all tables described below the views section on this page: https://docs.sqlalchemy.org/en/14/core/reflection.html#reflecting-views
I'm using version SQLAlchemy->1.4.32, Python 3.x and mySQL 8.0.28 on Mac if that's any help
I should add that I can query my SQL views using the text() constructor but it would be far more preferable to use select() if possible.
Any tips appreciated
I was using the mysql-connector client for interop with other code, but after switching to the mysqlclient, I was able to reflect the views.
https://docs.sqlalchemy.org/en/14/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqldb
So I have my location column using Point data type, I'm using Apollo Server and Prisma, and when I use "npx prisma db pull" generates this data type because is not currently supported on Prisma (generated script)
so I say "Ok, I'm using string and I manage how to insert this data type" so I changed to this script, surprise! didn't work enter image description here, try to find any approach to handling MySql Point data type in Prisma but no info at soever, I really appreciate any ideas
You cannot convert it to String and use it as it isn't supported yet. You need to leave it as unsupported and you can only add data via raw queries.
For now, only adding data is supported. You cannot query for it using PrismaClient.
We can query data using Prisma Client, via raw queries as SELECT id, ST_AsText(geom) as geom from training_data where geom has dataType geometry for using Unsupported("geometry").
I have some implementation that is best served by pickling a pandas dataframe and storing it in a DB.
This works fine if the database is sqlite but fails with a load error when it is MySQL
I have found other people with similar issues on stackoverflow and google but it seems that everybodys solution is to use sql to store the dataframe.
As a last resort I would go down that route but it would be a shame for this use case to do that.
Anybody got a solution to get the same behaviour from mysql as sqlite here?
I simply dump the dataframe with
pickledframe = pickle.dumps(frame)
and store pickledframe as a BinaryField
pickledframe = models.BinaryField(null=True)
I load it in with
unpickled = pickle.loads(pickledframe)
with sqlite it works fine, with mysql I get
Exception Type: UnpicklingError
Exception Value: invalid load key, ','.
upon trying to load it.
Thanks
I'm using a MySql database and was trying to find a MySQL alternative to tedious.js (a SQL server parameterised query builder).I'm using Node.js for my backend.
I read that the .raw() command from knex.js is susceptible to sql injection, if not used with bindings.
But are the other commands and knex.js as a whole safe to use to prevent sql injection? Or am I barking up the wrong tree?
Read carefully from knex documentation how to pass values to knex raw (http://knexjs.org/#Raw).
If you are passing values as parameter binding to raw like:
knex.raw('select * from foo where id = ?', [1])
In that case parameters and query string are passed separately to database driver protecting query from SQL injection.
Other query builder methods always uses binding format internally so they are safe too.
To see how certain query is passed to database driver one can do:
knex('foo').where('id', 1).toSQL().toNative()
Which will output SQL string and bindings that are given to driver for running the query (https://runkit.com/embed/2yhqebv6pte6).
Biggest mistake that one can do with knex raw queries is to use javascript template string and interpolate variables directly to SQL string format like:
knex.raw(`select * from foo where id = ${id}`) // NEVER DO THIS
One thing to note is that knex table/identifier names cannot be passed as bindings to driver, so with those one should be extra careful to not read table / column names from user and use them without properly validating them first.
Edit:
By saying that identifier names cannot be passed as bindings I mean that when one is using ?? knex -binding for identifier name, that will be rendered as part of SQL string when passed to the database driver.
I'm trying to index a particular value from stored DB2 nosql, for that I'm using json_val but I need to invoke UDF SYSTOOLS.BSON2JSON to the json_val and need to point the table from where I required data.
SELECT SYSTOOLS.BSON2JSON(CUSTOMERDATA)
FROM JSONPO
WHERE YEAR(JSON_VAL(CUSTOMERDATA,'PO.#orderDate', 'd')) = 2017;
The above query resulting in sql error -440 so I need to invoke the json_val. How would I invoke it? How would I write stored procedure for it?
In the year 2017, it should be called as SYSTOOLS.JSON_VAL if your database is enabled for JSON data. I see in your query, the schema name is missing.
In latest version of db2 as of today, your query should work. For more info, you can refer db2 documentation here: https://www.ibm.com/docs/en/db2/11.5?topic=functions-json-val