I need to create dynamic views from JSON string data
create or replace view schema.vw_tablename copy grants as
SELECT
v:Duration::int Duration,
v:Connectivity::string Connectivity
...
from public.tablename
This is a kind of manual view for one of the table but i want to code in generic way so that i will pass the table name which is having JSON data and view will be created and output will be tabular format.
If you are wanting to have the view created in snowflake driven by data (as compared to using a tool to create the views client side, which we do in our company) I think you only hope will be stored procedures. In the detailed usage doc's it reminds you DDL operations commits the current transaction (which is always good to remember) but also implies that you can do DDL, which is what you are asking. Which means you should be able to write some javascript that builds the create view command you are want based on data handed to it.
There is a nice 2 part blog that handles this requirement. Similar to what is mentioned in Simeon Pilgrim's answer, the blog also uses a Stored Proc to generate the View. Albeit it does so using Snowflake SQL.
https://www.snowflake.com/blog/automating-snowflakes-semi-structured-json-data-handling/
https://www.snowflake.com/blog/automating-snowflakes-semi-structured-json-data-handling-part-2/
Related
So I'm kind of stumped.
I have a MySql project that involves a database table that is being manipulated and altered by scripts on a regular basis. This isn't so unusual, but I need to automate a script to run (after hours, when changes aren't happening) that would save the result of the following:
SHOW CREATE TABLE [table-name];
This command generates the ready-to-run script that would create the (empty) table in it's current state.
In SqlWorkbench and Navicat it displays the result of this SHOW command in a field in a result set, as if it was the result of a SELECT statement.
Ideally, I want to take into a variable in a procedure, and change the table name; adding a '-mm-dd-yyyy' to end of it, so I could show the day-to-day changes in the table schema on an active server.
However, I can't seem to be able to do that. Unlike a Select result set, I can't use it like that. I can't get it in a variable, or save it to a temporary, or physical table or anything. I even tried to return this as a value in a function, from which I got the error that a function cannot return a result set - which explains why it's displayed like one in the db clients.
I suspect that this is a security thing in MySql? If so, I can totally understand why and see the dangers exposed to a hacker, but this isn't a public-facing box at all, and I have full root/admin access to it. Hopefully somebody has already tackled this problem before.
This is on MySql 8, btw.
[Edit] After my first initial comments, I need to add; I'm not concerned about the data with this question whatsoever, but rather just these schema changes.
What I'd really -like- to do is this:
SELECT `Create Table` FROM ( SHOW CREATE TABLE carts )
But this seems to be mixing apples and oranges, as SHOW and SELECT aren't created equal, although they both seem to return the same sort of object
You cannot do it in the MySQL stored procedure language.
https://dev.mysql.com/doc/refman/8.0/en/show.html says:
Many MySQL APIs (such as PHP) enable you to treat the result returned from a SHOW statement as you would a result set from a SELECT; see Chapter 29, Connectors and APIs, or your API documentation for more information. In addition, you can work in SQL with results from queries on tables in the INFORMATION_SCHEMA database, which you cannot easily do with results from SHOW statements. See Chapter 26, INFORMATION_SCHEMA Tables.
What is absent from this paragraph is any mention of treating the results of SHOW commands like the results of SELECT queries in other contexts. There is no support for setting a variable to the result of a SHOW command, or using INTO, or running SHOW in a subquery.
So you can capture the result returned by a SHOW command in a client programming language (Java, Python, PHP, etc.), and I suggest you do this.
In theory, all the information used by SHOW CREATE TABLE is accessible in the INFORMATION_SCHEMA tables (mostly TABLES and COLUMNS), but formatting a complete CREATE TABLE statement is a non-trivial exercise, and I wouldn't attempt it. For one thing, there are new features in every release of MySQL, e.g. new data types and table options, etc. So even if you could come up with the right query to produce this output, in a couple of years it would be out of date and it would be a thankless code maintenance chore to update it.
The closest solution I can think of, in pure MySQL, is to regularly clone the table structure (no data), like so:
CREATE TABLE backup_20220618 LIKE my_table;
As far as I know, to get your hands on the full explicit CREATE TABLE statement, as a string, would require the use of an external tool like mysqldump which was designed specifically for that purpose.
I am trying to match two MySQL Queries (for now, the target is "Create VIEW") to analyze if the result of execution would result in the same effect to Database.
The source of the queries is not the same, making the syntax across the queries inconsistent.
To further simplify the question, let me add more details:
Let's say there is an already existing View in the database.
This View was created using a Create VIEW ... SQL statement.
There is a possibility that the Create VIEW ... statement get's updated, hence to reflect the changes in the database currently this statement is executed at the time of migration.
But, I want to avoid this situation, if the statement Create VIEW ... will result in the same structure as of the existing View in the database, I want to avoid executing it.
To generate the CREATE VIEW from database I am using SHOW CREATE VIEW... (comparing this with the query originally used to create the VIEW).
The primary restriction is I need to make this decision only at the time of migration and cannot presume any conclusions (say, using git diff or commit history...).
I have already done some search to look for a solution for this:
Found no direct solution for this problem (like a SQL engine to which I can feed both queries and know if the result would be the same).
Decided to Parse the queries and to achieve that ended up looking into ANTLR (also used by MYSQL WorkBench)
ANTLR's approach looks promising but, this will require an extensive rule-based parsing and creating a query match program from scratch.
I realized that just parsing queries is not enough, I have to create my own POJOs to store the atomic lexers from queries and then compare the queries based on some rules.
Even if I could find predefined POJOs, that would allow to quickly create a solution for this problem.
Normally successful execution of
sp_help [object_name]
in SQL Server returns a total of 7 output windows with various results out of which normally I am interested in only 2 windows namely the one with all the column information and the one with the constraints.
Is there a way I can tell SQLserver to only display these while formulating the command?
Short answer: no, you can't do this directly because the procedure is written to return that data, and TSQL has no mechanism for accessing specific result sets.
Long answer: but you can easily get the same information from other procedures or directly from the system catalog:
sp_columns, sp_helpconstraint (this is actually called by sp_help) etc.
sys.columns, sys.objects etc.
There's also the option of copying the source code from sp_help and using it as the basis of a new procedure that you create yourself, although personally I would just write it myself from scratch. If you do decide to write your own stored proc, you might find this question relevant too.
In MySQL I have a single database with one schema. In Microsoft Sql Server it is recommended to use a "Tenant View Filter" so in Microsoft Sql Server this gives me exactly what I need.
CREATE VIEW TenantEmployees AS
SELECT * FROM Employees WHERE TenantID = SUSER_SID()
What is the best way to accomplish the same in MySQL? An equivalent to the "Tenant View Filter" will work if it is performs well.
Thanks!!
The query you suggest (that I could find in MSDN) has text afterwards that explains exactly what are its assumptions. In particular, it mentions that it assumes that the "owner" of a row in the Employees table is specified in the TenantID field that is populated according to the SID of the user(s) you are partitioning for.
What that means is that you can replicate the same idea whatever way you decide to implement your data as long as you have clearly defined partitions of the data and know exactly how to associate it with the table you are creating a view for.
In particular, if you configure your system so that each partition accesses the DB with its own credentials, you could use the CURRENT_USER or USER constructs of MySQL as the IDs defining your partitions and the query to create the view would be basically the same as the one suggested in MSDN replacing SUSER_ID with CURRENT_USER.
But if you use the same user to access from all the partitions, then the suggested method is irrelevant on either database server.
Since you need to use your tenantId value to perform filtering, a table valued user defined function would be ideal, as a view normally does not accept parameters. Unfortunately, unlike many other database products MySQL doesn't support table-valued functions. However, there are MySQL hacks that claim to emulate parametrized views. These could be useful for you.
It's a little tricky in MySQL, but it can be done:
CREATE OR REPLACE VIEW {viewName}
AS
SELECT {fieldListWithoutTenantID}
FROM {tableName}
WHERE (id_tenant = SUBSTRING_INDEX(USER( ),'#',1))
I wrote up a full blog post on how I converted a single-tenant MySQL application to multi-tenant in one weekend with minimal changes. https://opensource.io/it/mysql-multi-tenant/
I am evaluating a Mondrian-Saiku solution for a client.
After analyzing their current database schemas, I realize that what constitutes as their 'fact table data' is currently being stored in XML's. The XML 's themselves are stored as blob datatypes in a MySQL table. Think of it like this: the table holds all the transactions of the company; the details of each transaction are stored in their own XML; each XML string is stored as one of the field values in a given transaction row.
This presents a slight dilemma since the Mondrian XML schema requires the explicit use of column names.
Short of having to extract and transfer the XML data to new tables (not realistic for my purposes due to the size of data and dependencies from other systems), is there any way I can work my client's existing setup for the purposes of a Mondrian-Saiku implementation?
You need to expose the data in a traditional table way. What is the database here? Can you create a database view which does some xml processing on the XML in the blob and exposes the columns?
Alternatively maybe something like composite or jboss teiid can help here. These tools allow you to expose as a standard looking table, virtually anything. It may not be quick enough though!