I have a table where we have more than 2 Billions rows and my search is always based on Index field .
No new thing that need to try is to get the filed of xml string which is stored in a column which is longtext and varchar(4000)
Both of this column stores xml files one is big xml files and another one is small xmls file.
I need to query on the nodes or filed which is inside the xml files .
for example i need to get the statusCode which is present in fist xml file (DS column 6th column) and providerT which is present in second xml file (SY column last column)
Th search will happen always based on CT_ID which is index column and then on DS SY column .
so i am searching for a query something like
select * from AD_TABLE where CT_ID ='6ththeurtyiru' and DS.statusCode='COMPLETED' AND SY.providerT='noOfNRe';
I know my query is wrong but this can work if i have json but do we have any such thing for xml in MySql?
i have done some search but the query looks very very difficult to understand specially parsing query .
Can some one help me with this please .
ID,VERSION,AN_TYPE,EN_TYPE,CT_ID,DS,OT_TYPE,DATE_TIME,GP_ID,OT_NAME,OT_ID,UR_NAME,UR_ID,PT_ID,NS,SY
6uyuitygjh82mcrzz,0,SD_CASE,MYCHECK,6ththeurtyiru
,<?xml version="1.0" encoding="UTF-8" standalone="yes"?><sCPayload><cId>suerywe7r-21cf-4c7e-8071-suerywe7r</cId><statusCode>COMPLETED</statusCode><sngState>INPROGRESS</sngState><noOfNRe>1</noOfNRe><noOfRRR>0</noOfRRR><noOfER>0</noOfER><noOfARR>1</noOfARR><providerT>WATCHLIST</providerT></sCPayload>,CASE,9/16/2020 9:45,bd7c9519-d726-4672-8599-83d21927bec5,,5f53b903-21cf-4c7e-8071-suerywe7r,System User,USER_SYSTEM,,,<?xml version="1.0" encoding="UTF-8" standalone="yes"?><screenCES><providerT>MYWATCH</providerT><noOfNRe>1</noOfNRe></screenCES>,
In MySQL8.0, you can use ExtractValue
For MariaDB (10.5) the docs are: ExtractValue
mysql> set #xml='<sCPayload><cId>suerywe7r-21cf-4c7e-8071-suerywe7r</cId><statusCode>COMPLETED</statusCode><sngState>INPROGRESS</sngState><noOfNRe>1</noOfNRe><noOfRRR>0</noOfRRR><noOfER>0</noOfER><noOfARR>1</noOfARR><providerT>WATCHLIST</providerT></sCPayload>';
Query OK, 0 rows affected (0.00 sec)
mysql> select ExtractValue(#xml,'//statusCode');
+-----------------------------------+
| ExtractValue(#xml,'//statusCode') |
+-----------------------------------+
| COMPLETED |
+-----------------------------------+
1 row in set (0.00 sec)
mysql>
You can also put the result from extract in a permanent virtual field. so you can build a index or combined index to get a fast result and not a FULL TABLE SCAN.
This field will be automaticly be updated if you change the XML
Add this field to your Table
ALTER TABLE yourTable
ADD COLUMN
`statusCode` varchar(16) AS (ExtractValue(DS,'//statusCode')) PERSISTENT;
Related
I have the following problem that I'm trying to find the best solution for. Let's say I have a view such as the following:
CREATE VIEW myView AS (
SELECT
country_code,
other_column,
COUNT(1) as cnt
FROM mytable
JOIN otherDatabase.otherTable ON (id)
GROUP BY 1,2 ORDER BY 1 LIMIT 1
)
What would be the fastest way to resolve the field names and types of the view? For example, on the above I am looking to get something along the lines of:
{
country_code: VARCHAR,
other_column: BOOL,
cnt: INT
}
The first approach is just to run the query (with a limit, if necessary) and then get the types of the result-set from the driver. The downside of this is what if the query takes 50 minutes to resolve?
The second approach I thought of is to 'follow' the columns to get their types and then do some parsing to resolve any expressions/literals/etc. This would involve a lot of code but would be orders of magnitude faster than the above. However, the potential downside of this is we may have access to the view but not have access to a table (possibly in another database on the server) that contains the column type, so it's possible we might not be able to resolve all field names.
What would be the best way to resolve the types of a view? Note I have tagged this as MySQL, but I'm also wondering if there's a more generic way to resolve types or if it's something that is non-standard and more needs to be done on a per-database basis?
Update: I believe the correct answer is just to run a DESCRIBE myView, and that would give me the column names and types without running the query?
In the current version of MySQL at least, INFORMATION_SCHEMA.COLUMNS holds metadata for views as well as base tables:
mysql> create table mytable (id serial primary key, x int);
Query OK, 0 rows affected (0.01 sec)
mysql> create view v as select * from mytable;
mysql> select column_name, data_type from information_schema.columns where table_name='v';
+-------------+-----------+
| COLUMN_NAME | DATA_TYPE |
+-------------+-----------+
| id | bigint |
| x | int |
+-------------+-----------+
A related issue...
SHOW CREATE TABLE myView;
or
SHOW CREATE VIEW myView;
will fully qualify all the columns.
(When writing a JOIN, it is wise to always qualify the column names.)
I wanted to enter data in MySql table's primary key field with respect to case sensitivity.
But default it is not considering case sensitivity for table data.
Here is my query.
mysql> select id from product where id = 'a1';
+----+
| id |
+----+
| A1 |
+----+
1 row in set (0.00 sec)
mysql> insert into product values('a1', 'SomeName', 'SomeName', 200, 10, 10);
ERROR 1062 (23000): Duplicate entry 'a1' for key 'product.PRIMARY'
Also i have tried Collation while creating table but not getting result as required.
can any one suggest which collation has to use or any other technique to make table's column domain case sensitive.
ALTER TABLE product
MODIFY COLUMN id VARCHAR(...) COLLATION ..._bin NOT NOT NULL;
Where the ... are the current column size and character set.
The only case-sensitive things I can think of are
base-64
Unix file names
But those do not seem likely as PKs. What is your use case? Most things are better off being case-insensitive.
(A Comment links to a SQL Server suggestion using ALTER DATABASE; that will not work for MySQL since that only changes the default for subsequently created tables.)
I want to create a new variable/column in my dataset that signifies the presence or lack there of a data mining query.
I thought it would be:
SELECT * INTO #variable1
FROM dataset
WHERE CONTENTS RLIKE '[[:<:]]foo[[:>:]]'
OR CONTENTS RLIKE '[[:<:]]example[[:>:]]'
However I get the error:
Operand should contain 1 column(s).
I think I somehow need to say SELECT as a binary (not as *), but not sure how to do this.
You need to do two things in order to accomplish this.
First, you must alter dataset to have a column for your result:
ALTER TABLE dataset ADD CONTENTS_FLAG tinyint(1) NOT NULL default 0.
Now, you need to update the new dataset column with the results of your query:
UPDATE dataset SET CONTENTS_FLAG = 1
WHERE CONTENTS RLIKE '[[:<:]]foo[[:>:]]'
OR CONTENTS RLIKE '[[:<:]]example[[:>:]]'
I recently encountered a problem caused by a typo in the database creation script, whereby a column in the database was created as varchar(0) instead of varchar(20).
I expected that I would have gotten an error for 0-length string field, but I didn't. What is the purpose of varchar(0) or char(0) as I wouldn't be able to store any data in this column anyway.
It's not allowed per the SQL-92 standard, but permitted in MySQL. From the MySQL manual:
MySQL permits you to create a column of type CHAR(0). This is useful primarily when you have to be compliant with old applications that depend on the existence of a column but that do not actually use its value. CHAR(0) is also quite nice when you need a column that can take only two values: A column that is defined as CHAR(0) NULL occupies only one bit and can take only the values NULL and '' (the empty string).
Just checked MySQL, it's true that it allows zero-length CHAR and VARCHAR.
Not that it can be extremely useful but I can think of a situation when you truncate a column to 0 length when you no longer need it but you don't want to break existing code that writes something there. Anything you assign to a 0-length column will be truncated and a warning issued, but warnings are not errors, they don't break anything.
As they're similar types, char and varchar, I'm going to venture to guess that the use-case of varchar(0) is the same as char(0).
From the documentation of String Types:
MySQL permits you to create a column of type CHAR(0). This is useful
primarily when you have to be compliant with old applications that
depend on the existence of a column but that do not actually use its
value. CHAR(0) is also quite nice when you need a column that can take
only two values: A column that is defined as CHAR(0) NULL occupies
only one bit and can take only the values NULL and '' (the empty
string).
It's useful in combination with a unique index for if you want to mark one specific row in your table (for instance, because it serves as a default). The unique index ensures that all other rows have to be null, so you can always retrieve the row by that column.
You can use it to store boolean values.
Look this code:
mysql> create table chartest(a char(0));
Query OK, 0 rows affected (0.26 sec)
mysql> insert into chartest value(NULL);
Query OK, 1 row affected (0.01 sec)
mysql> insert into chartest value('');
Query OK, 1 row affected (0.00 sec)
mysql> select 'true' from chartest where a is null;
+------+
| true |
+------+
| true |
+------+
1 row in set (0.00 sec)
mysql> select 'false' from chartest where a is not null;
+-------+
| false |
+-------+
| false |
+-------+
1 row in set (0.00 sec)
We can use NULL to represent true and '' (empty string) to represent false!
According to MySQL reference manual, only NULL occupies one bit.
I have a service which produce formatted text files with information from certain tables in a MYSQL db.
Currently I programmatically query the DB and dump it in a TEXT file.
I wonder if i can do the same from within the MYSQL db.
This means that when a new record is added to a specific table, MYSQL will run a piece of code that will generate that formatted text file.
You didn't tell much about your use cases, but maybe the CSV engine might be what you're looking for?
From the manual:
When you create a CSV table, the server creates a table format file in the database directory. The file begins with the table name and has an .frm extension. The storage engine also creates a data file. Its name begins with the table name and has a .CSV extension. The data file is a plain text file. When you store data into the table, the storage engine saves it into the data file in comma-separated values format.
mysql> CREATE TABLE test (i INT NOT NULL, c CHAR(10) NOT NULL)
-> ENGINE = CSV;
Query OK, 0 rows affected (0.12 sec)
mysql> INSERT INTO test VALUES(1,'record one'),(2,'record two');
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM test;
+------+------------+
| i | c |
+------+------------+
| 1 | record one |
| 2 | record two |
+------+------------+
2 rows in set (0.00 sec)
Starting with MySQL 5.1.9, creating a CSV table also creates a corresponding Metafile that stores the state of the table and the number of rows that exist in the table. The name of this file is the same as the name of the table with the extension CSM.
If you examine the test.CSV file in the database directory created by executing the preceding statements, its contents should look like this:
"1","record one"
"2","record two"
This format can be read, and even written, by spreadsheet applications such as Microsoft Excel or StarOffice Calc.