I see the tables in hbase when I say show tables;
jdbc:drill:zk=<zkhost>> USE hbase;
+-------+------------------------------------+
| ok | summary |
+-------+------------------------------------+
| true | Default schema changed to [hbase] |
+-------+------------------------------------+
ns is the namespace
table1,2,3 are tables inside the namespace.
: jdbc:drill:zk=<zkhost>> show tables;
+---------------+----------------------------------+
| TABLE_SCHEMA | TABLE_NAME |
+---------------+----------------------------------+
| hbase | ATLAS_ENTITY_AUDIT_EVENTS |
| hbase | ns:table1 |
| hbase | ns:table2 |
| hbase | ns:table3 |
| hbase | atlas_janus |
+---------------+----------------------------------+
6 rows selected (30.111 seconds)
But when I try to query the table, drill doesnt identify the tables inside the namespace.
0: jdbc:drill:zk=vginthw496> select * from hbase.table2 limit 10;
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 19: Object 'table2' not found within 'hbase'
[Error Id: 02353729-39b2-4894-a21f-b6662a2a7a3c on <hostname>:31010] (state=,code=0)
How do I query hbase tables inside a namespace using apache drill?
using ``, like this:
select * from `namespace:table` limit 1;
Related
here is a simplified version of my question:
csv file:
| id | colA | colB |
| 1 | | 1.5 |
| 2 | | 2.2 |
| 3 | 3.3 | 3.5 |
...
I am trying to perform a "load data local infile" operation, but I keep getting warnings on "colA"'s first two entries: because they are empty strings for DB when the DB read the file.
Is there a way I can replace them during the load data step?
Thank you so much!
Use a user variable to transform the column
LOAD DATA LOCAL INFILE "filename"
INTO TABLE tablename
(id, #colA, #colB)
SET colA = NULLIF(#colA, ''), colB = NULLIF(#colB, '')
I am using Vertica DB (and DBeaver as SQL Editor) - I am new to both tools.
I have a view with multiple columns:
someint | xyz | c | json
5 | 1542 | none | {"range":23, "rm": 51, "spx": 30}
5 | 1442 | none | {"range":24, "rm": 50, "spx": 3 }
3 | 1462 | none | {"range":24, "rm": 50, "spx": 30}
(int) | (int) | (Varchar) | (Long Varchar)
I want to create another view (or for the beginning, just be able to query it properly) of the above, but with the "json" column separated into the individual fields/columns "range", "rm" and "spx".
I imagine the output of the query / the new view to be something like the following:
someint | xyz | c | range | rm | spx
5 | 1542 | none | 23 | 51 | 30
5 | 1442 | none | 24 | 50 | 3
....
So far I have not been able to even query the "range" for example.
Hence my questions:
How can I separate the json column key-value structure into individual columns (in a query output)?
How can I transfer the desired output into a new view in Vertica?
I haven't found much help in the documentation as the procedure there is to load json text files from a drive or operate on tables, which I cannot do as I only have access to a view.
I have found a solution, so for anyone else encountering this problem:
SELECT a, xyza, cont,
MAPLOOKUP(MapJSONExtractor(json), 'range') AS range,
MAPLOOKUP(MapJSONExtractor(json), 'rm') AS rm,
MAPLOOKUP(MapJSONExtractor(json), 'spx') AS spx
FROM test;
Is it possible to find the process ID or the command that is executed for a rollback ?
More specifically :
The Information_Schema of MySQL has a specific table named processlist. This contains the details about every ongoing process in MySQL. An example of the table looks like this:
mysql> select * from processlist;
+----+------+-----------+--------------------+---------+------+------------+---------------------------+
| ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO |
+----+------+-----------+--------------------+---------+------+------------+---------------------------+
| 5 | root | localhost | information_schema | Query | 5 | User sleep | select sleep(20) |
| 4 | root | localhost | information_schema | Query | 0 | executing | select * from processlist |
+----+------+-----------+--------------------+---------+------+------------+---------------------------+
Is it possible for me to intercept a Rollback and grab its ID or INFO from this table ? The issue I have been facing is that any Rollback I execute gets completed before I am able to intercept its ID from the Table.
I'm thinking to use Redis to cache some user data snapshot(s) in order to speed up the access to that data (one of the reasons is because my MySQL table(s) suffer of lock contention) and I'm looking for the best way to import in one step a table like this(which may contain from a few record to millions of records):
mysql> select * from mytable where snapshot = 1133;
+------+--------------------------+----------------+-------------------+-----------+-----------+
| id | email | name | surname | operation | snapshot |
+------+--------------------------+----------------+-------------------+-----------+-----------+
| 2989 | example-2989#example.com | fake-name-2989 | fake-surname-2989 | 2 | 1133 |
| 2990 | example-2990#example.com | fake-name-2990 | fake-surname-2990 | 10 | 1133 |
| 2992 | example-2992#example.com | fake-name-2992 | fake-surname-2992 | 5 | 1133 |
| 2993 | example-2993#example.com | fake-name-2993 | fake-surname-2993 | 5 | 1133 |
| 2994 | example-2994#example.com | fake-name-2994 | fake-surname-2994 | 9 | 1133 |
| 2995 | example-2995#example.com | fake-name-2995 | fake-surname-2995 | 7 | 1133 |
| 2996 | example-2996#example.com | fake-name-2996 | fake-surname-2996 | 1 | 1133 |
+------+--------------------------+----------------+-------------------+-----------+-----------+
into the Redis key-value store.
I can have many "snapshots" to load into Redis, and the basic access pattern is (SQL like syntax)
select * from mytable where snapshot = ? and id = ?
these snapshots can also coming from others table, so the "global unique ID per snapshot" is the column snapshot, ex:
mysql> select * from my_other_table where snapshot = 1134;
+------+--------------------------+----------------+-------------------+-----------+-----------+
| id | email | name | surname | operation | snapshot |
+------+--------------------------+----------------+-------------------+-----------+-----------+
| 2989 | example-2989#example.com | fake-name-2989 | fake-surname-2989 | 1 | 1134 |
| 2990 | example-2990#example.com | fake-name-2990 | fake-surname-2990 | 8 | 1134 |
| 2552 | example-2552#example.com | fake-name-2552 | fake-surname-2552 | 5 | 1134 |
+------+--------------------------+----------------+-------------------+-----------+-----------+
The loaded snapshot into redis never change, they are available only for a week via TTL
There is a way to load in one step this kind of data(rows and columns) into redis combining redis-cli --pipe and HMSET?
What is the best model to use in redis in order to store/get this data (thinking at the access pattern)?
I have found the redis-cli --pipe Redis Mass Insertion (and also MySQL to Redis in One Step) but I can't figure out the best way to achieve my requirements (load from mysql in one step all rows/colums, best redis model for this) using HMSET
Thanks in advance
Cristian.
Model
To be able to query your data from Redis the same way as:
select * from mytable where snapshot = ?
select * from mytable where id = ?
You'll need the model below.
Note: select * from mytable where snapshot = ? and id = ? does not make a lot of sense here, since it's the same as select * from mytable where id = ?.
Key type and naming
[Key Type] [Key name pattern]
HASH d:{id}
ZSET d:ByInsertionDate
SET d:BySnapshot:{id}
Note: I used d: as a namespace but you may want to rename it with the name of your domain model.
Data insertion
Insert a new line from Mysql into Redis:
hmset d:2989 id 2989 email example-2989#example.com name fake-name-2989 ... snapshot 1134
zadd d:ByInsertionDate {current_timestamp} d:2989
sadd d:BySnapshot:1134 d:2989
Another example:
hmset d:2990 id 2990 email example-2990#example.com name fake-name-2990 ... snapshot 1134
zadd d:ByInsertionDate {current_timestamp} d:2990
sadd d:BySnapshot:1134 d:2990
Cron
Here is the algorithm that must be run each day or week depending on your requirements:
for key_name in redis(ZREVRANGEBYSCORE d:ByInsertionDate -inf {timestamp_one_week_ago})
// retrieve the snapshot id from d:{id}
val snapshot_id = redis(hget {key_name} snapshot)
// remove the hash (d:{id})
redis(del key_name)
// remove the hash entry from the set
redis(srem d:BySnapshot:{snapshot_id} {key_name})
// clean the zset from expired keys
redis(zremrangebyscore d:ByInsertionDate -inf {timestamp_one_week_ago})
Usage
select * from my_other_table where snapshot = 1134; will be either:
{snapshot_id} = 1134
for key_name in redis(smembers d:BySnapshot:{snapshot_id})
print(redis(hgetall {keyname}))
or write a lua script to do this directly on redis side. Finally:
select * from my_other_table where id = 2989; will be:
{id} = 2989
print(redis(hgetall d:{id}))
Import
This part is quite easy, just read the table and follow the above model. Depending on your requirements you may want to import all (or a part of) your data with an hourly/daily/weekly cron.
How do you retrieve the terminal view of a mysql select query?
Desired Result: below ; how do I get that in my result set? as opposed to just the numbers for specific columns?
+---------+----------+
| Dog_ID | Owner_ID |
+---------+----------+
| 1 | 1 |
| 2 | 1 |
| 3 | 2 |
+---------+----------+
MySQL uses that exact output in the Console. Open the MySQL Command Line Client that comes with MySQL installation and type your select like
select * from your_table;
Then copy the result out of the console output.