HBase table specs
DESCRIPTION ENABLED {NAME => 'hbase_2_hive_names', FAMILIES => [{NAME => 'age',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS =>
'3', COMPRESSION => true 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS
=> 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE
=> 'true'}, {NAME => 'id', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL =>
'2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false',
ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}, {NAME => 'name', DATA_BLOCK_ENCODING
=>'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '3', COMPRESSION =>
'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}]}
Create external table in Hive:
CREATE EXTERNAL TABLE hbase_hive_names(hbid INT, id INT, fn STRING, ln STRING)
PARTITIONED BY (age INT) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,id:id,name:fn,name:ln,age:age")
TBLPROPERTIES("hbase.table.name" = "hbase_2_hive_names");
Get following error
Error in metadata: java.lang.RuntimeException:
MetaException(message:org.apache.hadoop.hive.serde2.SerDeException
org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 4 elements while hbase.columns.mapping has 5 elements (counting the key if implicit)
List of HIVE columns (hbid INT, id INT, fn STRING, ln STRING) = 4 columns you are trying map into 5 qualifiers "hbase.columns.mapping" = ":key, id:id, name:fn, name:ln, age:age"
be attentive ;)
Hbase has row ,so you should add key string when you create external table like this
CREATE EXTERNAL TABLE
hbase_hive_names(key STRING,hbid INT, id INT, fn STRING, ln STRING)
PARTITIONED BY (age INT)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES
("hbase.columns.mapping" = ":key,id:id,name:fn,name:ln,age:age")
TBLPROPERTIES("hbase.table.name" = "hbase_2_hive_names");
Related
I have a table 'my_data'
Table structure
and field 'input' as type text. And I stored array value in that field as follow
array (
'msisdn' => '99999999999',
'keyword' => '',
'serviceid' => '0011001100',
'productid' => '111000111',
**'mode' => '02',**
'cli' => '0000',
'txnid' => '000000403401806110710441878004',
'startdate' => '2018-06-06 14:51:45',
'enddate' => '2018-06-12 00:00:00',
'type' => 'subscription',
'renewalon' => '2018-06-12 00:00:00',
'lastrenewalon' => '2018-06-11 13:06:52',
'fee' => 2.44,
'status' => '0',
'linkid' => '',
)
Now, how can I get the values group by 'mode' from the array value using mysql
You can chain two SUBSTRING_INDEX functions to first get the substring after 'mode', and then get the substring before ,'cli':
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(a, "'mode'", -1), ",'cli'", 1) AS group_condition
Insert the field text in the place of a.
Then you can use group_condition in the GROUP BY clause.
Huston, I have a problem;
Problem
So, I have a EAV models in my database, Entity, Attribute, Value. The relations between are:
Production hasMany Entity -> Entity belongsTo Production
Entity hasMany Attribute -> Attribute belongsTo Entity
Attribute hasMany Value -> Value belongsTo Attribute
The table values
---value (INT 3) ,
---label (VARCHAR 65),
---id (INT 11 AI)
I Can save the relations in my database, but, my application needs to return the SUM of Value.value a specific production, example:
The final user, need the consolidate of as Production by month of created. So, now, my application return is this:
(int) 0 => array(
'Production' => array(
'myconditions' => 'HereIsNotAProblem'
),
'Entity' => array(
(int) 0 => array(
'id' => '11',
'label' => 'Acompanhamento Familiar',
'production_id' => '8',
'Attribute' => array(
(int) 0 => array(
'id' => '33',
'label' => 'Nascidos vivos no m?s',
'entity_id' => '11',
'Value' => array(
(int) 0 => array(
'id' => '1',
'label' => '< 2500g',
'value' => '0',
'attribute_id' => '33'
),
)
)
)
)
I need to return:
(int) 0 => array(
'Production' => array(
'myconditions' => 'HereIsNotAProblem'
),
'Entity' => array(
(int) 0 => array(
'id' => '11',
'label' => 'Acompanhamento Familiar',
'production_id' => '8',
'Attribute' => array(
(int) 0 => array(
'id' => '33',
'label' => 'Nascidos vivos no m?s',
'entity_id' => '11',
'Value' => array(
(int) 0 => array(
'id' => '1',
'label' => '< 2500g',
'SUM' => 'TOTAL_SUM',
'attribute_id' => '33'
),
)
)
)
)
How I do this?
So when you query your data you need to left join Values to Attributes and use SUM expression. You can see similar example here, but using COUNT: https://book.cakephp.org/3.0/en/orm/query-builder.html#using-leftjoinwith
I want to create a table with a LONGBLOB field in it.
'image' => $this->binary(),
produces BLOB field in the table.
Is there any other way to produce LONGBLOB field except using raw SQL syntax for the specific field.
Below is my full code for creating the table.
$this->createTable('edition_images', [
'image_id' => $this->bigPrimaryKey()->unsigned(),
'embed_url' => $this->string()->notNull(),
'image_type' => $this->string(),
'image_md5' => $this->string(),
//'image' => $this->binary(),
'`image` longblob NULL',
'title_en' => $this->string(),
'title_bg' => $this->string(),
'title_ro' => $this->string(),
'order' => $this->bigInteger(20)->unsigned()->null(),
'edition_id' => $this->bigInteger(20)->unsigned()->notNull(),
'created_by' => $this->bigInteger(20)->unsigned()->notNull(),
'created_at' => $this->timestamp()->notNull()->defaultExpression('CURRENT_TIMESTAMP'),
'updated_by' => $this->bigInteger(20)->unsigned()->null(),
'updated_at' => $this->timestamp()->null()->defaultValue(null)->append('ON UPDATE CURRENT_TIMESTAMP'),
'deleted_by' => $this->bigInteger(20)->unsigned()->null(),
'deleted_at' => $this->timestamp()->null(),
'deleted' => $this->integer(1),
]);
You can pass the exact column type as text:
'image' => 'LONGBLOB'
You should be able to specify the length as a parameter in the binary method. Since the longblob is 4GB you have to specify this in bytes:
'image' => $this->binary(4294967295),
However, $length is being ignored. The code
$this->db->createCommand()->createTable("test_blob", [
"id" => $this->integer(),
"datum" => $this->binary(429496729),
"txt" => $this->string()
])->getSql();
returns the following SQL:
CREATE TABLE `test_blob` (
`id` int(11),
`datum` blob,
`txt` varchar(255)
);
I've added an issue on Github
How can I insert array data to mysql using code igniter ??
I tried example from CI documentation like this
$data = array(
'id_kls' => 'id_kls',
'fk__id_kls' => 'fk__id_kls',
'id_reg_pd' => 'id_reg_pd',
'nm_pd' => 'nm_pd',
'asal_data' => 'asal_data',
'nilai_angka' => 'nilai_angka',
'nilai_huruf' => 'nilai_huruf',
'nilai_indeks' => 'nilai_indeks',
);
$this->db->insert('master_nilai', $data);
// Executes: REPLACE INTO mytable (title, name, date) VALUES ('My title', 'My name', 'My date')
But not working ..
I have array data like this
Array
(
[error_code] => 0
[error_desc] =>
[result] => Array
(
[0] => Array
(
[id_kls] => f77294f7-2a5a-4876-860b-824d227d5b19
[fk__id_kls] => 02
[id_reg_pd] => 001be76b-4e58-4cea-96cf-fee2d8e0abdc
[nm_pd] => SUYATNO
[asal_data] => 9
[nilai_angka] =>
[nilai_huruf] => B
[nilai_indeks] => 3.00
)
Make sure you've enable config/autoload.php/$autoload['libraries'] = array('database');
and config/database.php/your hostname,username,dbname!
I'm using cakePHP 2.6.4.
I have two tables AgentChat and AgentChatMember.
AgentChat hasMany AgentChatMember,
AgentChatMember belongsTo AgentChat.
AgentChat have: id, team_id, agent_chat_member_count fields.
AgentChatMember have: id, agent_chat_id, user_id fields.
I want to find AgentChat by chat_team_id and agent_chat_member_count,
and AgentChatMember as contain with user_id conditions.
$options['contain'] = array('AgentChatMember');
$options['conditions']['AgentChat.agent_chat_member_count'] = count($user_ids);
$options['conditions']['AgentChat.chat_team_id'] = $chat_team_id;
$res = $this->AgentChat->find('first', $options);
and $res is:
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
(int) 0 => array(
'id' => '1',
'agent_chat_id' => '1',
'user_id' => '26',
'created' => '2015-05-06 09:52:31'
),
(int) 1 => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '21',
'created' => '2015-05-06 09:52:31'
)
)
By this i get what i want except fact i cant set user_id condition to AgentChatMember like this: $options['conditions']['AgentChatMember.user_id'] = $user_ids;
When i'm using joins instead of contain i can set user_id conditions, but i get result like this:
(int) 0 => array(
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '21',
'created' => '2015-05-06 09:52:31'
)
),
(int) 1 => array(
'AgentChat' => array(
'id' => '1',
'agent_message_count' => '0',
'agent_chat_member_count' => '2',
'chat_team_id' => '14',
'created' => '2015-05-06 09:52:31',
'updated' => '2015-05-06 09:52:31'
),
'AgentChatMember' => array(
'id' => '2',
'agent_chat_id' => '1',
'user_id' => '26',
'created' => '2015-05-06 09:52:31'
)
),
...
Its possible to get multiple records join into one table or contain with conditions on it?
How can i achieve that?
You can pass conditions to the contain:-
$res = $this->AgentChat->find(
'first',
array(
'contain' => array(
'AgentChatMember' => array(
'conditions' => array(
'AgentChatMember.user_id' => $userIds
)
)
),
'conditions' => array(
'AgentChat.agent_chat_member_count' => count($userIds),
'AgentChat.chat_team_id' => $chatTeamId
)
)
);
Just as an aside, you should really be camel casing your variables in CakePHP. See the Coding Standards in the documentation.